CN115272988A - Vehicle tracking method, device, equipment and medium - Google Patents

Vehicle tracking method, device, equipment and medium Download PDF

Info

Publication number
CN115272988A
CN115272988A CN202210841301.1A CN202210841301A CN115272988A CN 115272988 A CN115272988 A CN 115272988A CN 202210841301 A CN202210841301 A CN 202210841301A CN 115272988 A CN115272988 A CN 115272988A
Authority
CN
China
Prior art keywords
vehicle
image frame
target
feature vector
candidate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210841301.1A
Other languages
Chinese (zh)
Inventor
陈硕
陈金
李响
张渊佳
孟祥松
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianyi Cloud Technology Co Ltd
Original Assignee
Tianyi Cloud Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianyi Cloud Technology Co Ltd filed Critical Tianyi Cloud Technology Co Ltd
Priority to CN202210841301.1A priority Critical patent/CN115272988A/en
Publication of CN115272988A publication Critical patent/CN115272988A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the application provides a vehicle tracking method, a device, equipment and a medium, in the embodiment of the application, an electronic device determines a sub-feature vector corresponding to each region type of each vehicle in a current image frame and a region type corresponding to a region acquired by each vehicle, and the electronic device can match with a vehicle in a previous image frame based on a target region type and a target feature vector of the vehicle in the current image frame, so that the accuracy of feature matching of the vehicle is improved, and the accuracy of vehicle tracking is further improved.

Description

Vehicle tracking method, device, equipment and medium
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a vehicle tracking method, apparatus, device, and medium.
Background
The multi-target tracking algorithm is to give an image sequence, determine the position of each object in each image in the image sequence, and then determine the motion track of each object. Based on the method, the electronic equipment can realize the cross-image acquisition equipment tracking of the vehicle according to the position of the image acquisition equipment and the characteristics of each vehicle in the image frame acquired by each image acquisition equipment on the basis of a multi-target tracking algorithm.
When extracting the features of a vehicle in the prior art, each image frame acquired by an image acquisition device within a certain time period is generally acquired, for each vehicle, each feature of the vehicle in each image frame is determined, then, a weighted calculation or an average calculation is performed on each feature, and the calculated feature is determined as the feature of the vehicle. And then matching the characteristic with the characteristic of the vehicle determined in the last time period, determining a target vehicle matched with the vehicle in the last time period, and further determining the ID of the vehicle. However, since the moving speed of the vehicle is high, the relative position between the vehicle and the image capturing device changes rapidly, which causes an excessive difference in the characteristics of the vehicle in each image frame captured within a certain period of time, and further causes a decrease in the accuracy of matching the characteristics of the vehicle, and further causes a decrease in the vehicle tracking accuracy. Meanwhile, when the vehicle is tracked across the image acquisition equipment, the vehicle ID is required to be identified again by using the characteristics of the vehicle, however, the shooting angles of the plurality of image acquisition equipment are generally different, and the extracted characteristics of the vehicle have great difference, so that great error also exists when the vehicle ID is identified again, and the problem of tracking failure also occurs when the vehicle is tracked across the image acquisition equipment.
Disclosure of Invention
The application provides a vehicle tracking method, a vehicle tracking device, vehicle tracking equipment and a vehicle tracking medium, which are used for solving the problems that in the prior art, the accuracy of vehicle feature matching is reduced and the vehicle tracking accuracy is further reduced due to the fact that the feature difference of extracted vehicles of each frame is too large.
In a first aspect, an embodiment of the present application provides a vehicle tracking method, where the method includes:
acquiring a current image frame acquired by image acquisition equipment;
inputting the current image frame into a trained model, and acquiring a feature vector of each vehicle contained in the current image frame output by the model and a region category corresponding to a region acquired by each vehicle, wherein the feature vector comprises a sub-feature vector corresponding to each region category;
for each vehicle contained in the current image frame, determining a target vehicle matched with the vehicle in the last image frame according to the target feature vector and the target area category corresponding to the vehicle; and determining the ID corresponding to the target vehicle as the ID of the vehicle in the current image frame.
In a second aspect, an embodiment of the present application further provides a vehicle tracking apparatus, including:
the acquisition module is used for acquiring a current image frame acquired by the image acquisition equipment;
the feature extraction module is used for inputting the current image frame into a trained model, and acquiring a feature vector of each vehicle contained in the current image frame output by the model and an area category corresponding to an area acquired by each vehicle, wherein the feature vector comprises a sub-feature vector corresponding to each area category;
the matching module is used for determining a target vehicle matched with the vehicle in the previous image frame according to a target feature vector and a target area category corresponding to the vehicle for each vehicle contained in the current image frame; and determining the ID corresponding to the target vehicle as the ID of the vehicle in the current image frame.
In a third aspect, embodiments of the present application further provide an electronic device, which includes a processor, and the processor is configured to implement the steps of the vehicle tracking method as described above when executing the computer program stored in the memory.
In a fourth aspect, the present application further provides a computer-readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps of the vehicle tracking method as described above.
In the embodiment of the application, the electronic device acquires a current image frame acquired by an image acquisition device, inputs the current image frame into a trained model, and acquires a feature vector of each vehicle contained in the current image frame output by the model and a region category corresponding to a region acquired by each vehicle, wherein the feature vector comprises a sub-feature vector corresponding to each region category; for each vehicle contained in the current image frame, determining a target vehicle matched with the vehicle in the last image frame according to the target feature vector and the target area category corresponding to the vehicle; and determining the ID corresponding to the target vehicle as the ID of the vehicle in the current image frame. That is, in the embodiment of the present application, the electronic device determines the sub-feature vector corresponding to each area type of each vehicle in the current image frame and the area type corresponding to the area where each vehicle is collected, and the electronic device may match the vehicle in the previous image frame based on the target area type and the target feature vector of the vehicle in the current image frame, so as to improve the accuracy of feature matching of the vehicle, and further improve the accuracy of vehicle tracking.
Drawings
In order to more clearly illustrate the technical solutions of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic diagram of an alarm processing process provided in an embodiment of the present application;
fig. 2 is a schematic diagram of a current image frame acquired by an image acquisition device according to an embodiment of the present disclosure;
FIG. 3 is a schematic illustration of vehicles of various zone categories provided by embodiments of the present application;
fig. 4a is a region category of a vehicle a in a first image frame acquired by the image acquisition device 1 according to an embodiment of the present application;
fig. 4b is a region category of the vehicle a in the second image frame acquired by the image acquisition device 1 according to the embodiment of the present application;
fig. 4c is a region category of the vehicle a in a third image frame acquired by the image acquisition device 2 according to the embodiment of the present application;
fig. 4d is a region category of the vehicle a in a fourth image frame acquired by the image acquisition device 2 according to the embodiment of the present application;
FIG. 5 is a schematic diagram of a model structure provided in an embodiment of the present application;
FIG. 6 is a schematic diagram of a vehicle appearing in captured image frames of an image capture device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a risk data acquiring apparatus according to an embodiment of the present disclosure;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application clearer, the present application will be described in further detail with reference to the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In order to improve the accuracy of feature matching of a vehicle and improve the accuracy of vehicle tracking, the embodiment of the application provides a vehicle tracking method, a vehicle tracking device, vehicle tracking equipment and a vehicle tracking medium.
Example 1:
fig. 1 is a schematic diagram of a vehicle tracking process provided in an embodiment of the present application, where the process includes:
s101: and acquiring a current image frame acquired by the image acquisition equipment.
The vehicle tracking method provided by the embodiment of the application is applied to electronic equipment, and the electronic equipment can be a server, a PC (personal computer), image acquisition equipment or the like.
In the embodiment of the application, the image acquisition equipment monitors scenes and objects in a monitoring range in real time and acquires image frames. The electronic equipment acquires the current image frame acquired by the image acquisition equipment in real time and carries out vehicle tracking on the vehicle based on the current image frame.
Fig. 2 is a schematic diagram of a current image frame acquired by an image acquisition device according to an embodiment of the present disclosure, and as shown in fig. 2, the image acquisition device monitors a scene and an object in a monitoring range and acquires the image frame.
S102: and inputting the current image frame into a trained model, and acquiring a feature vector of each vehicle contained in the current image frame output by the model and an area category corresponding to an area acquired by each vehicle, wherein the feature vector comprises a sub-feature vector corresponding to each area category.
After the electronic device acquires the current image frame, the electronic device may determine each vehicle included in the current image frame, and determine a feature vector corresponding to each vehicle and a region category corresponding to a region where each vehicle is collected, so that the electronic device may perform vehicle tracking on each vehicle in the current image frame.
Specifically, when the electronic device performs vehicle tracking, in different image frames acquired by the same image acquisition device, the position of the same vehicle relative to the image acquisition device may change greatly, and therefore, in order to perform vehicle tracking on each vehicle better, in the embodiment of the present application, the electronic device may determine an area class corresponding to an area acquired by each vehicle, and when the electronic device generates a feature vector corresponding to a vehicle, the electronic device generates the feature vector based on each area class of the vehicle, that is, the feature vector of each vehicle includes a sub-feature vector corresponding to each area class. In the embodiment of the application, the area categories of the vehicle at least comprise a vehicle head, a vehicle body and a vehicle tail.
Fig. 3 is a schematic diagram of vehicles in various area categories provided by an embodiment of the application, and as shown in fig. 3, the area category corresponding to each vehicle in the first row is a vehicle head, the area category corresponding to each vehicle in the second row is a vehicle body, and the area category corresponding to each vehicle in the third row is a vehicle tail.
Specifically, in the embodiment of the present application, the electronic device inputs a current image frame into a trained model, the model identifies each vehicle included in the image frame, determines a feature vector of each vehicle in the current image frame, and outputs the feature vector and the region category corresponding to the region where each vehicle is collected.
Fig. 4a is a region category of a vehicle a in a first image frame acquired by an image acquisition device 1 according to an embodiment of the present application, and as shown in fig. 4a, a region category corresponding to a region acquired by the vehicle a is a vehicle head.
Fig. 4b is a region category of the vehicle a in the second image frame acquired by the image acquisition device 1 according to the embodiment of the present application, and as shown in fig. 4b, the region category corresponding to the region acquired by the vehicle a is a vehicle body.
Fig. 4c is a region type of the vehicle a in the third image frame captured by the image capturing device 2 according to the embodiment of the present application, and as shown in fig. 4c, a region type corresponding to a region where the vehicle a is captured is a vehicle body.
Fig. 4d is a region category of the vehicle a in the fourth image frame acquired by the image acquisition device 2 according to the embodiment of the present application, and as shown in fig. 4d, the region category corresponding to the region acquired by the vehicle a is a vehicle tail.
S103: for each vehicle contained in the current image frame, determining a target vehicle matched with the vehicle in the last image frame according to the target feature vector and the target area category corresponding to the vehicle; and determining the ID corresponding to the target vehicle as the ID of the vehicle in the current image frame.
In the embodiment of the application, for each vehicle contained in the current image frame, the electronic device determines a target vehicle matching the vehicle in the last image frame of the current image frame according to the target area category and the target feature vector of the vehicle in the current image frame, and determines the ID of the target vehicle as the ID of the vehicle in the current image frame. And the previous image frame and the current image frame are acquired by the same image acquisition equipment.
In addition, in the embodiment of the application, after determining the target vehicle matching with the vehicle, the electronic device may further verify whether the vehicle and the target vehicle are the same vehicle. Specifically, the electronic device predicts the motion state of each vehicle in the previous image frame and predicts the position of each vehicle in the current image frame by using Kalman filtering and Hungarian algorithms according to the position of each vehicle in the previous image frame, and further verifies whether the vehicle and the target vehicle are the same vehicle or not based on the prediction result. The prediction process is the prior art and is not described herein again.
In the embodiment of the application, the electronic device determines the sub-feature vector corresponding to each region type of each vehicle in the current image frame and the region type corresponding to the region where each vehicle is collected, and the electronic device can match the vehicle in the previous image frame based on the target region type and the target feature vector of the vehicle in the current image frame, so that the accuracy of feature matching of the vehicle is improved, and the accuracy of vehicle tracking is further improved.
Example 2:
in order to improve the accuracy of feature matching of vehicles, on the basis of the above embodiments, in an embodiment of the present application, the inputting the current image frame into a trained model, obtaining a feature vector of each vehicle included in the current image frame output by the model, and obtaining a region type corresponding to a region where each vehicle is collected includes:
inputting the current image frame into a first sub-model of the model, and determining a first intermediate image frame carrying position information of each vehicle;
inputting the first intermediate image frame into a second sub-model of the model, and determining a second intermediate image frame carrying a feature vector of each vehicle;
and inputting the second intermediate image frame into a third submodel of the model, determining a region category corresponding to the region of each vehicle, and outputting a feature vector and the region category corresponding to each vehicle.
In the embodiment of the application, when determining the feature vector and the region category corresponding to each vehicle in the current image frame, the model includes at least three submodels, which are respectively a first submodel for determining the position information of each vehicle included in the current image frame, a second submodel for determining the feature vector corresponding to each vehicle, and a third submodel for determining the region category of each vehicle.
Specifically, in the embodiment of the present application, the model inputs the current image frame into the first sub-model, which includes at least the heat map (heatmap), center offset (center offset), and bounding box size (bbox size). The heatmap is used for determining whether an object contained in the current image frame is a vehicle, the center offset is used for selecting each vehicle in the current image frame by adopting a rectangular shape and the like, the bbox size is used for constructing a coordinate system in the current image frame according to a pre-configured coordinate system construction method, and coordinates of preset vertexes of a rectangular frame corresponding to each vehicle and the length and the width of the rectangular frame are determined. The rectangular frame corresponding to each vehicle, the coordinates and the length and the width of the preset vertex of each rectangular frame form the position information of the vehicle, and the first sub-model outputs a first middle image frame carrying the position information of each vehicle.
The model inputs the first intermediate image frame into a second sub-model, and the second sub-model locates each vehicle in the first intermediate image frame according to the position information of each vehicle carried in the first intermediate image frame and determines the feature vector corresponding to each vehicle. The second sub-model outputs a second intermediate image frame carrying a feature vector corresponding to each vehicle.
The model inputs the second intermediate image frame into a third sub-model that determines the area class to which each vehicle corresponds. The model outputs a feature vector and a region class corresponding to each vehicle.
Fig. 5 is a schematic diagram of a model structure provided in an embodiment of the present application, and as shown in fig. 5, the model is a Deep Layer Aggregation network (DLA) model, and the model includes a first sub-model, a second sub-model, and a third sub-model. The first sub-model is a vehicle detection module, and the vehicle detection module comprises a heatmap, a center offset and a bbox size, wherein the heatmap is used for determining whether each object in the current image frame is a vehicle, the center offset is used for determining the position of each vehicle in the current image frame and framing out each vehicle, and the bbox size is used for determining the position information of a frame corresponding to each vehicle; the second sub-model is a Reid module and is used for determining a feature vector corresponding to each vehicle in the current image frame; the third sub-model is a vehicle form classification module, which is used for determining the region category corresponding to the region acquired by each vehicle.
Example 3:
in order to train the model better and enable the trained model to better determine the feature vector and the region class corresponding to each vehicle, on the basis of the foregoing embodiments, in an embodiment of the present application, a training process of the model includes:
acquiring each sample image frame stored in a training sample set, wherein each sample image frame carries initial position information and an initial area category corresponding to each vehicle;
for each sample image frame, inputting the sample image frame into a first sub-model of the model, and determining a predicted first intermediate image frame carrying predicted position information of each vehicle; inputting the predicted first intermediate image frame into a second sub-model of the model, wherein the second sub-model determines a predicted characteristic vector of each vehicle, allocates a predicted number to each vehicle according to the predicted characteristic vector of each vehicle, and determines a predicted second intermediate image frame carrying the predicted characteristic vector of each vehicle; inputting the predicted second intermediate image frame into a third submodel of the model, and determining a predicted region category corresponding to the region of each vehicle;
determining a first loss value corresponding to the first sub-model according to initial position information and predicted position information corresponding to each vehicle in each sample image frame;
determining a second loss value corresponding to the second submodel according to whether at least vehicles with the same prediction number exist in each sample image frame;
determining a third loss value corresponding to the third sub-model according to the initial region type and the predicted region type corresponding to each vehicle in each sample image frame;
and adjusting parameters of the model according to the first loss value, the second loss value and the third loss value.
In an embodiment of the present application, a training process for a model includes: and acquiring each sample image frame in the training sample set, wherein the sample image frame carries initial position information of each vehicle and an initial area category of each vehicle, and the initial position information comprises an initial vehicle frame and initial information corresponding to the initial vehicle frame. Inputting the sample image frame into the model aiming at each sample image frame, predicting the position information of the vehicle in the sample image frame by a first sub-model in the model, and outputting a predicted first intermediate image frame carrying the predicted position of the vehicle; a second sub-model in the model determines a predicted characteristic vector corresponding to each vehicle in the predicted first intermediate image frame according to the predicted first intermediate image frame, compares the predicted characteristic vectors, determines vehicles with similar predicted characteristic vectors as the same vehicle, and allocates a predicted number to each vehicle, wherein the predicted numbers of the same vehicle are the same, and the second sub-model outputs a predicted second intermediate image frame carrying the predicted characteristic vectors of the vehicles; the third submodel determines a predicted area category for each vehicle based on predicting the second intermediate image frame.
The electronic equipment determines a first loss value corresponding to the first submodel according to the initial position information and the predicted position information of each vehicle; determining a second loss value corresponding to a second submodel according to vehicles which are unlikely to have the same number in one image frame and a prediction number corresponding to each vehicle predicted by the second submodel; and determining a third loss value corresponding to the third submodel according to the initial region type and the predicted region type of each vehicle. The electronic equipment determines the sum of the first loss value, the second loss value and the third loss value, determines the sum as the total loss value of the model, and adjusts the parameters of the model according to the total loss value.
In the embodiment of the application, when determining a first loss value corresponding to a first sub-model, the electronic device determines the first sub-loss value according to whether each object in the sample image frame is a vehicle; and determining a second sub-loss value according to each piece of initial position information and the predicted position information. The electronic device determines a sum of the first sub-loss value and the second sub-loss value and determines the sum as a first loss value corresponding to the first sub-model.
Wherein, the first sub-loss value can be calculated by the following formula:
Figure BDA0003750480720000101
wherein L isheatmapRepresenting a first sub-loss value;
Figure BDA0003750480720000102
a value indicating whether each of the objects stored in advance is a vehicle, and if the object is a vehicle, the value is stored
Figure BDA0003750480720000103
Is 1, otherwise is others; m represents whether each object predicted by the first sub-model is a vehicle or not, if the object is the vehicle, the value of M is 1, otherwise, the value is other; n denotes the number of input sample image frames, and α and β are preset values.
Wherein the second sub-loss value can be calculated by using the following formula:
Figure BDA0003750480720000104
wherein L isboxA second sub-loss value is represented,
Figure BDA0003750480720000105
a predicted deviation amount, o, of the predicted vehicle frame representing the ith vehicleiAn initial offset amount of an initial vehicle frame of an i-th vehicle, wherein,
Figure BDA0003750480720000106
Figure BDA0003750480720000107
Figure BDA0003750480720000108
is the center coordinate of the ith initial vehicle frame,
Figure BDA0003750480720000109
indicates the length, s, corresponding to the ith predicted vehicle frameiIndicating the length of the ith initial vehicle frame, wherein,
Figure BDA00037504807200001010
Figure BDA00037504807200001011
for the coordinates of the upper left corner of the ith initial vehicle frame,
Figure BDA00037504807200001012
n represents the number of input sample image frames for the coordinates of the upper right corner of the ith initial vehicle frame.
Wherein the second loss value can be calculated by the following formula:
Figure BDA0003750480720000111
wherein L isidentityP (k) represents a second loss value, p (k) represents a probability of whether the prediction number of the kth vehicle overlaps with the prediction numbers of the other vehicles, p (k) is 1 if the number overlaps, p (k) is 0 if the number does not overlap, L (k) is a first preset value, and L (k) corresponding to each vehicle is the same.
Wherein the third loss value can be calculated by using the following formula:
Figure BDA0003750480720000112
wherein L iscar_shape_classesRepresents a third loss value, wherein yiAn initial area category preset for the ith vehicle,
Figure BDA0003750480720000113
n is the total number of samples for the predicted area category of the ith vehicle. In the embodiment of the present application, the area category may be represented by a number, for example, if the area category is a vehicle head, the number corresponds to 0, if the area category is a vehicle body, the number corresponds to 1, and if the area category is a vehicle tail, the number corresponds to 2, liAnd (yi) is a second preset value, and L (y) corresponding to each vehicle is the same.
Wherein, the total loss value can be calculated by adopting the following formula:
Ltotal＝Lheatmap+Lbox+Lidentity+Lcar_shape_classes
wherein L istotalDenotes the total loss value, LheatmapRepresents a first sub-loss value, LboxRepresents a second sub-loss value, LidentityRepresents a second loss value, Lcar_shape_classesRepresenting a third loss value.
Example 4:
in order to implement the cross-camera tracking of the vehicle, on the basis of the foregoing embodiments, in an embodiment of the present application, before determining the ID corresponding to the target vehicle as the ID of the vehicle in the current image frame, the method includes:
judging whether the first time number of the target vehicle appearing in the image frames acquired by the image acquisition equipment exceeds a preset first time threshold value or not;
if not, determining the first image acquisition equipment in front of the image acquisition equipment according to a pre-stored sequence corresponding to each image acquisition equipment;
obtaining the saved vehicle information of each first candidate vehicle leaving the monitoring range of the first image acquisition device, wherein the vehicle information comprises candidate IDs and candidate feature vectors of the first candidate vehicles;
determining a target first candidate vehicle matched with the target vehicle in each first candidate vehicle according to the candidate feature vector carried in the vehicle information of each first candidate vehicle and the feature vector of the target vehicle;
judging whether the target first candidate vehicle is matched with the vehicle or not according to the candidate feature vector of the target first candidate vehicle and the target feature vector of the vehicle, and if so, updating the stored second number of times that the target vehicle is matched with the target first candidate vehicle;
and if the updated second number of times exceeds a preset second number of times threshold, determining the target candidate ID of the target first candidate vehicle as the ID of the target vehicle.
In the embodiment of the application, a plurality of image acquisition devices exist on a road, the monitoring range of each image acquisition device is different, after a vehicle leaves the monitoring range of one image acquisition device, the vehicle enters the monitoring range of another image acquisition device, and the electronic device needs to ensure that the determined ID of the vehicle is not changed no matter which image acquisition device the vehicle is acquired by.
When a vehicle enters the monitoring range of an image acquisition device, the area of the area acquired by the vehicle may gradually change in the image frames acquired by the image acquisition device. Fig. 6 is a schematic diagram of a vehicle appearing in captured image frames of an image capturing device according to an embodiment of the present application, where, as shown in fig. 6, an image frame a is an image frame a where the vehicle first appears in the captured image frames of the image capturing device, and only 1/4 of the area of the vehicle appears in the image frame a; b, the image frame is the image frame which is acquired by the image acquisition equipment and appears for the second time, and at the moment, 1/2 area of the vehicle appears in the image frame b; c, the image frame is the image frame which is acquired by the image acquisition equipment and appears for the third time, and at the moment, the vehicle has 3/4 area appearing in the c image frame; the d image frame is the image frame which is acquired by the image acquisition device and is the fourth time the vehicle appears, and all areas of the vehicle appear in the d image frame at the moment.
In the embodiment of the application, when the electronic device determines the ID of a target vehicle, the electronic device determines whether a first number of times that the target vehicle appears in an image frame already acquired by an image acquisition device exceeds a preset first number threshold, where the first number threshold is used to indicate whether the image acquisition device has completely acquired the vehicle, and if the first number of times does not exceed the first preset threshold, it is determined that the image acquisition device has not completely acquired the vehicle, that is, it indicates that the vehicle has just entered a monitoring range of the image acquisition device, and at this time, it is detected that a feature vector of the vehicle is not sufficient. At this time, the electronic device needs to perform matching with the target vehicle a plurality of times based on the vehicle included in the image frame captured by the first image capturing device preceding the image capturing device, and determine the ID of the target vehicle. Wherein, the first preset threshold number of image frames may also be referred to as a feature matching buffer.
Specifically, according to the installation position of the image acquisition device, the electronic device prestores a sequence corresponding to each image acquisition device, determines a first image acquisition device before the image acquisition device according to the sequence, and acquires vehicle information of each first candidate vehicle which is saved and leaves the monitoring range of the first image acquisition device, wherein the vehicle information includes a candidate ID and a candidate feature vector of the first candidate vehicle. It should be noted that, a storage area is configured in advance in the electronic device, the storage area is divided into a plurality of sub-storage areas, each sub-storage area stores, for a corresponding image capturing device, vehicle information of each first candidate vehicle that leaves a monitoring range of the image capturing device, and the storage area may be named as a cross-camera track container.
The electronic device determines a target first candidate vehicle matched with the target vehicle in each first candidate vehicle according to the candidate feature vector carried in the vehicle information of the first candidate vehicle and the feature vector of the target vehicle, judges whether the target first candidate vehicle is matched with the vehicle according to the candidate feature vector of the target first candidate vehicle and the target feature vector of the vehicle, updates a second number of times that the stored target vehicle is matched with the target first candidate vehicle if the target first candidate vehicle is matched with the vehicle, and determines the target candidate ID of the target first candidate vehicle as the ID of the target vehicle if the updated second number of times exceeds a preset second number threshold.
In order to reduce the storage pressure of the electronic device, on the basis of the foregoing embodiments, in an embodiment of the present application, the method further includes:
and deleting the saved vehicle information of the target first candidate vehicle.
In the embodiment of the application, in order to relieve the storage pressure of the electronic device, after determining that the target vehicle is successfully matched with the target first candidate vehicle, the electronic device deletes the stored vehicle information of the target first candidate vehicle.
Example 5:
in order to improve the accuracy of vehicle matching, on the basis of the foregoing embodiments, in an embodiment of the present application, the determining, according to a target feature vector and a target area class corresponding to the vehicle, a target vehicle matching the vehicle in a previous image frame includes:
determining a target sub-feature vector corresponding to the target area category in the target feature vector and a sub-feature vector corresponding to the target area category in a feature vector corresponding to each second candidate vehicle in the previous image frame;
and determining the similarity between the target sub-feature vector and each sub-feature vector, and determining a second candidate vehicle corresponding to the highest similarity exceeding a similarity threshold as the target vehicle.
In the embodiment of the application, in order to improve the accuracy of vehicle matching, when the electronic device determines a target vehicle matched with the vehicle, the electronic device determines a target sub-feature vector corresponding to a target area type in a target feature vector corresponding to the vehicle, and a sub-feature vector corresponding to the target area type in a feature vector corresponding to each second candidate vehicle in a previous image frame. The electronic equipment determines the similarity between each target sub-feature vector and each sub-feature vector, and determines the second candidate vehicle corresponding to the highest similarity exceeding the similarity threshold as the target vehicle.
In the embodiment of the application, the feature vector corresponding to the vehicle comprises a plurality of sub-feature vectors, each sub-feature vector corresponds to one region type, and when the electronic device determines the target vehicle matched with the vehicle in the previous image frame according to the target feature vector and the target region type corresponding to the vehicle, only the similarity between the target feature vector and the sub-feature vector corresponding to the target region type in the feature vector needs to be determined, so that the accuracy of vehicle matching and tracking is improved, and the calculation pressure of the electronic device is reduced.
Example 6:
for further tracking the vehicle, on the basis of the foregoing embodiments, in an embodiment of the present application, the method further includes:
judging whether a third candidate vehicle which does not appear in the current image frame exists or not according to the ID of each vehicle in the current image frame and the ID of the vehicle in each image frame acquired by the image acquisition equipment;
if the third candidate vehicle exists, counting a first frame number and a second disappearing frame number of the third candidate vehicle in the image frames acquired by the image acquisition equipment, and if the first frame number exceeds a preset appearing frame number threshold and the second frame number exceeds a preset disappearing frame number threshold, determining that the third candidate vehicle is a vehicle leaving the monitoring range of the image acquisition equipment;
and acquiring a feature vector and an ID of the third candidate vehicle in the last appearing image frame as vehicle information of the third candidate vehicle, and storing the vehicle information.
In the embodiment of the present application, there may be a case where a vehicle appears in the last image frame but does not appear in the current image frame, that is, a case where a vehicle leaves the monitoring range of the image capturing apparatus. In order to achieve cross-image capture device tracking of vehicles, in this embodiment of the application, the electronic device may store vehicle information of vehicles that are away from the monitoring range of the image capture device. In order to eliminate the false detection, in the embodiment of the present application, a screening mechanism is stored in the electronic device in advance, and only when the number of frames of the vehicle reaches the threshold of the number of appearing frames and the number of disappearing frames of the vehicle reaches the threshold of the number of disappearing frames, the electronic device considers that the vehicle leaves the monitoring range of the image acquisition device.
Specifically, in the embodiment of the present application, the electronic device determines whether there is a third candidate vehicle that does not appear in the current image frame, according to the ID of each vehicle in the current image frame and the ID of the vehicle in each image frame that has been captured by the image capturing device. If the third candidate vehicle exists, the electronic device counts a first frame number and a second disappearing frame number of the third candidate vehicle in the image frame acquired by the image acquisition device, if the first frame number exceeds a preset appearing frame number threshold value and the second frame number exceeds a preset disappearing frame number threshold value, the third candidate vehicle is determined to be a vehicle leaving the monitoring range of the image acquisition device, the electronic device obtains a feature vector and an ID of the third candidate vehicle in the image frame appearing at the last time as the vehicle information of the third candidate vehicle, and the vehicle information is stored.
Example 7:
fig. 7 is a schematic structural diagram of a vehicle tracking device according to an embodiment of the present application, where the device includes:
an obtaining module 701, configured to obtain a current image frame acquired by an image acquisition device;
a feature extraction module 702, configured to input the current image frame into a trained model, obtain a feature vector of each vehicle included in the current image frame output by the model, and an area category corresponding to an area where each vehicle is collected, where the feature vector includes a sub-feature vector corresponding to each area category;
a matching module 703, configured to determine, for each vehicle included in the current image frame, a target vehicle that is matched with the vehicle in a previous image frame according to a target feature vector and a target area category corresponding to the vehicle; and determining the ID corresponding to the target vehicle as the ID of the vehicle in the current image frame.
In a possible implementation, the feature extraction module 702 is specifically configured to input the current image frame into a first sub-model of the model, and determine a first intermediate image frame carrying position information of each vehicle; inputting the first intermediate image frame into a second sub-model of the model, and determining a second intermediate image frame carrying a feature vector of each vehicle; and inputting the second intermediate image frame into a third submodel of the model, determining a region category corresponding to the region of each vehicle, and outputting a feature vector and the region category corresponding to each vehicle.
In a possible embodiment, the apparatus further comprises:
a training module 704, configured to obtain each sample image frame stored in a training sample set, where each sample image frame carries initial position information and an initial area category corresponding to each vehicle; for each sample image frame, inputting the sample image frame into a first sub-model of the model, and determining a predicted first intermediate image frame carrying predicted position information of each vehicle; inputting the predicted first intermediate image frame into a second sub-model of the model, wherein the second sub-model determines a predicted characteristic vector of each vehicle, allocates a prediction number to each vehicle according to the predicted characteristic vector of each vehicle, and determines a predicted second intermediate image frame carrying the predicted characteristic vector of each vehicle; inputting the predicted second intermediate image frame into a third submodel of the model, and determining a predicted region category corresponding to the region of each vehicle; determining a first loss value corresponding to the first sub-model according to initial position information and predicted position information corresponding to each vehicle in each sample image frame; determining a second loss value corresponding to the second sub-model according to whether at least vehicles with the same prediction numbers exist in each sample image frame; determining a third loss value corresponding to the third sub-model according to the initial region type and the predicted region type corresponding to each vehicle in each sample image frame; and adjusting the parameters of the model according to the first loss value, the second loss value and the third loss value.
In a possible embodiment, the apparatus further comprises:
the processing module 705 is further configured to determine whether a first number of times that the target vehicle appears in the image frames acquired by the image acquisition device exceeds a preset first number threshold; if not, determining the first image acquisition equipment in front of the image acquisition equipment according to a pre-stored sequence corresponding to each image acquisition equipment; obtaining the saved vehicle information of each first candidate vehicle leaving the monitoring range of the first image acquisition device, wherein the vehicle information comprises the candidate ID and the candidate feature vector of the first candidate vehicle; determining a target first candidate vehicle matched with the target vehicle in each first candidate vehicle according to the candidate feature vector carried in the vehicle information of each first candidate vehicle and the feature vector of the target vehicle; judging whether the target first candidate vehicle is matched with the vehicle or not according to the candidate characteristic vector of the target first candidate vehicle and the target characteristic vector of the vehicle, and if so, updating the stored second number of matching between the target vehicle and the target first candidate vehicle; and if the updated second number of times exceeds a preset second number of times threshold, determining the target candidate ID of the target first candidate vehicle as the ID of the target vehicle.
In a possible implementation, the processing module 705 is further configured to delete the saved vehicle information of the target first candidate vehicle.
In a possible implementation manner, the matching module 703 is specifically configured to determine a target sub-feature vector corresponding to the target area category in the target feature vector, and a sub-feature vector corresponding to the target area category in a feature vector corresponding to each second candidate vehicle in the previous image frame; and determining the similarity between the target sub-feature vector and each sub-feature vector, and determining a second candidate vehicle corresponding to the highest similarity exceeding a similarity threshold as the target vehicle.
In a possible implementation manner, the processing module 705 is further configured to determine whether a third candidate vehicle that does not appear in the current image frame exists according to the ID of each vehicle in the current image frame and the ID of the vehicle in each image frame that has been acquired by the image acquisition device; if the third candidate vehicle exists, counting a first frame number and a second disappearing frame number of the third candidate vehicle in the image frames acquired by the image acquisition equipment, and if the first frame number exceeds a preset appearing frame number threshold and the second frame number exceeds a preset disappearing frame number threshold, determining that the third candidate vehicle is a vehicle leaving the monitoring range of the image acquisition equipment; and acquiring a feature vector and an ID of the third candidate vehicle in the last appearing image frame as vehicle information of the third candidate vehicle, and storing the vehicle information.
Example 8:
on the basis of the foregoing embodiments, an electronic device is further provided in an embodiment of the present application, and fig. 8 is a schematic structural diagram of an electronic device provided in an embodiment of the present application, as shown in fig. 8, including: the system comprises a processor 801, a communication interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communication interface 802 and the memory 803 complete mutual communication through the communication bus 804;
the memory 803 has stored therein a computer program which, when executed by the processor 801, causes the processor 801 to perform the steps of:
acquiring a current image frame acquired by image acquisition equipment;
inputting the current image frame into a trained model, and acquiring a feature vector of each vehicle contained in the current image frame output by the model and an area category corresponding to an area acquired by each vehicle, wherein the feature vector comprises a sub-feature vector corresponding to each area category;
for each vehicle contained in the current image frame, determining a target vehicle matched with the vehicle in the last image frame according to the target feature vector and the target area category corresponding to the vehicle; and determining the ID corresponding to the target vehicle as the ID of the vehicle in the current image frame.
In one possible implementation, the processor is further configured to:
inputting the current image frame into a first sub-model of the model, and determining a first intermediate image frame carrying position information of each vehicle;
inputting the first intermediate image frame into a second sub-model of the model, and determining a second intermediate image frame carrying a feature vector of each vehicle;
and inputting the second intermediate image frame into a third submodel of the model, determining the region category corresponding to the region of each vehicle, and outputting the feature vector and the region category corresponding to each vehicle.
In one possible implementation, the processor is further configured to:
acquiring each sample image frame stored in a training sample set, wherein each sample image frame carries initial position information and an initial area category corresponding to each vehicle;
for each sample image frame, inputting the sample image frame into a first sub-model of the model, and determining a predicted first intermediate image frame carrying predicted position information of each vehicle; inputting the predicted first intermediate image frame into a second sub-model of the model, wherein the second sub-model determines a predicted characteristic vector of each vehicle, allocates a prediction number to each vehicle according to the predicted characteristic vector of each vehicle, and determines a predicted second intermediate image frame carrying the predicted characteristic vector of each vehicle; inputting the predicted second intermediate image frame into a third submodel of the model, and determining a predicted region category corresponding to the region of each vehicle;
determining a first loss value corresponding to the first sub-model according to initial position information and predicted position information corresponding to each vehicle in each sample image frame;
determining a second loss value corresponding to the second submodel according to whether at least vehicles with the same prediction number exist in each sample image frame;
determining a third loss value corresponding to the third sub-model according to the initial region type and the predicted region type corresponding to each vehicle in each sample image frame;
and adjusting the parameters of the model according to the first loss value, the second loss value and the third loss value.
In one possible implementation, the processor is further configured to:
judging whether the first time number of the target vehicle appearing in the image frames acquired by the image acquisition equipment exceeds a preset first time threshold value or not;
if not, determining a first image acquisition device before the image acquisition devices according to a pre-stored sequence corresponding to each image acquisition device;
obtaining the saved vehicle information of each first candidate vehicle leaving the monitoring range of the first image acquisition device, wherein the vehicle information comprises candidate IDs and candidate feature vectors of the first candidate vehicles;
determining a target first candidate vehicle matched with the target vehicle in each first candidate vehicle according to the candidate feature vector carried in the vehicle information of each first candidate vehicle and the feature vector of the target vehicle;
judging whether the target first candidate vehicle is matched with the vehicle or not according to the candidate characteristic vector of the target first candidate vehicle and the target characteristic vector of the vehicle, and if so, updating the stored second number of matching between the target vehicle and the target first candidate vehicle;
and if the updated second number of times exceeds a preset second number of times threshold, determining the target candidate ID of the target first candidate vehicle as the ID of the target vehicle.
In one possible implementation, the processor is further configured to:
and deleting the saved vehicle information of the target first candidate vehicle.
In one possible implementation, the processor is further configured to:
determining a target sub-feature vector corresponding to the target area category in the target feature vector and a sub-feature vector corresponding to the target area category in a feature vector corresponding to each second candidate vehicle in the previous image frame;
and determining the similarity between the target sub-feature vector and each sub-feature vector, and determining a second candidate vehicle corresponding to the highest similarity exceeding a similarity threshold as the target vehicle.
In one possible implementation, the processor is further configured to:
judging whether a third candidate vehicle which does not appear in the current image frame exists or not according to the ID of each vehicle in the current image frame and the ID of the vehicle in each image frame acquired by the image acquisition equipment;
if the third candidate vehicle exists, counting a first frame number and a second disappearing frame number of the third candidate vehicle in the image frames acquired by the image acquisition equipment, and if the first frame number exceeds a preset appearing frame number threshold and the second frame number exceeds a preset disappearing frame number threshold, determining that the third candidate vehicle is a vehicle leaving the monitoring range of the image acquisition equipment;
and acquiring a feature vector and an ID of the third candidate vehicle in the last appearing image frame as vehicle information of the third candidate vehicle, and storing the vehicle information.
Since the principle of solving the problem by the electronic device is similar to that of the vehicle tracking method, the implementation of the electronic device can be referred to the embodiment of the method, and repeated descriptions are omitted.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus. The communication interface 802 is used for communication between the above-described electronic apparatus and other apparatuses. The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the aforementioned processor.
The Processor may be a general-purpose Processor, including a central processing unit, a Network Processor (NP), and the like; but may also be a Digital instruction processor (DSP), an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc.
Example 9:
on the basis of the foregoing embodiments, the present invention further provides a computer-readable storage medium, in which a computer program executable by a processor is stored, and when the program is run on the processor, the processor is caused to execute the following steps:
acquiring a current image frame acquired by image acquisition equipment;
inputting the current image frame into a trained model, and acquiring a feature vector of each vehicle contained in the current image frame output by the model and a region category corresponding to a region acquired by each vehicle, wherein the feature vector comprises a sub-feature vector corresponding to each region category;
for each vehicle contained in the current image frame, determining a target vehicle matched with the vehicle in the last image frame according to the target feature vector and the target area category corresponding to the vehicle; and determining the ID corresponding to the target vehicle as the ID of the vehicle in the current image frame.
In a possible implementation manner, the inputting the current image frame into a trained model, obtaining a feature vector of each vehicle included in the current image frame output by the model, and acquiring a region category corresponding to a region where each vehicle is collected includes:
inputting the current image frame into a first sub-model of the model, and determining a first intermediate image frame carrying position information of each vehicle;
inputting the first intermediate image frame into a second submodel of the model, and determining a second intermediate image frame carrying the feature vector of each vehicle;
and inputting the second intermediate image frame into a third submodel of the model, determining the region category corresponding to the region of each vehicle, and outputting the feature vector and the region category corresponding to each vehicle.
In one possible embodiment, the training process of the model includes:
acquiring each sample image frame stored in a training sample set, wherein each sample image frame carries initial position information and an initial area type corresponding to each vehicle;
for each sample image frame, inputting the sample image frame into a first sub-model of the model, and determining a predicted first intermediate image frame carrying predicted position information of each vehicle; inputting the predicted first intermediate image frame into a second sub-model of the model, wherein the second sub-model determines a predicted characteristic vector of each vehicle, allocates a predicted number to each vehicle according to the predicted characteristic vector of each vehicle, and determines a predicted second intermediate image frame carrying the predicted characteristic vector of each vehicle; inputting the predicted second intermediate image frame into a third submodel of the model, and determining a predicted region category corresponding to the region of each vehicle;
determining a first loss value corresponding to the first sub-model according to initial position information and predicted position information corresponding to each vehicle in each sample image frame;
determining a second loss value corresponding to the second sub-model according to whether at least vehicles with the same prediction numbers exist in each sample image frame;
determining a third loss value corresponding to the third sub-model according to the initial region category and the predicted region category corresponding to each vehicle in each sample image frame;
and adjusting parameters of the model according to the first loss value, the second loss value and the third loss value.
In one possible embodiment, before determining the ID corresponding to the target vehicle as the ID of the vehicle in the current image frame, the method includes:
judging whether the first time number of the target vehicle appearing in the image frames acquired by the image acquisition equipment exceeds a preset first time threshold value or not;
if not, determining a first image acquisition device before the image acquisition devices according to a pre-stored sequence corresponding to each image acquisition device;
obtaining the saved vehicle information of each first candidate vehicle leaving the monitoring range of the first image acquisition device, wherein the vehicle information comprises candidate IDs and candidate feature vectors of the first candidate vehicles;
determining a target first candidate vehicle matched with the target vehicle in each first candidate vehicle according to the candidate feature vector carried in the vehicle information of each first candidate vehicle and the feature vector of the target vehicle;
judging whether the target first candidate vehicle is matched with the vehicle or not according to the candidate feature vector of the target first candidate vehicle and the target feature vector of the vehicle, and if so, updating the stored second number of times that the target vehicle is matched with the target first candidate vehicle;
and if the updated second number of times exceeds a preset second number of times threshold, determining the target candidate ID of the target first candidate vehicle as the ID of the target vehicle.
In one possible embodiment, the method further comprises:
and deleting the saved vehicle information of the target first candidate vehicle.
In a possible implementation, the determining, according to the target feature vector and the target area class corresponding to the vehicle, the target vehicle matching the vehicle in the previous image frame includes:
determining a target sub-feature vector corresponding to the target area category in the target feature vector and a sub-feature vector corresponding to the target area category in a feature vector corresponding to each second candidate vehicle in the previous image frame;
and determining the similarity between the target sub-feature vector and each sub-feature vector, and determining a second candidate vehicle corresponding to the highest similarity exceeding a similarity threshold as the target vehicle.
In one possible embodiment, the method further comprises:
judging whether a third candidate vehicle which does not appear in the current image frame exists or not according to the ID of each vehicle in the current image frame and the ID of the vehicle in each image frame acquired by the image acquisition equipment;
if the third candidate vehicle exists, counting a first frame number and a second disappearing frame number of the third candidate vehicle in the image frames acquired by the image acquisition equipment, and if the first frame number exceeds a preset appearing frame number threshold and the second frame number exceeds a preset disappearing frame number threshold, determining that the third candidate vehicle is a vehicle leaving the monitoring range of the image acquisition equipment;
and acquiring a feature vector and an ID of the third candidate vehicle in a last-appearing image frame as vehicle information of the third candidate vehicle, and storing the vehicle information.
Since the principle of solving the problem of the computer-readable storage medium is similar to that of the vehicle tracking method, the implementation of the computer-readable storage medium can be referred to the embodiment of the method, and repeated descriptions are omitted.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims (10)

1. A method of vehicle tracking, the method comprising:
acquiring a current image frame acquired by image acquisition equipment;
inputting the current image frame into a trained model, and acquiring a feature vector of each vehicle contained in the current image frame output by the model and a region category corresponding to a region acquired by each vehicle, wherein the feature vector comprises a sub-feature vector corresponding to each region category;
for each vehicle contained in the current image frame, determining a target vehicle matched with the vehicle in the last image frame according to the target feature vector and the target area category corresponding to the vehicle; and determining the ID corresponding to the target vehicle as the ID of the vehicle in the current image frame.
2. The method according to claim 1, wherein the inputting the current image frame into a trained model, obtaining a feature vector of each vehicle included in the current image frame output by the model, and a region category corresponding to a region where each vehicle is collected comprises:
inputting the current image frame into a first sub-model of the model, and determining a first intermediate image frame carrying position information of each vehicle;
inputting the first intermediate image frame into a second sub-model of the model, and determining a second intermediate image frame carrying a feature vector of each vehicle;
and inputting the second intermediate image frame into a third submodel of the model, determining a region category corresponding to the region of each vehicle, and outputting a feature vector and the region category corresponding to each vehicle.
3. The method of claim 2, wherein the training process of the model comprises:
acquiring each sample image frame stored in a training sample set, wherein each sample image frame carries initial position information and an initial area category corresponding to each vehicle;
for each sample image frame, inputting the sample image frame into a first sub-model of the model, and determining a predicted first intermediate image frame carrying predicted position information of each vehicle; inputting the predicted first intermediate image frame into a second sub-model of the model, wherein the second sub-model determines a predicted characteristic vector of each vehicle, allocates a prediction number to each vehicle according to the predicted characteristic vector of each vehicle, and determines a predicted second intermediate image frame carrying the predicted characteristic vector of each vehicle; inputting the predicted second intermediate image frame into a third submodel of the model, and determining a predicted region category corresponding to the region of each vehicle;
determining a first loss value corresponding to the first sub-model according to initial position information and predicted position information corresponding to each vehicle in each sample image frame;
determining a second loss value corresponding to the second sub-model according to whether at least vehicles with the same prediction numbers exist in each sample image frame;
determining a third loss value corresponding to the third sub-model according to the initial region category and the predicted region category corresponding to each vehicle in each sample image frame;
and adjusting the parameters of the model according to the first loss value, the second loss value and the third loss value.
4. The method of claim 1, wherein determining the ID corresponding to the target vehicle as being before the ID of the vehicle in the current image frame comprises:
judging whether the first time number of the target vehicle appearing in the image frames acquired by the image acquisition equipment exceeds a preset first time threshold value or not;
if not, determining the first image acquisition equipment in front of the image acquisition equipment according to a pre-stored sequence corresponding to each image acquisition equipment;
obtaining the saved vehicle information of each first candidate vehicle leaving the monitoring range of the first image acquisition device, wherein the vehicle information comprises candidate IDs and candidate feature vectors of the first candidate vehicles;
determining a target first candidate vehicle matched with the target vehicle in each first candidate vehicle according to the candidate feature vector carried in the vehicle information of each first candidate vehicle and the feature vector of the target vehicle;
judging whether the target first candidate vehicle is matched with the vehicle or not according to the candidate feature vector of the target first candidate vehicle and the target feature vector of the vehicle, and if so, updating the stored second number of times that the target vehicle is matched with the target first candidate vehicle;
and if the updated second number of times exceeds a preset second number of times threshold, determining the target candidate ID of the target first candidate vehicle as the ID of the target vehicle.
5. The method of claim 4, further comprising:
and deleting the saved vehicle information of the target first candidate vehicle.
6. The method of claim 1, wherein determining the target vehicle matching the vehicle in the previous image frame according to the target feature vector and the target area class corresponding to the vehicle comprises:
determining a target sub-feature vector corresponding to the target area category in the target feature vector and a sub-feature vector corresponding to the target area category in a feature vector corresponding to each second candidate vehicle in the previous image frame;
and determining the similarity between the target sub-feature vector and each sub-feature vector, and determining a second candidate vehicle corresponding to the highest similarity exceeding a similarity threshold as the target vehicle.
7. The method of claim 1, further comprising:
judging whether a third candidate vehicle which does not appear in the current image frame exists or not according to the ID of each vehicle in the current image frame and the ID of the vehicle in each image frame acquired by the image acquisition equipment;
if the third candidate vehicle exists, counting a first frame number and a second disappearing frame number of the third candidate vehicle in the image frames acquired by the image acquisition equipment, and if the first frame number exceeds a preset appearing frame number threshold and the second frame number exceeds a preset disappearing frame number threshold, determining that the third candidate vehicle is a vehicle leaving the monitoring range of the image acquisition equipment;
and acquiring a feature vector and an ID of the third candidate vehicle in the last appearing image frame as vehicle information of the third candidate vehicle, and storing the vehicle information.
8. A vehicle tracking apparatus, characterized in that the apparatus comprises:
the acquisition module is used for acquiring a current image frame acquired by the image acquisition equipment;
the feature extraction module is used for inputting the current image frame into a trained model, and acquiring a feature vector of each vehicle contained in the current image frame output by the model and an area category corresponding to an area acquired by each vehicle, wherein the feature vector comprises a sub-feature vector corresponding to each area category;
the matching module is used for determining a target vehicle matched with the vehicle in the previous image frame according to a target feature vector and a target area category corresponding to the vehicle for each vehicle contained in the current image frame; and determining the ID corresponding to the target vehicle as the ID of the vehicle in the current image frame.
9. An electronic device, characterized in that the electronic device comprises a processor for implementing the steps of the vehicle tracking method according to any of claims 1-7 when executing a computer program stored in a memory.
10. A computer-readable storage medium, characterized in that it stores a computer program which, when being executed by a processor, carries out the steps of a vehicle tracking method according to any one of claims 1 to 7.
CN202210841301.1A 2022-07-18 2022-07-18 Vehicle tracking method, device, equipment and medium Pending CN115272988A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210841301.1A CN115272988A (en) 2022-07-18 2022-07-18 Vehicle tracking method, device, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210841301.1A CN115272988A (en) 2022-07-18 2022-07-18 Vehicle tracking method, device, equipment and medium

Publications (1)

Publication Number Publication Date
CN115272988A true CN115272988A (en) 2022-11-01

Family

ID=83766589

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210841301.1A Pending CN115272988A (en) 2022-07-18 2022-07-18 Vehicle tracking method, device, equipment and medium

Country Status (1)

Country Link
CN (1) CN115272988A (en)

Similar Documents

Publication Publication Date Title
CN109035299B (en) Target tracking method and device, computer equipment and storage medium
CN107851318B (en) System and method for object tracking
US9767570B2 (en) Systems and methods for computer vision background estimation using foreground-aware statistical models
US10896495B2 (en) Method for detecting and tracking target object, target object tracking apparatus, and computer-program product
CN113284168A (en) Target tracking method and device, electronic equipment and storage medium
CN110706261A (en) Vehicle violation detection method and device, computer equipment and storage medium
CN109035295B (en) Multi-target tracking method, device, computer equipment and storage medium
Bedruz et al. Real-time vehicle detection and tracking using a mean-shift based blob analysis and tracking approach
CN109766867B (en) Vehicle running state determination method and device, computer equipment and storage medium
CN108898057B (en) Method, device, computer equipment and storage medium for tracking target detection
CN111639653A (en) False detection image determining method, device, equipment and medium
CN111402293A (en) Vehicle tracking method and device for intelligent traffic
CN111199554A (en) Target tracking anti-blocking method and device
EP3726421A2 (en) Recognition method and apparatus for false detection of an abandoned object and image processing device
US20230342951A1 (en) Object tracking apparatus, object tracking method, and program
CN113674317A (en) Vehicle tracking method and device of high-order video
CN115272988A (en) Vehicle tracking method, device, equipment and medium
CN113762027B (en) Abnormal behavior identification method, device, equipment and storage medium
US11620360B2 (en) Methods and systems for recognizing object using machine learning model
CN110781710B (en) Target object clustering method and device
KR20230030996A (en) Object tracking apparatus and method
CN109949335B (en) Image processing method and device
JP4818430B2 (en) Moving object recognition method and apparatus
JP7488673B2 (en) MOVING OBJECT TRACKING DEVICE, MOVING OBJECT TRACKING METHOD, AND MOVING OBJECT TRACKING PROGRAM
CN112906495B (en) Target detection method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination