CN112069969A - A method and system for cross-mirror vehicle tracking in expressway surveillance video - Google Patents

Abstract

The invention relates to a method and a system for tracking a highway monitoring video mirror-crossing vehicle, belongs to the technical field of computer vision images, and solves the problems that the conventional mirror-crossing vehicle tracking method is difficult to implement in an actual scene and the algorithm is low in applicability. Acquiring frame images in a plurality of camera video files of a highway to be monitored, and carrying out vehicle detection on each frame image based on an improved YOLO target detection model to obtain a vehicle detection image containing a complete vehicle rectangular frame; inputting the vehicle detection image into a multi-target tracking model to obtain a vehicle tracking result; establishing a vehicle information database according to the vehicle detection image and the vehicle tracking result; the target vehicle image is intercepted based on a certain vehicle detection image corresponding to any camera number in the vehicle information database, and the motion track of the target vehicle corresponding to the target vehicle image is matched according to the vehicle information database, so that cross-mirror tracking is realized, the implementation difficulty of the tracking method is reduced, and the applicability is improved.

Description

A method and system for cross-mirror vehicle tracking in expressway surveillance video

technical field

The invention relates to the technical field of computer vision images, in particular to a method and system for tracking a vehicle across mirrors in a highway surveillance video.

Background technique

At present, the highway monitoring system is getting more and more perfect, and the cameras are more and more densely arranged, but the highway is still the most prone to accidents. The reason is that the highway speed is fast, there are many large vehicles, and the surveillance cameras are limited.

The existing cross-mirror vehicle re-identification technology is based on the data sets that have been labeled. Although the vehicle re-identification technology has been further developed with the update and release of high-quality data sets, the data set scenarios are limited, and the algorithm model It has strong limitations, it is difficult to implement in practical scenarios and the applicability of the algorithm is low.

SUMMARY OF THE INVENTION

In view of the above analysis, the embodiments of the present invention aim to provide a method and system for cross-mirror vehicle tracking in expressway surveillance video, so as to solve the problem that the existing cross-mirror vehicle tracking method is difficult to implement in actual scenarios and the algorithm is applicable. lower problem.

On the one hand, an embodiment of the present invention provides a method for tracking a vehicle across a mirror in a highway surveillance video, including the following steps:

Obtain frame images from multiple camera video files of the highway to be monitored, and perform vehicle detection on each frame image based on the improved YOLO target detection model to obtain a vehicle detection image containing a complete vehicle rectangular frame;

Inputting the vehicle detection image into a multi-target tracking model to obtain a vehicle tracking result; the vehicle tracking result includes a vehicle ID and a vehicle trajectory;

Establish a vehicle information database according to the vehicle detection image and the vehicle tracking result;

The target vehicle image is intercepted based on a vehicle detection image corresponding to any camera number in the vehicle information database, and the motion track of the target vehicle corresponding to the target vehicle image is matched according to the vehicle information database to realize cross-mirror tracking.

Further, the improved YOLO target detection model includes a feature extraction network layer and a YOLO detection layer; wherein, the feature extraction network layer includes a stem unit and an OSA unit;

The stem unit is used for down-sampling the frame images in the video files of the multiple cameras of the expressway to obtain an image with a size of 304*304*128;

The OSA unit is used to convolve an image with an input size of 304*304*128 to obtain an image with a size of 19*19*512;

The YOLO detection layer is used to obtain a vehicle detection image containing a complete vehicle rectangular frame according to the image with a size of 19*19*512 output by the OSA unit.

Further, the multi-target tracking model includes a motion prediction unit and a depth appearance feature extraction unit:

The motion prediction unit is used for predicting the vehicle prediction image of the current frame according to the vehicle detection image of the previous frame;

The deep appearance feature extraction unit includes a re-identification network, obtains a vehicle trajectory based on the vehicle detection image and the vehicle prediction image input to the re-identification network, and numbers the vehicle trajectory to obtain a vehicle ID corresponding to the vehicle trajectory.

Further, establishing the vehicle information database according to the vehicle tracking result specifically includes: storing the vehicle detection image and the vehicle trajectory in the database based on the camera number and the vehicle ID to obtain the vehicle information database.

Further, the target vehicle image is intercepted based on a vehicle detection image corresponding to any camera number in the vehicle information database, and the motion trajectory of the target vehicle corresponding to the target vehicle image is matched according to the vehicle information database to realize cross-mirror tracking, including: Follow the steps below:

Obtain a vehicle detection image corresponding to any camera number in the vehicle information database, and intercept the target vehicle image;

Based on the target vehicle image and a certain vehicle detection image corresponding to other camera numbers, respectively obtain a depth feature matrix of the target vehicle image and a depth feature matrix of a vehicle detection image corresponding to other camera numbers;

Based on the depth feature matrix of the target vehicle image and the depth feature matrix of a vehicle detection image corresponding to other camera numbers, the re-identification network is used to obtain the cosine of the vehicle in the target vehicle and a vehicle detection image corresponding to other camera numbers similarity distance;

Classify and sort the cosine similarity distance according to the camera number to obtain the minimum cosine similarity distance corresponding to the same camera number;

Judging whether the minimum cosine similarity distance is less than the similarity threshold, if so, the vehicle in the corresponding vehicle detection image is the target vehicle, if not, judging that the target vehicle has left the highway to be monitored;

Based on the camera number and vehicle ID corresponding to the vehicle detection image, the vehicle information database is matched to obtain the motion trajectory of the target vehicle, thereby realizing cross-mirror tracking.

On the other hand, an embodiment of the present invention provides a cross-mirror vehicle tracking system for highway surveillance video, including:

The detection module is used to obtain frame images in the video files of multiple cameras on the highway to be monitored, and based on the improved YOLO target detection model, vehicle detection is performed on each frame image to obtain a vehicle detection image containing a complete vehicle rectangular frame;

a tracking module for inputting the vehicle detection image into a multi-target tracking model to obtain a vehicle tracking result; the vehicle tracking result includes a vehicle ID and a vehicle trajectory;

a vehicle information database obtaining module, used for establishing a vehicle information database according to the vehicle detection image and the vehicle tracking result;

The motion trajectory obtaining module is used to intercept the target vehicle image according to a vehicle detection image corresponding to any camera number in the vehicle information database, and match the motion trajectory of the target vehicle corresponding to the target vehicle image according to the vehicle information database to achieve Tracking across mirrors.

Further, the detection module includes a feature extraction network layer and a YOLO detection layer; wherein, the feature extraction network layer includes a stem unit and an OSA unit;

The stem unit is used for down-sampling the frame images in the video files of the multiple cameras of the expressway to obtain an image with a size of 304*304*128;

The OSA unit is used to convolve an image with an input size of 304*304*128 to obtain an image with a size of 19*19*512;

The YOLO detection layer is used to obtain a vehicle detection image containing a complete vehicle rectangular frame according to the image with a size of 19*19*512 output by the OSA unit.

Further, the tracking module includes a motion prediction unit and a depth appearance feature extraction unit:

The motion prediction unit is used for predicting the vehicle prediction image of the current frame according to the vehicle detection image of the previous frame;

The deep appearance feature extraction unit includes a re-identification network, obtains a vehicle trajectory based on the vehicle detection image and the vehicle prediction image input to the re-identification network, and numbers the vehicle trajectory to obtain a vehicle ID corresponding to the vehicle trajectory.

Further, the vehicle information database obtaining module stores the vehicle detection image and the vehicle track in the database according to the camera number and the vehicle ID to obtain the vehicle information database.

Further, the motion trajectory obtaining module executes the following process:

Obtain a vehicle detection image corresponding to any camera number in the vehicle information database, and intercept the target vehicle image;

Based on the target vehicle image and a certain vehicle detection image corresponding to other camera numbers, respectively obtain a depth feature matrix of the target vehicle image and a depth feature matrix of a vehicle detection image corresponding to other camera numbers;

Based on the depth feature matrix of the target vehicle image and the depth feature matrix of a vehicle detection image corresponding to other camera numbers, the re-identification network is used to obtain the cosine of the vehicle in the target vehicle and a vehicle detection image corresponding to other camera numbers similarity distance;

Classify and sort the cosine similarity distance according to the camera number to obtain the minimum cosine similarity distance corresponding to the same camera number;

Judging whether the minimum cosine similarity distance is less than the similarity threshold, if so, the vehicle in the corresponding vehicle detection image is the target vehicle, if not, judging that the target vehicle has left the highway to be monitored;

Based on the camera number and vehicle ID corresponding to the vehicle detection image, the vehicle information database is matched to obtain the motion trajectory of the target vehicle, thereby realizing cross-mirror tracking.

Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

1. A cross-mirror vehicle tracking method for highway surveillance video. The vehicle is detected through the improved YOLO target detection model, and the vehicle detection image is obtained. The vehicle is tracked through the multi-target tracking model to obtain the vehicle ID and vehicle trajectory. Finally, based on The re-identification network obtains the cosine similarity, and then splices to obtain the complete motion trajectory of the target vehicle, which provides an efficient, fast and high-precision video analysis technology for the highway management department to safely monitor and search for the target vehicle.

2. By replacing the backbone network in YOLOv3 with VoVNet, a variant of DenseNet with better learning ability, the feature extraction network layer is obtained. At the same time, according to the size of the vehicle in the highway surveillance video image, the large, medium and small three of the YOLOv3 detection layer are used. The scale is reduced to two scales of medium and small, and the improved YOLO target detection model is obtained, which makes the model smaller, the calculation speed is faster, and the calculation amount is reduced, and the vehicle detection image containing the complete vehicle rectangular frame can be obtained more quickly.

3. The vehicle information database can be obtained by saving the vehicle detection image and vehicle trajectory to the database according to the naming rule of camera number-vehicle ID, which provides data support and basis for the re-identification of vehicle trajectory and the complete splicing of target vehicle trajectory.

4. Intercept the target vehicle image from a vehicle detection image corresponding to any camera in the vehicle information database by using the depth appearance feature extraction unit, and calculate the difference between the target vehicle image and multiple vehicles in a vehicle detection image in other cameras. Cosine similarity, and through the cosine similarity, the target vehicle is matched with multiple vehicles in a vehicle detection image in other cameras, and then the motion trajectory of the target vehicle is obtained by splicing to achieve cross-mirror tracking, with high matching efficiency and high precision. .

In the present invention, the above technical solutions can also be combined with each other to achieve more preferred combination solutions. Additional features and advantages of the invention will be set forth in the description which follows, and some of the advantages may become apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by means of particularly pointed out in the description and drawings.

Description of drawings

The drawings are for the purpose of illustrating specific embodiments only and are not to be considered limiting of the invention, and like reference numerals refer to like parts throughout the drawings.

Figure 1 is the overall structure diagram of the cross-mirror vehicle tracking method for highway surveillance video;

FIG. 2 is a schematic flowchart of a method for tracking a vehicle across a mirror in a highway surveillance video;

Figure 3 is a schematic structural diagram of the improved YOLO target detection model;

Figure 4 is a schematic diagram of the structure of the DeepSort multi-target tracking model;

Figure 5 is a schematic structural diagram of a cross-mirror vehicle tracking system for highway surveillance video;

Reference number:

100-detection module, 200-tracking module, 300-vehicle information database acquisition module, 400-movement trajectory acquisition module.

Detailed ways

The preferred embodiments of the present invention are specifically described below with reference to the accompanying drawings, wherein the accompanying drawings constitute a part of the present application, and together with the embodiments of the present invention, are used to explain the principles of the present invention, but are not used to limit the scope of the present invention.

Cross-camera tracking refers to tracking the motion trajectory of the target vehicle based on video files captured by multiple cameras, and splicing the motion trajectory of the target vehicle corresponding to each camera to obtain the complete motion trajectory of the target vehicle. The existing cross-mirror vehicle re-identification technology is based on the data sets that have been labeled. Although the vehicle re-identification technology has been further developed with the update and release of high-quality data sets, the data set scenarios are limited, and the algorithm model It has strong limitations, it is difficult to implement in practical scenarios and the applicability of the algorithm is low. To this end, the present application proposes a method and system for vehicle tracking across mirrors in highway surveillance video, as shown in Figure 1, through the improved YOLO target detection model to monitor each frame image in the highway video files of multiple cameras to perform vehicle tracking The vehicle detection image is obtained by detection, and the vehicle detection image is input into the multi-target tracking model to obtain the vehicle trajectory and the corresponding vehicle ID. The vehicle ID is the number of the vehicle trajectory; the vehicle detection image and vehicle trajectory are stored according to the camera number and vehicle ID. Finally, according to the vehicle information database, the motion trajectory of the target vehicle corresponding to the target vehicle image is matched to realize cross-mirror tracking. In this application, the vehicle trajectory information corresponding to the video captured by each surveillance camera is extracted, and the trajectory of the target vehicle is connected across the lenses to obtain the complete motion trajectory of the target vehicle on the highway to be monitored, thereby realizing the vehicle's cross-mirror tracking and solving the problem. The existing cross-mirror vehicle tracking method is difficult to implement in actual scenarios and the applicability of the algorithm is low. It provides an efficient, fast and high-precision video analysis technology for the highway management department to monitor and search for target vehicles. Has high practical value.

A specific embodiment of the present invention discloses a method for tracking vehicles across mirrors in expressway surveillance video, as shown in FIG. 2 . It includes the following steps:

Step S1: Obtain frame images from multiple camera video files of the expressway to be monitored, and perform vehicle detection on each frame image based on the improved YOLO target detection model to obtain a vehicle detection image containing a complete vehicle rectangular frame. Considering the characteristics of highway surveillance video images, the highway surveillance cameras are about 10 meters away from the ground, the image resolution is poor, and the vehicle target is small, the angle is mostly oblique downward, and the vehicle target is mostly looking down at the sideways body, and the image There are few obstacles, etc. Therefore, this application uses the improved YOLO target detection model to perform vehicle detection on each frame image obtained from multiple camera video files of the highway to be monitored, so as to obtain a vehicle detection that contains a complete vehicle rectangular frame. image. Among them, the improved YOLO target detection model is to replace the backbone network in the original YOLOv3 with VoVNet (feature extraction network layer), a variant of DenseNet with better learning ability. The model is smaller and faster. At the same time, for high-speed The size of the vehicle captured by the road surveillance video image reduces the three scales of large, medium and small in the YOLOv3 detection layer to two scales, which further reduces the amount of computation.

Preferably, the improved YOLO target detection model includes a feature extraction network layer and a YOLO detection layer; wherein, the feature extraction network layer includes a stem unit and an OSA unit; the stem unit is used for the frame images in the video files of multiple cameras on the highway. Downsampling to obtain an image with a size of 304*304*128; OSA unit, used to convolve an image with an input size of 304*304*128 to obtain an image with a size of 19*19*512; YOLO detection layer, with According to the image with a size of 19*19*512 output by the OSA unit, a vehicle detection image containing a complete vehicle rectangle frame is obtained, wherein the vehicle detection image includes images of multiple vehicles framed by the complete vehicle rectangle frame.

Specifically, as shown in Figure 3, the improved YOLO target detection model includes a feature extraction network layer and a YOLO detection layer. Among them, the feature extraction network layer includes a stem unit and an OSA unit. The function of the stem unit is to downsample the frame images in the video files of multiple cameras on the highway to obtain an image with a size of 304*304*128. The OSA unit includes four OSA subunits. The function of the first OSA subunit is to convolve the image output by the stem unit with a size of 304*304*128 to obtain an image with a size of 152*152*128; the second one The role of the OSA subunit is to convolve the image with the size of 152*152*128 output by the first OSA subunit to obtain an image of size 76*76*256; the role of the third OSA subunit is to The image of size 76*76*256 output by the two OSA subunits is convolved to obtain an image of size 38*38*384; the function of the fourth OSA subunit is to convert the output size of the third OSA subunit Convolve an image of 38*38*384 to get an image of size 19*19*512. At the same time, the YOLO detection layer in the improved YOLO target detection model is to reduce the three scales of the original YOLOv3 detection layer, large, medium and small, to two small and medium scales. 512 to obtain a vehicle detection image containing a complete vehicle rectangle. Based on the obtained improved YOLO target detection model, the road monitoring dataset is used to train the model, and the hyperparameter search is performed, that is, for each new generation of hyperparameters, the previous generation with the highest adaptability (in all previous generations) is selected. ), all parameters are mutated at the same time according to a normal distribution with about 20% 1-sigma to obtain hyperparameters such as appropriate learning rate, weight of each part of the loss function, multi-scale training, and finally save the best performance of the network Good generation hyperparameters as hyperparameters for formal training.

The feature extraction network layer is obtained by replacing the backbone network in YOLOv3 with VoVNet, a variant of DenseNet with better learning ability. It is reduced to two scales of medium and small, and an improved YOLO target detection model is obtained, which makes the model smaller, the calculation speed is faster, and the calculation amount is reduced, and the vehicle detection image containing the complete vehicle rectangular frame can be obtained more quickly.

Step S2, input the vehicle detection image into the multi-target tracking model to obtain the vehicle tracking result; the vehicle tracking result includes the vehicle ID and the vehicle trajectory. Preferably, the multi-target tracking model includes a motion prediction unit and a depth appearance feature extraction unit: the motion prediction unit is used to predict and obtain the vehicle prediction image of the current frame according to the vehicle detection image of the previous frame; the depth appearance feature extraction unit includes a re-identification network, based on The vehicle detection image and vehicle prediction image of the re-identification network are input to obtain the vehicle trajectory, and the vehicle trajectory is numbered to obtain the vehicle ID corresponding to the vehicle trajectory.

Specifically, as shown in Figure 4, the DeepSort multi-target tracking model includes a motion prediction unit and a depth appearance feature extraction unit. The function of the motion prediction unit is to predict the current frame vehicle prediction image according to the previous frame of vehicle detection image, and the depth The appearance feature extraction unit mainly calculates the depth feature information of the vehicle detection image and the vehicle prediction image through the re-identification network, takes the vehicle prediction image that has been successfully associated and matched as the vehicle position of the vehicle in the current frame, and connects the center point coordinates of the vehicle positions in multiple frames. The vehicle track is obtained, and the vehicle track is numbered while the vehicle track is obtained, and the vehicle ID corresponding to the vehicle track can be obtained.

Step S3, establishing a vehicle information database according to the vehicle detection image and the vehicle tracking result.

Preferably, establishing the vehicle information database according to the vehicle tracking result specifically includes: storing the vehicle detection image and the vehicle trajectory in the database based on the camera number and the vehicle ID to obtain the vehicle information database. Specifically, based on the vehicle detection image obtained in step S1 and the vehicle ID corresponding to the vehicle trajectory obtained in step S2, the vehicle information database can be obtained by saving the vehicle detection image and vehicle trajectory to the database according to the naming rule of camera number-vehicle ID.

The vehicle information database can be obtained by saving the vehicle detection image and vehicle trajectory to the database according to the naming rule of camera number-vehicle ID, which provides data support and basis for the re-identification of vehicle trajectory and the complete splicing of target vehicle trajectory.

Step S4: Intercept the target vehicle image based on a vehicle detection image corresponding to any camera number in the vehicle information database, and match the motion trajectory of the target vehicle corresponding to the target vehicle image according to the vehicle information database to realize cross-mirror tracking. Specifically, in step S3, after the vehicle detection image and the vehicle trajectory are stored in the database with the camera number-vehicle ID, a vehicle detection image corresponding to any camera number can be selected to intercept the target vehicle image, and then the target vehicle image can be searched for other vehicles according to the target vehicle image. A vehicle corresponding to the camera detects the target vehicle in the image, and splices the motion trajectories of the target vehicle in each camera to obtain the complete motion trajectory of the target vehicle. Preferably, the target vehicle image is intercepted based on the vehicle detection image in the vehicle information database, and the motion trajectory of the target vehicle corresponding to the target vehicle image is matched according to the vehicle information database to realize cross-mirror tracking, including the following steps:

Step S401 , acquiring a vehicle detection image corresponding to any camera in the vehicle information database, and intercepting the image of the target vehicle. Specifically, after the vehicle detection image and vehicle trajectory are saved to the database with the camera number-vehicle ID in step S3, the target vehicle image can be intercepted from a vehicle detection image corresponding to any camera number in the vehicle information database.

Step S402 , based on the target vehicle image and a certain vehicle detection image corresponding to other cameras, obtain a depth feature matrix of the target vehicle image and a depth feature matrix of a certain vehicle detection image corresponding to other camera numbers, respectively. Specifically, the target vehicle image and a certain vehicle detection image corresponding to other camera numbers are input into the depth appearance feature extraction unit, and the depth feature matrix of the target vehicle image and the depth feature matrix of a vehicle detection image corresponding to other camera numbers can be obtained. Here, the depth feature matrix is obtained according to a vehicle detection image corresponding to each camera. Compared with the depth feature matrix obtained by processing each frame of images in real time in the process of obtaining the vehicle trajectory in step S2, the amount of calculation reduced and more accurate.

Step S403, based on the depth feature matrix of the target vehicle image and the depth feature matrix of a vehicle detection image corresponding to other camera numbers, use the re-identification network to obtain the cosines of multiple vehicles in a vehicle detection image corresponding to the target vehicle and other cameras. similarity distance. Specifically, the formula for calculating the cosine similarity distance is:

In the formula, cos(θ) is the cosine similarity distance, a is the depth feature matrix corresponding to the target vehicle image, b is the depth feature matrix corresponding to the vehicle detection image, and x _i is the ith in the depth feature matrix corresponding to the target vehicle image. dimension element, y _i is the i-th dimension element in the depth feature matrix corresponding to the vehicle detection image, n is the dimension of the depth feature matrix, 1≤i≤n.

Step S404 , classify and sort the cosine similarity distances according to the camera numbers to obtain the minimum cosine similarity distances corresponding to the same camera numbers. Specifically, a vehicle detection image is selected based on each camera number, and each vehicle detection image includes images of multiple vehicles. Based on step S404, the cosine similarity distances of multiple vehicles in a certain vehicle detection image corresponding to the target vehicle and other cameras can be calculated, and the cosine similarity distances are classified and sorted according to the camera number to obtain the minimum cosine corresponding to the same camera number. similarity distance.

Step S405, determine whether the minimum cosine similarity distance is less than the similarity threshold, if so, the vehicle corresponding to the minimum cosine similarity distance in the corresponding vehicle detection image is the target vehicle, if not, determine that the target vehicle leaves the highway to be monitored. Specifically, a vehicle detection image corresponding to each other camera can obtain a minimum cosine similarity distance, and based on the minimum cosine similarity distance, it can be determined whether there is a target in a vehicle detection image corresponding to each other camera vehicle. Among them, the similarity threshold is obtained by taking the average of a large number of experiments. For different external conditions, such as light, rain and other conditions that affect the monitoring and shooting, different vehicle models correspond to different similarity thresholds.

Step S406 , matching the vehicle information database based on the camera number and the vehicle ID corresponding to the vehicle detection image to obtain the motion trajectory of the target vehicle, so as to realize cross-mirror tracking. Based on the previous step, when it is determined that the vehicle in the vehicle detection image corresponding to each other camera is the target image, the vehicle information database can be queried through the camera-vehicle ID to obtain the corresponding vehicle trajectory of the target vehicle in the vehicle detection image corresponding to each camera. , and splicing the corresponding vehicle trajectories, the complete motion trajectory of the target vehicle can be obtained to achieve cross-mirror tracking.

By using the depth appearance feature extraction unit, the target vehicle image is intercepted from a vehicle detection image corresponding to any camera in the vehicle information database, and the cosine similarity between the target vehicle image and multiple vehicles in a vehicle detection image in other cameras is calculated. The cosine similarity is used to match the target vehicle with multiple vehicles in a vehicle detection image in his camera, and then the motion trajectory of the target vehicle is obtained to achieve cross-mirror tracking, with high matching efficiency and high accuracy.

Compared with the prior art, this embodiment provides a cross-mirror vehicle tracking method for expressway surveillance video. The vehicle is detected by the improved YOLO target detection model, the vehicle detection image is obtained, and the vehicle is tracked by the multi-target tracking model. , obtain the vehicle ID and vehicle trajectory, and finally obtain the cosine similarity based on the re-identification network, and then splicing to obtain the complete motion trajectory of the target vehicle, which provides an efficient, fast and high-precision video analysis for the highway management department to monitor and search the target vehicle. technology.

Another specific embodiment of the present invention discloses a cross-mirror vehicle tracking system for highway surveillance video, as shown in FIG. 5 , including a detection module 100 for acquiring frame images in video files of multiple cameras on the highway to be monitored , based on the improved YOLO target detection model, vehicle detection is performed on each frame image to obtain a vehicle detection image containing a complete vehicle rectangular frame; the tracking module 200 is used to input the vehicle detection image into the multi-target tracking model to obtain vehicle tracking results. The result includes vehicle ID and vehicle trajectory; the vehicle information database acquisition module 300 is used to establish a vehicle information database according to the vehicle detection image and the vehicle tracking result; the motion trajectory acquisition module 400 is used to intercept the target vehicle according to the vehicle detection image in the vehicle information database. image, and match the motion trajectory of the target vehicle corresponding to the target vehicle image according to the vehicle information database to achieve cross-mirror tracking. Specifically, the system can select video files, perform tracking and display tracking results, that is, the image of each vehicle, and display the coordinates of the center point of the vehicle in a form to obtain the vehicle trajectory, that is, the vehicle ID corresponding to it, and also There is a pause resume tracking function.

A cross-mirror vehicle tracking system for highway surveillance video, detects vehicles through an improved YOLO target detection model, obtains vehicle detection images, tracks vehicles through a multi-target tracking model, obtains vehicle IDs and vehicle trajectories, and finally based on re-identification The network obtains the cosine similarity, and then splices to obtain the complete motion trajectory of the target vehicle, which provides an efficient, fast and high-precision video analysis technology for the highway management department to monitor and search the target vehicle safely.

Preferably, the detection module includes a feature extraction network layer and a YOLO detection layer; wherein, the feature extraction network layer includes a stem unit and an OSA unit; the stem unit is used to downsample the frame images in the video files of multiple cameras on the highway, and obtain An image of size 304*304*128; OSA unit, used to convolve an image with an input size of 304*304*128 to obtain an image of size 19*19*512; YOLO detection layer, used according to the OSA unit The output size is 19*19*512, and the vehicle detection image containing the complete vehicle rectangle is obtained.

The detection module replaces the backbone network in YOLOv3 with VoVNet, a variant of DenseNet with better learning ability to obtain the feature extraction network layer. The scale is reduced to two scales of medium and small, and the improved YOLO target detection model is obtained, which makes the model smaller, the calculation speed is faster and the amount of calculation is reduced, and the vehicle detection image containing the complete vehicle rectangular frame can be obtained more quickly.

Preferably, the tracking module includes a motion prediction unit and a depth appearance feature extraction unit, the motion prediction unit is used to predict and obtain the vehicle prediction image of the current frame according to the vehicle detection image of the previous frame; the depth appearance feature extraction unit includes a re-identification network, based on The vehicle detection image and vehicle prediction image of the re-identification network are input to obtain the vehicle trajectory, and the vehicle trajectory is numbered to obtain the vehicle ID corresponding to the vehicle trajectory.

Preferably, the vehicle information database obtaining module stores the vehicle detection image and the vehicle track in the database according to the camera number and the vehicle ID to obtain the vehicle information database.

Through the vehicle information database acquisition module, the vehicle detection image and vehicle trajectory are saved to the database according to the naming rule of camera number-vehicle ID, and the vehicle information database can be obtained, which provides data support for the re-identification of vehicle trajectory and the complete splicing of target vehicle trajectory. in accordance with.

Preferably, the motion trajectory obtaining module executes the following process:

Obtain a vehicle detection image corresponding to any camera number in the vehicle information database, and use the depth appearance feature extraction unit to intercept the target vehicle image;

Based on the target vehicle image and a vehicle detection image corresponding to other camera numbers, the depth feature matrix of the target vehicle image and the depth feature matrix of a vehicle detection image corresponding to other camera numbers are obtained respectively;

Based on the depth feature matrix of the target vehicle image and the depth feature matrix of a vehicle detection image corresponding to other camera numbers, the re-identification network is used to obtain the cosine similarity distance between the target vehicle and a vehicle detection image corresponding to other camera numbers;

Categorize and sort the cosine similarity distance according to the camera number to obtain the minimum cosine similarity distance corresponding to the same camera number;

Determine whether the minimum cosine similarity distance is less than the similarity threshold, if so, the vehicle in the corresponding vehicle detection image is the target vehicle, if not, determine that the target vehicle leaves the highway to be monitored;

Based on the camera number and vehicle ID corresponding to the vehicle detection image, the vehicle information database is matched to obtain the motion trajectory of the target vehicle, thereby realizing cross-mirror tracking.

The motion trajectory acquisition module can calculate the similarity between the target vehicle and all the tracked vehicles, and display it on the computer interface according to the size of the similarity. The higher the order is, the closer it is to the target vehicle. The square outputs the successfully matched vehicle IDs and splices the trajectory to obtain the complete trajectory of the target vehicle. Use the depth appearance feature extraction unit to intercept the target vehicle image from a vehicle detection image corresponding to any camera in the vehicle information database, and calculate the cosine similarity between the target vehicle image and multiple vehicles in a vehicle detection image from other cameras. , and use the cosine similarity to match the target vehicle with multiple vehicles in a vehicle detection image in his camera, and then obtain the motion trajectory of the target vehicle to achieve cross-mirror tracking, with high matching efficiency and high accuracy.

The above description is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited to this. Substitutions should be covered within the protection scope of the present invention.

Claims (10)

1. a kind of highway monitoring video cross-mirror vehicle tracking method, is characterized in that, comprises the steps: Obtain frame images from multiple camera video files of the highway to be monitored, and perform vehicle detection on each frame image based on the improved YOLO target detection model to obtain a vehicle detection image containing a complete vehicle rectangular frame; Inputting the vehicle detection image into a multi-target tracking model to obtain a vehicle tracking result; the vehicle tracking result includes a vehicle ID and a vehicle trajectory; Establish a vehicle information database according to the vehicle detection image and the vehicle tracking result; The target vehicle image is intercepted based on a vehicle detection image corresponding to any camera number in the vehicle information database, and the motion track of the target vehicle corresponding to the target vehicle image is matched according to the vehicle information database to realize cross-mirror tracking. 2. The cross-mirror vehicle tracking method for expressway surveillance video according to claim 1, wherein the improved YOLO target detection model comprises a feature extraction network layer and a YOLO detection layer; wherein, the feature extraction network layer comprises stem unit and OSA unit; The stem unit is used for down-sampling the frame images in the video files of the multiple cameras of the expressway to obtain an image with a size of 304*304*128; The OSA unit is used to convolve an image with an input size of 304*304*128 to obtain an image with a size of 19*19*512; The YOLO detection layer is used to obtain a vehicle detection image containing a complete vehicle rectangular frame according to the image with a size of 19*19*512 output by the OSA unit. 3. The expressway surveillance video cross-mirror vehicle tracking method according to claim 1, wherein the multi-target tracking model comprises a motion prediction unit and a depth appearance feature extraction unit: The motion prediction unit is used for predicting the vehicle prediction image of the current frame according to the vehicle detection image of the previous frame; The deep appearance feature extraction unit includes a re-identification network, obtains a vehicle trajectory based on the vehicle detection image and the vehicle prediction image input to the re-identification network, and numbers the vehicle trajectory to obtain a vehicle ID corresponding to the vehicle trajectory. 4. The expressway monitoring video cross-mirror vehicle tracking method according to claim 3, wherein establishing a vehicle information database according to the vehicle tracking result specifically comprises: storing the vehicle detection image and the vehicle trajectory based on the camera number and the vehicle ID Go to the database to get the vehicle information database. 5 . The cross-mirror vehicle tracking method for expressway surveillance video according to claim 4 , wherein the target vehicle image is intercepted based on a vehicle detection image corresponding to any camera number in the vehicle information database, and according to the vehicle information database. 6 . The vehicle information database matches the motion trajectory of the target vehicle corresponding to the target vehicle image to achieve cross-mirror tracking, including the following steps: Obtain a vehicle detection image corresponding to any camera number in the vehicle information database, and intercept the target vehicle image; Based on the target vehicle image and a certain vehicle detection image corresponding to other camera numbers, respectively obtain a depth feature matrix of the target vehicle image and a depth feature matrix of a vehicle detection image corresponding to other camera numbers; Based on the depth feature matrix of the target vehicle image and the depth feature matrix of a vehicle detection image corresponding to other camera numbers, the re-identification network is used to obtain multiple vehicles in a vehicle detection image corresponding to the target vehicle and other camera numbers The cosine similarity distance of ; Classify and sort the cosine similarity distance according to the camera number to obtain the minimum cosine similarity distance corresponding to the same camera number; Determine whether the minimum cosine similarity distance corresponding to other camera numbers is less than the similarity threshold, if so, the vehicle in the corresponding vehicle detection image is the target vehicle, if not, determine that the target vehicle has left the expressway to be monitored; Based on the camera number and vehicle ID corresponding to the vehicle detection image, the vehicle information database is matched, and the motion trajectory of the target vehicle is obtained by splicing, so as to realize cross-mirror tracking. 6. A cross-mirror vehicle tracking system for highway surveillance video, characterized in that, comprising: The detection module is used to obtain frame images in the video files of multiple cameras on the highway to be monitored, and based on the improved YOLO target detection model, vehicle detection is performed on each frame image to obtain a vehicle detection image containing a complete vehicle rectangular frame; a tracking module for inputting the vehicle detection image into a multi-target tracking model to obtain a vehicle tracking result; the vehicle tracking result includes a vehicle ID and a vehicle trajectory; a vehicle information database obtaining module, used for establishing a vehicle information database according to the vehicle detection image and the vehicle tracking result; The motion trajectory obtaining module is used to intercept the target vehicle image according to a vehicle detection image corresponding to any camera number in the vehicle information database, and match the motion trajectory of the target vehicle corresponding to the target vehicle image according to the vehicle information database to achieve Tracking across mirrors. 7. The highway surveillance video cross-mirror vehicle tracking system according to claim 6, wherein the detection module comprises a feature extraction network layer and a YOLO detection layer; wherein, the feature extraction network layer comprises a stem unit and an OSA unit; The stem unit is used for down-sampling the frame images in the video files of the multiple cameras of the expressway to obtain an image with a size of 304*304*128; The OSA unit is used to convolve an image with an input size of 304*304*128 to obtain an image with a size of 19*19*512; The YOLO detection layer is used to obtain a vehicle detection image containing a complete vehicle rectangular frame according to the image with a size of 19*19*512 output by the OSA unit. 8. The highway surveillance video cross-mirror vehicle tracking system according to claim 7, wherein the tracking module comprises a motion prediction unit and a depth appearance feature extraction unit: The motion prediction unit is used for predicting the vehicle prediction image of the current frame according to the vehicle detection image of the previous frame; The deep appearance feature extraction unit includes a re-identification network, obtains vehicle trajectories based on the vehicle detection images and vehicle prediction images input to the re-identification network, and numbers the vehicle trajectories to obtain vehicle IDs corresponding to the vehicle trajectories. 9. The highway monitoring video cross-mirror vehicle tracking system according to claim 8, wherein the vehicle information database acquisition module stores the vehicle detection image and the vehicle track in the database to obtain the vehicle information database according to the camera number and the vehicle ID. . 10. The highway monitoring video cross-mirror vehicle tracking system according to claim 9, wherein the motion trajectory obtaining module executes the following process: Obtain a vehicle detection image corresponding to the camera number in the vehicle information database, and intercept the target vehicle image; Based on the target vehicle image and a certain vehicle detection image corresponding to other camera numbers, respectively obtain a depth feature matrix of the target vehicle image and a depth feature matrix of a vehicle detection image corresponding to other camera numbers; Based on the depth feature matrix of the target vehicle image and the depth feature matrix of a vehicle detection image corresponding to other camera numbers, the re-identification network is used to obtain the cosine of the vehicle in a vehicle detection image corresponding to the target vehicle and other camera numbers similarity distance; Classify and sort the cosine similarity distance according to the camera number to obtain the minimum cosine similarity distance corresponding to the same camera number; Determine whether the minimum cosine similarity distance corresponding to other camera numbers is less than the similarity threshold, if so, the vehicle in the corresponding vehicle detection image is the target vehicle, if not, determine that the target vehicle has left the expressway to be monitored; Based on the camera number and vehicle ID corresponding to the vehicle detection image, the vehicle information database is matched, and the motion trajectory of the target vehicle is obtained by splicing, so as to realize cross-mirror tracking.

Images