WO2024017413A2

WO2024017413A2 - Method and apparatus for port entry/exit detection of vehicle, device, and storage medium

Info

Publication number: WO2024017413A2
Application number: PCT/CN2023/120391
Authority: WO
Inventors: 张恒瑞
Original assignee: 顺丰科技有限公司
Priority date: 2022-07-22
Filing date: 2023-09-21
Publication date: 2024-01-25
Also published as: CN117496387A; WO2024017413A3

Abstract

The present application discloses a method and apparatus for port entry/exit detection of a vehicle, a device, and a storage medium. The method comprises: determining depth information of a target vehicle in each transportation image according to position-depth relation data associated with a target site and position information of the target vehicle in each transportation image; obtaining a depth information change pattern of the target vehicle according to the depth information of the target vehicle in each transportation image; and obtaining an port entry/exit detection result for the target vehicle according to the depth information change pattern. The technical solution of the present application can improve the accuracy of port entry/exit detection.

Description

Vehicle entry and exit detection methods, devices, equipment and storage media

Technical field

This application relates to the field of logistics technology, and in particular to a vehicle entry and exit detection method, device, equipment and storage medium.

Background of the invention

In the logistics transfer yard, there are many bays for loading or unloading trucks. In order to improve the operating efficiency of the transfer yard, managers are required to conduct vehicle entry and exit inspections.

Under normal circumstances, surveillance cameras are installed at the loading and unloading bays, so vehicle entry and exit detection can be carried out based on computer vision technology. The existing vehicle entry and exit detection solution based on computer vision technology detects whether there is a carriage in the collected video images based on target recognition technology, and detects the vehicle entry and exit through qualitative analysis. However, the vehicle is easily blocked during driving. , so that the carriage cannot be detected in the collected video images, resulting in inaccurate detection results of vehicle entry and exit detection.

Contents of the invention

Embodiments of the present application provide a vehicle entry and exit detection method, device, equipment and storage medium to improve the accuracy of existing vehicle entry and exit detection solutions based on computer vision technology.

On the one hand, embodiments of the present application provide a vehicle entry and exit detection method, which method includes:

Obtaining multiple continuous transportation images of the target vehicle in the target site, and corresponding position information of the target vehicle in each of the multiple continuous transportation images;

Determine the depth information corresponding to the target vehicle in the transportation image according to the position-depth relationship data associated with the target site and the position information; the position-depth relationship data includes each position information in the target site and Depth information corresponding to each position information. The depth information represents the distance between the point corresponding to the position information in the transportation image and the camera that collects the transportation image when it is mapped to the target site;

According to the depth information corresponding to the multiple consecutive transportation images, the depth information change trend of the target vehicle is obtained;

According to the change trend of the depth information, the entry and exit detection results of the target vehicle are obtained.

In some embodiments of the present application, before determining the corresponding depth information of the target vehicle in the transportation image based on the position depth relationship data associated with the target site and the location information, the vehicle entry and exit detection method Also includes:

Collect a target image corresponding to the target site; the target image is an image in which no vehicle exists in the target site;

Perform depth estimation on the target image through a trained depth estimation network to obtain depth information corresponding to at least one pixel in the target image;

Obtain the position information of the at least one pixel point in the target image, associate the position information of the at least one pixel point with the depth information corresponding to the at least one pixel point, and obtain the position-depth relationship data.

In some embodiments of the present application, before performing depth estimation on the target image through the trained depth estimation network to obtain depth information corresponding to at least one pixel in the target image, the vehicle entry and exit detection method further includes :

Set up multiple test points in the target site;

Collect sample images corresponding to the target site after setting the plurality of test points;

Input the sample image into a pre-trained depth estimation network to obtain test depth values corresponding to the multiple test points in the sample image;

If the error between the test depth value corresponding to one of the plurality of test points and the true distance corresponding to the one test point is greater than the preset error threshold, then according to the test depth value corresponding to the one test point The training loss value between the real distance corresponding to the one test point, adjusting the parameters of the pre-trained depth estimation network until the errors corresponding to the multiple test points are less than or equal to the preset error threshold, we get The trained depth estimation network; wherein the real distance represents the real distance between the test point and the camera.

In some embodiments of the present application, obtaining the depth information change trend of the target vehicle based on the depth information corresponding to the multiple consecutive transportation images includes:

Compare the depth information of the target vehicle in the multiple consecutive transportation images with the preset distance threshold respectively to obtain the change trend of the depth information of the target vehicle; or

According to the difference between the depth information of the target vehicle in the multiple consecutive transportation images, the change trend of the depth information of the target vehicle is obtained.

In some embodiments of the present application, the depth information of the target vehicle in the multiple consecutive transportation images is compared with a preset distance threshold respectively to obtain the change trend of the depth information of the target vehicle, including :

Sort the multiple consecutive transportation images according to the time order corresponding to the multiple consecutive transportation images, and obtain the multiple sorted transportation images and the position of the target vehicle in the sorted multiple transportation images. Corresponding depth information;

Compare the corresponding depth information of the target vehicle in the plurality of sorted transportation images with a preset distance threshold to obtain the change trend of the depth information of the target vehicle.

In some embodiments of the present application, the corresponding depth information of the target vehicle in the plurality of sorted transportation images is compared with a preset distance threshold to obtain the change trend of the depth information of the target vehicle, include:

Difference the corresponding depth information of the target vehicle in the plurality of sorted transportation images with the preset distance threshold to obtain a difference sequence;

The depth information change trend of the target vehicle is obtained according to the difference sequence.

In some embodiments of the present application, after obtaining the depth information change trend of the target vehicle based on the difference sequence, the vehicle entry and exit detection method further includes:

If the change trend of the depth information is decreasing, and the time period in the difference sequence that is less than or equal to the preset difference threshold is greater than the preset time length, then the entry and exit detection result of the target vehicle is port arrival;

If the change trend of the depth information is increasing, and the corresponding depth information of the target vehicle in the plurality of sorted transportation images contains target depth information greater than the preset value, then the entry and exit of the target vehicle The port test result is outbound.

In some embodiments of the present application, obtaining the change trend of the depth information of the target vehicle based on the difference between the depth information of the target vehicle in the multiple consecutive transportation images includes:

In the time period corresponding to the multiple consecutive transportation images, the depth information in the transportation image corresponding to the target vehicle at each moment in the time period is compared with the depth information in the transportation image corresponding to the next moment. information for comparison;

If the depth information corresponding to each moment is greater than the depth information corresponding to the next moment, it is determined that the change trend of the depth information of the target vehicle is increasing;

If the depth information corresponding to each moment is less than the depth information corresponding to the next moment, it is determined that the change trend of the depth information of the target vehicle is decreasing.

In some embodiments of the present application, after obtaining the entry and exit detection results of the target vehicle, the vehicle entry and exit detection method further includes:

According to the time difference between the departure time and the arrival time in the entry and exit detection results, the first residence time of the target vehicle at the target site is obtained;

Obtain at least one second residence time, wherein the at least one second residence time is the residence time of the target vehicle in at least one subsequent site of the target site;

Determining the maximum residence time among the first residence time and the at least one second residence time;

The transportation route and transportation volume of the target vehicle are adjusted according to the maximum detention time, the number of the maximum detention time, and the site corresponding to the maximum detention time.

In some embodiments of the present application, the acquisition of multiple continuous transportation images of the target vehicle in the target site, and the corresponding position information of the target vehicle in each of the multiple continuous transportation images, include:

Obtain the video image of the target site, extract the continuous frame images containing the target vehicle in the video image, and set the continuous frame images as multiple continuous transportation images of the target vehicle in the target site. ;

Perform target detection on each transportation image in the multiple consecutive transportation images to obtain the bounding box coordinates of the target vehicle in the transportation image;

Calculate the midpoint coordinates of the bounding box of the target vehicle according to the bounding box coordinates;

The midpoint coordinates of the target vehicle corresponding to each transportation image in the multiple continuous transportation images are set to the corresponding midpoint coordinates of the target vehicle in each transportation image of the multiple continuous transportation images. location information.

In some embodiments of the present application, obtaining a video image of the target site and extracting consecutive frame images containing the target vehicle in the video image includes:

Obtain the video image of the target site, perform vehicle detection on each frame of the video image, and obtain the detection result of the target vehicle in each frame of image; the detection result represents whether the target exists in the frame image vehicle;

Taking the frame image in which the target vehicle exists as the starting frame, extract consecutive frame images containing the target vehicle in the video image.

In some embodiments of the present application, performing vehicle detection on each frame of the video image includes:

Set the collected image when there is no vehicle in the target site as the reference image;

Difference the first image in the video image and the reference image to obtain a difference image;

If there is a first region with a pixel value greater than the preset pixel value in the difference image, and the average pixel value of the first region is greater than the preset average value, then it is determined that the third image in the video image is located after the first image. The difference image corresponding to the two images;

If the average pixel value of the second area in the differential image corresponding to the second image is greater than the preset pixel value, it is determined that the target vehicle exists in the video image.

On the other hand, embodiments of the present application provide a vehicle entry and exit detection device, which includes:

An acquisition module, configured to acquire multiple continuous transportation images of the target vehicle in the target site, and the corresponding position information of the target vehicle in each of the multiple continuous transportation images;

Depth determination module, configured to determine the depth information corresponding to the target vehicle in the transportation image according to the position-depth relationship data associated with the target site and the location information; the position-depth relationship data includes the target site Each position information in the transportation image and the depth information corresponding to each position information. The depth information represents the distance between the point and the camera that collects the transportation image when the point corresponding to the position information in the transportation image is mapped to the target site. distance;

A depth trend determination module, configured to obtain the depth information change trend of the target vehicle based on the depth information corresponding to the multiple consecutive transportation images;

A detection module is used to obtain the entry and exit detection results of the target vehicle according to the change trend of the depth information.

On the other hand, embodiments of the present application provide a vehicle entry and exit detection device, including a memory and a processor; the memory stores an application program, and the processor is used to run the application program in the memory to execute the Operations in the vehicle entry and exit detection method.

On the other hand, embodiments of the present application provide a storage medium that stores multiple instructions, and the instructions are suitable for loading by a processor to execute the steps in the vehicle entry and exit detection method.

The embodiment of the present application obtains multiple continuous transportation images of the target vehicle in the target site, and the corresponding position information of the target vehicle in each of the multiple continuous transportation images; according to the target site association The position-depth relationship data and the position information are used to determine the depth information corresponding to the target vehicle in the transportation image; and the depth information of the target vehicle is obtained according to the depth information corresponding to the multiple consecutive transportation images. Change trend: According to the change trend of the depth information, the entry and exit detection results of the target vehicle are obtained. In this way, the entry and exit detection results of the target vehicle are determined through the change trend of the depth information of the target vehicle at the target site, and the entry and exit detection is adjusted from the qualitative analysis of the carriage detection to the quantitative calculation of the depth change trend, thereby improving the accuracy of the entry and exit detection. .

Brief description of the drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without exerting creative efforts.

Figure 1 is a schematic flow diagram of an embodiment of a vehicle entry and exit detection method provided by an embodiment of the present application.

Figure 2 is a schematic flowchart of step 101 provided by the embodiment of the present application.

FIG. 3 is a schematic structural diagram of an embodiment of a carriage detection model provided by an embodiment of the present application.

Figure 4 is a schematic flowchart of an embodiment of a method for determining whether a target vehicle leaves the port provided by the embodiment of the present application.

Figure 5 is a schematic structural diagram of an embodiment of a depth estimation network provided by an embodiment of the present application.

FIG. 6 is a schematic structural diagram of a depth estimation layer provided by an embodiment of the present application.

Figure 7 is a schematic flowchart of an embodiment of a method for testing the adaptability of a pre-trained depth estimation network provided by an embodiment of the present application.

FIG. 8 is a schematic structural diagram of an embodiment of the vehicle entry and exit detection device provided by the embodiment of the present application.

Figure 9 is a schematic structural diagram of an embodiment of the vehicle entry and exit detection equipment provided by the embodiment of the present application.

Ways to practice the invention

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts fall within the scope of protection of this application.

As mentioned in the background art, the existing technology is based on the target detection technology in computer vision technology, and performs carriage detection on the collected video images, and determines the vehicle's entry and exit detection results based on the carriage detection results, resulting in inaccurate detection results.

In order to solve the problem of low accuracy caused by only determining vehicle entry and exit detection results through carriage detection, the existing technology has made improvements from both hardware and algorithm aspects; among them, in terms of hardware improvements, multiple cameras are deployed to collect a large number of vehicle parking processes Although multiple cameras can reduce the error in detection results caused by occlusion while the vehicle is driving, deploying multiple cameras will increase the deployment cost; in terms of algorithm improvement, the existing vehicle entry and exit detection based on computer vision technology The program collects a large number of pictures of the vehicle parking process in the scene, covering scenes where the vehicle is fully parked, partially parked, and without a car, in order to improve the accuracy of vehicle entry and exit detection, which allows existing detection solutions to be applied to new scenes. , a large amount of data needs to be supplemented, which increases the cost of data collection.

Based on the above research, in order to improve the accuracy of vehicle entry and exit detection and at the same time reduce the hardware cost in vehicle entry and exit detection applications, embodiments of the present application provide a vehicle entry and exit detection method, based on the depth information of the target vehicle in each transportation image , determine the change trend of the depth information of the target vehicle; according to the change trend of the depth information, obtain the entry and exit detection results of the target vehicle; in this way, determine the entry and exit detection results of the target vehicle through the depth information change trend of the target vehicle at the target site, and then determine the entry and exit detection results of the target vehicle. Port detection is adjusted from qualitative analysis to quantitative calculation, and the accuracy of entry and exit detection is improved through data; at the same time, there is no need to deploy additional cameras, only the surveillance cameras at the target site are used, reducing the hardware cost in vehicle entry and exit detection applications; and by querying the target The location-depth relationship data associated with the site obtains the depth information of the target vehicle in each transportation image, reducing the computational cost in vehicle entry and exit detection applications and improving detection efficiency.

As shown in Figure 1, Figure 1 is a flow diagram of an embodiment of the vehicle entry and exit detection method provided by the embodiment of the present application. The vehicle entry and exit detection method shown includes steps 101 to 104:

Step 101: Obtain multiple continuous transportation images of the target vehicle in the target site, and corresponding position information of the target vehicle in the multiple continuous transportation images.

The target site refers to the transfer site and station during vehicle transportation, such as logistics transfer site, passenger vehicle transfer site, bus station, railway station, etc. Multiple continuous transportation images can be surveillance videos of the target site. Multiple continuous transportation images can also be continuous frame images extracted from surveillance videos of the target site. Multiple continuous transportation images can also be preset images. It is obtained by continuously shooting the target site during the interval. Transportation images refer to images of target vehicles parking and driving in the target site. Target vehicles include but are not limited to freight cars, trucks, passenger cars, motor cars, trucks, and buses.

The position information refers to the spatial coordinates of the target vehicle in the entire transportation image in the acquired transportation image. In some implementations of this application, in order to facilitate the extraction of the position information in each transportation image, target detection can be performed on each transportation image. Determine the image area where the target vehicle is located in each transportation image, mark the image area where the target vehicle is located through the envelope box, and determine the position information of the target vehicle in each transportation image based on the coordinates of the corner points in the envelope box. Among them, the center point coordinates of the envelope frame can be determined based on the coordinates of each corner point in the envelope frame, and the center point coordinates can be set as the position information of the target vehicle in each transportation image; it can also be based on the same vertical vertical position in the envelope frame. The coordinates of the two horizontal corner points of the coordinates determine the midpoint coordinates of the upper or lower boundary in the envelope box, and set the midpoint coordinates of the upper or lower boundary in the envelope box as the coordinates of the target vehicle in each transportation image. location information.

Step 102: Determine the corresponding depth information of the target vehicle in each transportation image based on the position-depth relationship data associated with the target site and the corresponding position information of the target vehicle in each transportation image.

The position-depth relationship data includes each position information in the target site and the depth information corresponding to the position information.

Depth information refers to the distance between the point corresponding to the position information in the transportation image and the camera that captured the transportation image when it is mapped to the real target site. In the vehicle entry and exit detection, considering that the surveillance cameras used to monitor the target site are usually deployed at the loading and unloading bays of the target site, where the loading and unloading bays can be loading and unloading bays, pick-up and unloading bays, so the target site association can be queried The position-depth relationship data is used to determine the depth information that matches the position information in each transportation image, and the depth information is used as an indicator to quantify the judgment of the target vehicle entering and leaving the port.

103. According to the corresponding depth information of the target vehicle in each transportation image, determine the change trend of the depth information of the target vehicle.

The change trend of depth information is used to characterize the changes in the distance between the target vehicle and the camera. The change trend of the depth information of the target vehicle includes a decrease in the value corresponding to the depth information of the target vehicle, an increase in the value corresponding to the depth information of the target vehicle, and a constant value corresponding to the depth information of the target vehicle within the time period corresponding to multiple consecutive transportation images. In some embodiments of the present application, when the change trend of the depth information is decreasing, that is, when the distance between the target vehicle and the camera gradually decreases, it means that the target vehicle is approaching the camera, and the arrival of the target vehicle is determined.

In some embodiments of the present application, the depth information of the target vehicle in multiple consecutive transportation images can be compared with a preset distance threshold to determine the change trend of the depth information of the target vehicle, where the preset distance threshold can be the target The distance from the edge of the loading and unloading platform to the camera in the scene. For example, during the time period corresponding to multiple consecutive transportation images, when the value of the depth information of the target vehicle gradually decreases from the preset distance threshold, the change trend of the depth information of the target vehicle is decreasing.

In some embodiments of the present application, the change trend of the depth information of the target vehicle can also be determined based on the difference between the depth information of the target vehicle in each transportation image, for example, within the time period corresponding to multiple consecutive transportation images. , compare the depth information in the transportation image corresponding to the target vehicle at each moment in the time period with the depth information corresponding to the next moment. If the value of the depth information corresponding to the next moment is larger, it means that the target vehicle is far away from the camera, then the target The changing trend of the depth information of the vehicle is increasing; if the value of the depth information corresponding to the next moment is smaller, it means that the target vehicle is approaching the camera, and the changing trend of the depth information of the target vehicle is decreasing.

104. According to the changing trend of the depth information, obtain the entry and exit detection results of the target vehicle.

In some embodiments of the present application, step 104 includes: if the depth information change trend is decreasing, determine the target vehicle to enter the port; if the depth information change trend is increasing, determine the target vehicle to leave the port.

In some embodiments of the present application, if the change trend of the depth information is decreasing, it is determined that the target vehicle has entered the port, and the arrival time of the target vehicle is determined based on the time information corresponding to multiple consecutive transportation images, and multiple subsequent consecutive transportation images are obtained. For transportation images, determine whether the target vehicle leaves the port based on the depth information of the target vehicle in each subsequent transportation image; if the target vehicle leaves the port, determine the departure time of the target vehicle based on the time information corresponding to multiple subsequent consecutive transportation images; based on the departure time and The time difference between arrival times gives the target vehicle's dwell time at the target site.

In some embodiments of the present application, when the target vehicle involves multiple target sites during driving or transportation, the target vehicle can be obtained based on the length of stay of the target vehicle in the multiple target sites where it has stopped and the traveling speed of the target vehicle. predict The time it takes for the target vehicle to reach the next target site and the time it takes to drive out of the next target site. In some embodiments of the present application, the detention time is related to the condition of the target vehicle, the cargo loading and unloading volume of the target vehicle at the target site, the cargo loading and unloading time, and the passenger flow and vehicle flow at the target site. Determine the target site's detention time at each target site where the target vehicle transports goods or has a larger passenger flow; based on the maximum value of each stay time, obtain the number of occurrences and duration of the maximum value, and based on the number of occurrences and duration And the corresponding target site adjusts the transportation route and transportation volume of the target vehicle; for example, taking the target vehicle as a logistics transport vehicle as an example, when the target vehicle stays at a certain target site for a long time and appears frequently, then Obtain the detention time of the remaining vehicles in the target site. If the detention time of the other vehicles is less than the detention time of the target vehicle, it means that the target vehicle transports more goods and requires a longer loading and unloading time. Then increase the transportation corresponding to the target vehicle. The number of cargo vehicles reduces the detention time of the target vehicle, thereby improving the transportation efficiency of the transport vehicle.

In the embodiment of the present application, the depth information of the target vehicle is determined based on the position depth relationship data associated with the target site and based on the location information of multiple consecutive transportation images in the target site, and the arrival and departure port detection results of the target vehicle are determined based on the depth value. , reduce the amount of calculations in the detection process and improve detection efficiency, in which the depth value can represent the distance value corresponding to the depth information; and through the corresponding depth information of the target vehicle in multiple consecutive transportation images of the target site, the target vehicle's The change trend of depth information determines the entry and exit detection results of the target vehicle based on the depth change trend. In this way, the entry and exit detection is adjusted from qualitative analysis to quantitative calculation, and the accuracy of the entry and exit detection is improved through data.

In some embodiments of the present application, as shown in Figure 2, in step 101, multiple consecutive transportation images of the target vehicle can be extracted from the video image of the target site, and the image area of the target vehicle in each transportation image is determined. , the position information of the target vehicle in each transportation image is obtained based on the midpoint coordinates of the image area. Specifically, step 101 may include steps 1011 to 1013:

Step 1011: Obtain the video image of the target site, extract the continuous frame images containing the target vehicle from the video image, and set the continuous frame images as multiple continuous transportation images of the target vehicle in the target site.

In some embodiments of the present application, in order to reduce the hardware cost of the vehicle entry and exit detection method in practical applications, the target site is monitored through a camera installed at the target site, and the video image of the target site collected by the camera is obtained. The video image may be a real-time surveillance image or a surveillance image in the past period of time. Among them, the past period of time can be the past 1 hour, the past 2 hours, the past 3 hours, etc.

In some embodiments of the present application, in order to improve the accuracy of vehicle entry and exit detection, after obtaining the video image of the target site, vehicle detection is performed on each frame of the video image to determine whether there is a vehicle in the video image; if the video image is If there is no vehicle in the video image, it means that there is no vehicle entering or leaving the target site in the corresponding time period of the video image, and then the video image of the next time period is obtained; if there is a vehicle in the video image, then the vehicle in the video image is Vehicle identification recognition, determine the number of vehicles contained in the video image and the vehicle identification corresponding to each vehicle; obtain multiple consecutive transportation images of the target vehicle corresponding to each vehicle identification in the video image in the target site based on the vehicle identification.

In some embodiments of the present application, vehicle detection can be performed on each frame of the video image by performing image difference on each frame of the video image. Specifically, the collected image when there is no vehicle in the target site is set as the reference image, and each frame image (first image) in the video image is compared with the reference image frame by frame to obtain a difference image; the difference image is obtained The area in which the pixel value is greater than the preset pixel value (the first area). If the mean value of the pixel value in this area is greater than the preset mean value, it means that the frame image is different from the reference image, and then the subsequent frame image in the frame image (the second area) is determined. Is the average pixel value of the area where the pixel value is greater than the preset pixel value in the difference image of the image) greater than the preset average value; if the pixel value of the area (the second area) where the pixel value is greater than the preset pixel value in the difference image of the subsequent frame image If the mean value is greater than the preset mean value, it is determined that there is a vehicle in the video image; if the mean pixel value of the area is less than or equal to the preset mean value, or there is no area with a pixel value greater than the preset pixel value in the difference image, it means that the frame image is different from the preset mean value. If the reference images are the same, determine whether there is an area greater than the preset pixel value in the difference image of the subsequent frame image in the video image; if there is no area greater than the preset pixel value in the difference image of the subsequent frame image in the video image, Then it is determined that there is no vehicle in the video image.

In some embodiments of the present application, vehicle detection can also be performed on each frame of the video image by determining whether there is a carriage in each frame of the video image. For example, each frame of the video image can be detected through a trained carriage detection model. Frame images for vehicle detection. The carriage detection model can be a machine learning model, such as a dictionary learning model, a logistic regression model type, the carriage detection model can also be a detection model based on a neural network. For example, YOLO-based detection model, SSD-based detection model, and RCNN-based detection model.

Step 1012: Perform target detection on each transportation image in multiple consecutive transportation images to obtain the bounding box coordinates of the target vehicle in the transportation image.

In some embodiments of the present application, the bounding box coordinates of the target vehicle in each transportation image may be the bounding box coordinates of the compartment border of the target vehicle in each transportation image, such as the corner coordinates of the compartment boundary box, the compartment boundary box The center point coordinates of , the midpoint coordinates of the upper boundary of the carriage boundary, or the midpoint coordinates of the lower boundary of the carriage bounding box.

Step 1013: Calculate the midpoint coordinates of the target vehicle's bounding box based on the bounding box coordinates, and set the corresponding midpoint coordinates of the target vehicle in each of the multiple continuous transportation images as the target vehicle in multiple continuous transportation images. Corresponding location information in each transportation image in the transportation image.

In some embodiments of the present application, considering that during vehicle entry and exit detection, if the coordinates of the vehicle's body or front are used for detection, due to the irregular shape of the vehicle's front, it may be more difficult to calculate the bounding box coordinates of the target vehicle. This causes an increase in the amount of calculation in the detection of carriages entering and exiting the port. Therefore, the midpoint coordinates of the lower boundary of the carriage bounding box are set as the position information of multiple consecutive transportation images of the target vehicle in the target site.

In some embodiments of the present application, a target detection algorithm can be used to detect the carriage target on each transportation image, and obtain the bounding box of the carriage of the target vehicle in each transportation image. Among them, target detection algorithms include but are not limited to RCNN, SSD, YOLO, etc.

In some embodiments of the present application, after obtaining the carriage boundary of the target vehicle in each transportation image, the center of the transportation image can be determined, and the center of the transportation image can be used as the coordinate origin. According to the corner point of the carriage boundary and the transportation The horizontal pixel length and vertical pixel length between the centers of the image determine the coordinates of the corner points of the carriage boundary. According to the coordinates of each corner point of the carriage boundary point, the coordinates of the center point of the carriage boundary, the coordinates of the midpoint of the upper boundary of the carriage boundary, or The coordinates of the midpoint of the lower boundary of the carriage boundary. It should be noted that the above method of determining coordinates is only an exemplary explanation. In actual application scenarios, the method of determining coordinates can be adjusted, for example, a grid image is made in advance, where the image size of the grid image is the same as that of each frame in the video image. The image sizes of the images are the same. Coordinates are set for each cell in the grid image. After obtaining the compartment boundary of the target vehicle in each transportation image, the grid image is aligned with each transportation image. According to the alignment The small grid where each corner point of the carriage boundary is located in the final transportation image is determined, and the coordinates of each corner point are determined.

In some embodiments of the present application, vehicle detection can be performed on each frame of the video image through a trained carriage detection model to obtain the boundary coordinates of the target vehicle in each transportation image, and the location of the target vehicle at the target site is determined based on the boundary coordinates. Corresponding position information in multiple consecutive transportation images. As shown in the figure, Figure 3 is a schematic structural diagram of an embodiment of a carriage detection model provided by an embodiment of the present application. The carriage detection model shown includes an input layer 201, a carriage detection layer 202 and an image output layer 203 connected in sequence.

The input layer 201 obtains the video image of the target site, and inputs the video image of the target site into the vehicle detection layer; the vehicle detection layer performs vehicle detection on each frame of the video image, and obtains multiple continuous transportation images of the target vehicle in the target site. image, perform target detection on each transportation image, obtain the bounding box coordinates of the target vehicle in each transportation image, and output the bounding box coordinates of the target vehicle to the image output layer 203; the image output layer 203 outputs the boundary of the target vehicle box coordinates.

As shown in Figure 3, the car detection layer 202 includes a car detection unit 20201 and a boundary marking unit 20202 connected in sequence. The carriage detection unit 20201 performs vehicle detection on each frame of the video image, obtains multiple continuous transportation images of the target vehicle in the target site, and inputs multiple continuous transportation images of the target vehicle in the target site to the boundary mark. Unit 20202, the boundary marking unit 20202 performs target detection on each transportation image, obtains the boundary box coordinates of the target vehicle in each transportation image, and outputs the boundary coordinates of the target vehicle to the image output layer 203.

In some embodiments of the present application, the carriage detection unit 20201 also includes a carriage detection subunit and a vehicle logo recognition subunit connected in sequence. The carriage detection subunit performs vehicle detection on each frame of the video image. If there is a vehicle in the video image, vehicle, the video is input to the vehicle logo recognition subunit; the vehicle logo recognition subunit performs vehicle logo recognition on the vehicles in the video image, determines the number of vehicles contained in the video image and the vehicle logo corresponding to each vehicle, and based on the vehicle logo Obtain multiple consecutive transportation images of the target vehicle corresponding to each vehicle identification in the video image in the target site.

In some embodiments of the present application, the image output layer 203 can also calculate the intermediate coordinates of the lower boundary of the compartment boundary based on the coordinates of the target vehicle's compartment boundary box, and set the intermediate coordinates of the lower boundary of the compartment boundary to the position of the target vehicle in the target site. The position information of multiple consecutive transportation images is output.

In some embodiments of the present application, the carriage detection model can be trained through historical surveillance videos collected by surveillance cameras deployed in the target site to obtain a trained carriage detection model. Specifically, the carriage detection model training process includes: from the target site From the historical surveillance videos collected by surveillance cameras, select the sample video image of the vehicle's departure process, and mark the vehicle's rear compartment bounding box. The sample video image marked with the vehicle's real compartment rear bounding box will be used as the real sample video image; the sample video image Each frame of sample image is input into the carriage detection model to obtain the predicted image corresponding to each frame of sample image, in which the predicted carriage rear bounding box is marked on the predicted image; the difference between the predicted image and the real sample video image is calculated based on the preset loss function The error between the real sample images corresponding to each frame of sample image is regarded as the training loss; the parameters of the carriage detection model are adjusted according to the training loss until the carriage detection model reaches the preset convergence condition, and the trained carriage detection model is obtained. The preset loss function can be a mean square error function or a mean absolute error loss function, which is understood as: analyzing the predicted carriage rear boundary box of each frame of sample image according to the loss function and the real carriage rear boundary corresponding to each frame of sample image. The mean square error between frames is used as the training loss of the carriage detection model; according to the loss function, the predicted carriage rear bounding box of each frame of sample image is analyzed and the real carriage rear bounding box corresponding to each frame of sample image is analyzed. The average absolute error between the two is used as the training loss of the carriage detection model. The preset convergence condition can be that the training loss is less than or equal to the preset loss threshold, or the number of training times reaches the preset number. For example, when the training loss is less than or equal to 0.0001, or the number of training times reaches 1000, it is determined that the carriage detection model has reached the preset convergence condition. , to obtain the trained carriage detection model. Historical surveillance videos can be surveillance videos collected by surveillance cameras in the past period of time. In some embodiments of the present application, in order to improve the detection accuracy of the carriage detection model, sample video images of the vehicle's departure and entry process can be selected, and historical surveillance videos collected by surveillance cameras deployed at multiple target sites can be obtained. In some embodiments of the present application, in order to improve the detection accuracy of the carriage detection model in environments with poor visibility such as rainy days, foggy days, snowy days, cloudy days, etc., the obtained data collected by surveillance cameras deployed at multiple target sites can be Noise is added to the historical surveillance video to simulate the historical surveillance video collected in rainy, foggy, snowy and cloudy environments. Sample video images of the vehicle departure process and entry process selected from the processed historical surveillance video are used for carriage detection. The model is trained to improve the generalization ability of the carriage detection model.

In the embodiment of this application, video images of the target site are collected through surveillance cameras that have been deployed at the target site. There is no need to deploy additional cameras, which reduces the hardware cost of vehicle entry and exit detection; and by detecting the coordinates of the carriage boundary box of the target vehicle, the target is determined The position information of multiple consecutive transportation images of vehicles in the target site is used to improve the convenience of vehicle entry and exit detection by taking advantage of the shape rules of the carriages and easy identification. In some embodiments of the present application, as shown in Figure 4, after obtaining the video image of the target site, it can be determined whether there is a frame image in the video image in which the target vehicle cannot be detected, and whether the frame image is in the video image. The location information of the target vehicle is determined to determine whether the target vehicle has left the port. Specifically, obtaining the video image of the target site, and extracting the continuous frame images containing the target vehicle in the video image may include steps 301 to 302:

Step 301: Obtain the video image of the target site, perform vehicle detection on each frame of the video image, and obtain the detection result of the target vehicle in each frame of image.

Step 302: Taking the frame image in which the target vehicle exists as the starting frame, extract consecutive frame images containing the target vehicle in the video image.

In some embodiments of the present application, vehicle detection can be performed on each frame of the video image by performing vehicle detection on each frame of the video image in step 1011 to determine whether a vehicle exists in the video image. When a vehicle is detected, the identification of the vehicle is obtained, the frame image in the video image where the vehicle corresponding to the identification is detected for the first time is obtained, and step 302 is performed on subsequent consecutive frame images of the frame image.

In some embodiments of the present application, when a vehicle exists in the video image, the identification of the vehicle is obtained, the recorded vehicle entry and exit data is queried, the entry and exit time of the target vehicle corresponding to the identification is obtained, and the latest time corresponding to the entry and exit time is obtained. If the entry and exit record corresponding to the latest moment is departure, the target vehicle corresponding to the logo in the video image may be driving into the target site, and the last departure time corresponding to the target logo is obtained from the vehicle entry and exit data. The time difference from the next arrival time, determined if the time difference between each departure and the next arrival is equal Is the time between the difference and the latest time and the time corresponding to the video image equal? If the times are equal, it means that the target vehicle corresponding to the logo in the video image has entered the target site again, then the target vehicle is output and the target is obtained. The arrival time of the vehicle is written into the entry and exit record corresponding to the identification of the target vehicle in the recorded vehicle entry and exit data, and the vehicle entry and exit data is updated.

In some embodiments of the present application, considering that only judging whether the target vehicle has entered the port based on the time difference between each departure and the next arrival may cause inaccurate judgment, if the entry and exit record corresponding to the latest time is a departure, Then according to steps 101 to 103, it is detected whether the changing trend of the position information of the compartment of the target vehicle corresponding to the identification in the subsequent frame images in the video image is decreasing; if the changing trend of the position information of the compartment of the target vehicle corresponding to the identification is is decreasing, then the output detection result of the target vehicle's arrival and departure is that the vehicle has entered the port; if the transformation trend of the position information of the target vehicle's compartment corresponding to the mark is increasing or unchanged, it means that the target vehicle has not completely driven out of the target site or target. When the vehicle stops at the target site, continuous frame images of the vehicle corresponding to the remaining markers in the video image are obtained.

In some embodiments of the present application, if the entry and exit record corresponding to the latest moment is an arrival, then the frame image in the video image where the vehicle corresponding to the logo is detected for the first time is obtained, and the subsequent consecutive frame images of the frame image are Execute step 302.

In some embodiments of the present application, for each frame image, if there is no cabin of the target vehicle in the frame image, it is detected whether the cabin of the target vehicle exists in the image after the frame image.

In some embodiments of the present application, cabin detection is performed on each frame of consecutive frame images in the video image to determine whether there is a frame image of the target vehicle's cabin in the subsequent frame image; if there is no frame image of the target vehicle's cabin in the frame image, , then detect whether the target vehicle compartment exists in the image after the frame image.

In some embodiments of the present application, when it is detected in step 301 that the entry and exit record corresponding to the latest moment of the target vehicle is an arrival, then the frame image in the video image where the vehicle corresponding to the identification is detected for the first time is obtained, and the Carriage detection is performed on each subsequent frame image of the frame image to determine whether there is a frame image of the target vehicle's cabin in the subsequent frame image; if there is no frame image of the target vehicle's cabin in the frame image, the entry and exit detection of the target vehicle is output. The result is vehicle departure.

In some embodiments of the present application, if the target vehicle's compartment does not exist in the image after the frame image, then it is determined that the target vehicle's entry and exit detection result is the vehicle leaving the port; if the target vehicle's compartment exists in the image after the frame image, then the target vehicle's compartment is determined as The frame image of the compartment where the target vehicle exists is the starting frame, and the continuous frame images containing the target vehicle in the video image are extracted.

In the embodiment of the present application, after acquiring the video image of the target site, vehicle detection is performed on each frame of the video image, and by determining whether there is a compartment of the target vehicle in the subsequent frame image in which the compartment of the target vehicle does not exist, Determine the departure detection result of the target vehicle in the video image; and extract the entry and departure time information of the target vehicle from the recorded vehicle entry and exit data through the identification of the vehicle logo, and determine the departure detection result of the target vehicle in the video image based on the time information , providing a variety of exit detection methods for target vehicles in video images.

In some embodiments of the present application, after acquiring multiple consecutive transportation images of the target vehicle in the target site and the location information of the target vehicle in each transportation image, the location associated with the target site is obtained based on the site identifier of the target site. Depth relation data: obtain the target position information in the position depth relation data that matches the position information in each transportation image, and obtain the depth information of the target vehicle in each transportation image based on each target position information.

In some embodiments of the present application, in order to reduce the computational complexity of the vehicle entry and exit detection method, the target image corresponding to the target site can be collected when there are no vehicles and goods at the target site; wherein, the target image represents the absence of vehicles in the target site. and goods; perform depth estimation on the target image through the trained depth estimation network to obtain the depth information corresponding to at least one pixel in the target image; obtain the position information of at least one pixel in the target image, and convert the position of at least one pixel into The information is associated with the depth information corresponding to at least one pixel to obtain position-depth relationship data. Further, the location depth relationship data can be associated with the site identification of the target site.

As shown in Figure 5, Figure 5 is a schematic structural diagram of a depth estimation network provided by an embodiment of the present application. The depth estimation network shown includes a coding layer 401 and a depth estimation layer 402 connected in sequence.

Among them, the encoding layer 401 is constructed based on the pre-trained EfficientNet B5 encoder and the standard feature upsampling decoder. The target image is input to the encoding layer 401 in the depth estimation network for feature extraction to obtain decoding features; where the size of the target image is (H*W*3), the decoding feature output by the encoding layer 401 is a tensor of (H*W*Cd).

The encoding layer 401 inputs the decoding features to the depth estimation layer 402. The depth estimation layer 402 estimates depth information based on the input decoding features and outputs a depth image with a size of (H*W*1). As shown in Figure 6, Figure 6 provides a schematic structural diagram of the depth estimation layer. The depth estimation layer shown includes a conversion unit 501, a convolution unit 502, a depth unit 503 and an output unit 504. The output of the conversion unit 501 is two branches, one of which outputs a range attention map and inputs the range attention map to the convolution unit 502. The Softmax score of each pixel in the range attention map output by the convolution unit 502 is input to the output unit 504; the other branch outputs The unit width vector is input to the depth unit 503, the depth unit 503 calculates the depth unit center, and inputs the depth unit center to the output unit 504. The output unit 504 predicts the depth value of the pixel based on the Softmax score and the depth unit center of the pixel, and combines the depth values of each pixel to obtain a depth image corresponding to the target image.

In some embodiments of this application, in order to reduce the cost of the vehicle entry and exit detection method, the depth estimation network can be trained through transfer learning. Considering that there are many vehicles in the transfer yard, it is similar to the scene of the autonomous driving data set KITTI. Therefore, the AdaBins network trained on KITTI is selected as the pre-trained depth estimation network. Sample images of the target site are collected through the surveillance camera of the target site, and the adaptability test of the pre-trained depth estimation network is performed based on the sample images; if the pre-trained depth If the suitability test of the estimation network passes, the pre-trained depth estimation network is set to the trained depth estimation network; if the suitability test of the pre-trained depth estimation network fails, the pre-trained depth is adjusted based on the sample image The parameters of the network are estimated to obtain the trained depth estimation network. Specifically, as shown in Figure 7, the method for testing the adaptability of the pre-trained depth estimation network includes steps 601 to 604:

Step 601: Set multiple test points in the target site to obtain the true distance between each test point and the camera.

In some embodiments of this application, the real distance between each test point and the camera can be obtained through radar, laser, sonar or infrared ranging. Specifically, radar, laser, sonar or infrared equipment is deployed at the same position of the camera. Collect the true distance between each test point and the camera based on radar, laser, sonar or infrared equipment.

Step 602: Collect sample images corresponding to the target site after setting multiple test points, input the sample images into the pre-trained depth estimation network, and obtain test depth values corresponding to the multiple test points in the sample image.

In some embodiments of the present application, step 602 includes: collecting sample images corresponding to the target site after setting multiple test points, inputting the sample images into the pre-trained depth estimation network, and obtaining the predicted depth image corresponding to the sample image; based on the multiple test points The position coordinates of each test point in the sample image are obtained, the depth value of the pixel corresponding to the same position coordinate in the predicted depth image is obtained, and each depth value is set to the test depth value corresponding to each test point in the sample image.

Step 603: If the error between the test depth values corresponding to the multiple test points and the real distance is less than or equal to the preset error threshold, the pre-trained depth estimation network is set as the trained depth estimation network.

In some embodiments of the present application, if the error between the test depth value corresponding to each test point and the true distance is less than or equal to the preset error threshold, it means that the adaptability test of the pre-trained depth estimation network has passed, and the pre-trained The depth estimation network is set to the trained depth estimation network.

Step 604: If the error between the test depth values corresponding to multiple test points and the true distance is greater than the preset error threshold, adjust the parameters of the pre-trained depth estimation network according to the training loss value between the test depth value and the true distance until multiple The error between the test depth value corresponding to each test point and the real distance is less than or equal to the preset error threshold, and a trained depth estimation network is obtained.

In some embodiments of the present application, step 604 includes: If the error between the test depth value corresponding to at least one test point and the true distance is greater than the preset error threshold, it means that the pre-trained depth estimation network adaptability test fails, then Collect multiple sample images of the target scene to obtain a sample data set; input the sample images into the pre-trained depth estimation network to obtain a predicted depth image, and predict the test depth value of each test point in the depth image; calculate the test depth value according to the preset loss function The training loss value of the distance from the true distance, and the parameters of the pre-trained depth estimation network are adjusted according to the training loss. When the pre-trained depth estimation network reaches the prediction convergence condition, the trained depth estimation network is obtained. Among them, the preset loss function can be the mean square error function or the average absolute error loss function, which is understood as: analyzing the mean square between the test depth value of each test point in the sample image and the true distance corresponding to each test point according to the loss function Error, the mean square error is used as the training loss of the depth estimation network; according to the loss function, the average absolute error between the test depth value of each test point in the sample image and the true distance corresponding to each test point is analyzed, and the average absolute error is used as the depth estimate The training loss of the network. The prediction convergence condition can be that the training loss value is less than or equal to the threshold, or the number of training iterations reaches the prediction number.

In some embodiments of the present application, step 604 also includes: if the error between the test depth value corresponding to at least one test point and the true distance is greater than the preset error threshold, it means that the pre-trained depth estimation network adaptability test fails, Then deploy a depth camera, collect sample images of multiple target scenes and the real depth images corresponding to the sample images through the depth camera to form a training sample data set; input each sample image into the pre-trained depth estimation network to obtain the predicted depth image; through the preset The loss function, the predicted depth image corresponding to each sample image and the real sample image corresponding to each sample image determine the training loss of the pre-trained depth estimation network, and iteratively adjust the parameters of the pre-trained depth estimation network based on the training loss. When the pre-trained depth When the estimation network reaches the preset convergence condition, the training is stopped and the trained depth estimation network is obtained. Among them, the preset loss function can be a mean square error function or a mean absolute error loss function, which is understood as: analyzing the mean square error between the predicted depth image corresponding to the sample image and the real depth image corresponding to the sample image according to the loss function, and The mean square error is used as the training loss of the depth estimation network; the average absolute error between the predicted depth image corresponding to the sample image and the real depth image corresponding to the sample image is analyzed according to the loss function, and the average absolute error is used as the training loss of the depth estimation network; prediction The convergence condition can be that the training loss value is less than or equal to the threshold, or the number of training iterations reaches the predicted number. Among them, the depth camera is detachable, and the installation height and angle of the depth camera are similar to the height and angle of the camera in each target scene.

In the embodiment of this application, when there are no vehicles or goods in the target scene, the target image is collected, and the trained depth estimation network is run to obtain the position-depth relationship data of the target scene. When subsequently obtaining depth information, only the position-depth relationship data needs to be queried. , reducing the computational complexity of the algorithm; and when training the depth estimation network, the depth estimation network is trained through transfer learning to obtain a pre-trained depth estimation network. When the pre-trained depth estimation network needs to be deployed to different target sites, only It requires adaptability testing and no further training, and can be copied to other staging areas, which solves the problem of previous classification-based visual solutions that require continuous data addition.

In some embodiments of the present application, after determining the corresponding depth information of the target vehicle in multiple consecutive transportation images, the transportation images are sorted according to the time sequence of each transportation image to obtain multiple sorted transportation images. , obtain the depth information of the target vehicle in the multiple sorted transportation images based on the sorted multiple transportation images, and compare the depth information of the target vehicle in the multiple sorted transportation images with the preset distance threshold respectively, and get The changing trend of the depth information of the target vehicle.

In some embodiments of the present application, the corresponding depth information of the target vehicle in the multiple sorted transportation images can be compared with the preset distance threshold to obtain a difference sequence, and the depth of the target vehicle can be obtained based on the difference sequence. Information trends. In some embodiments of the present application, each difference value in the difference sequence can be forward differentiated, and the change trend of the depth information of the target vehicle can be determined based on the difference result. For example, when the difference result is less than 0, it means that the difference value corresponding to the next moment is less than the difference value corresponding to the previous moment, and the change trend of the depth information of the target vehicle is decreasing; when the difference result is greater than 0, it means that the difference value corresponding to the next moment moment is decreasing. The difference is greater than the corresponding difference at the previous moment, then the change trend of the depth information of the target vehicle is increasing. For example, taking the preset distance threshold of 1 meter as an example, when the depth information of the target vehicle in the sorted transportation image is {9, 6, 5, 3, 2}, the target vehicle will be The depth information in the transportation image is compared with the preset distance threshold to obtain the difference sequence {8, 5, 2, 1}. Since the difference sequence {8, 5, 2, 1} the difference result is {-3, - 3, -1} are both less than 0, then the change trend of the depth information of the target vehicle is decreasing.

In order to improve the accuracy of the vehicle's entry and exit detection results, after determining the target vehicle's change trend based on the depth information, the target vehicle's entry and exit detection results can be determined based on the difference sequence, specifically including: If the depth information change trend is decreasing, and the difference If the time period corresponding to the difference value in the value sequence that is less than or equal to the preset difference threshold is greater than the preset time length, the target vehicle's entry and exit detection result is port arrival; if the change trend of the depth information is increasing, and the target vehicle is in the sorted Among the depth information in multiple transportation images, if there is target depth information greater than the preset value, the target vehicle's entry and exit detection result is port departure.

In some embodiments of the present application, after determining the corresponding depth information of the target vehicle in multiple consecutive transportation images, the depth information of the target vehicle in each transportation image can be determined based on the time sequence corresponding to the multiple consecutive transportation images. The depth information is sorted, and the depth information of the target vehicle in each sorted transportation image is differentiated according to the difference sequence difference method to obtain the depth difference result. The change trend of the depth information of the target vehicle is determined based on the depth difference result; if the depth The information change trend is decreasing, and the depth information in each sorted transportation image contains at least one target depth information that is less than or equal to the preset depth information, then the entry and exit detection result of the target vehicle is port arrival. According to the sorted each transportation The time information in the image records the arrival time of the target vehicle; if the change trend of the depth information is increasing, and the depth information in each sorted transportation image contains target depth information greater than the preset value, the detection result of the target vehicle's entry into the port will be To leave the port, the departure time of the target vehicle is recorded based on the time information in each sorted transportation image.

The embodiment of this application uses depth information to detect vehicle entry and exit, and changes the vehicle entry and exit detection from qualitative analysis to quantitative calculation, thereby improving the accuracy of vehicle entry and exit detection.

In order to better implement the vehicle entry and exit detection method provided by the embodiment of the present application, based on the vehicle entry and exit detection method, the embodiment of the present application provides a vehicle entry and exit detection device, as shown in Figure 8. Figure 8 is the implementation of the present application. The example provides a structural schematic diagram of an embodiment of a vehicle entry and exit detection device. The vehicle entry and exit detection device shown includes:

The acquisition module 701 is used to acquire multiple continuous transportation images of the target vehicle in the target site, and the corresponding position information of the target vehicle in each transportation image of the multiple continuous transportation images;

Depth determination module 702 is used to determine the depth information corresponding to the target vehicle in the transportation image based on the position depth relationship data associated with the target site and the location information; the position depth relationship data includes each location information and each location information in the target site. Corresponding depth information, which represents the distance between the point and the camera that collects the transportation image when the point corresponding to the position information in the transportation image is mapped to the target site;

The depth trend determination module 703 is used to obtain the depth information change trend of the target vehicle based on the depth information corresponding to multiple consecutive transportation images;

The detection module 704 is used to obtain the entry and exit detection results of the target vehicle based on the depth information change trend.

In some embodiments of this application, the vehicle entry and exit detection device also includes:

Depth estimation module 705 is used to collect the target image corresponding to the target site; the target image is an image without a vehicle in the target site; perform depth estimation on the target image through the trained depth estimation network to obtain at least one pixel corresponding to the target image Depth information; obtain the position information of at least one pixel in the target image, associate the position information of at least one pixel with the depth information corresponding to at least one pixel, and obtain position-depth relationship data.

In some embodiments of the present application, the depth estimation module 705 is also used to:

Set up multiple test points in the target site;

Collect sample images corresponding to the target site after setting multiple test points;

Input the sample image into the pre-trained depth estimation network to obtain the test depth values corresponding to multiple test points in the sample image;

If the error between the test depth value corresponding to one test point among multiple test points and the real distance corresponding to one test point is greater than the preset error threshold, then the test depth value corresponding to one test point and the true distance corresponding to one test point will be The training loss value between the real distances is used to adjust the parameters of the pre-trained depth estimation network until the errors corresponding to multiple test points are less than or equal to the preset error threshold, and the trained depth estimation network is obtained; among them, the real distance represents the test points The actual distance from the camera.

In some embodiments of the present application, the depth trend determination module 703 is used to:

Sort multiple consecutive transportation images according to the time sequence corresponding to the multiple consecutive transportation images, and obtain the sorted multiple transportation images and the corresponding depth information of the target vehicle in the sorted multiple transportation images;

Compare the corresponding depth information of the target vehicle in the sorted multiple transportation images with the preset distance threshold to obtain the change trend of the depth information of the target vehicle.

Difference the corresponding depth information of the target vehicle in the multiple sorted transportation images with the preset distance threshold to obtain a difference sequence;

The depth information change trend of the target vehicle is obtained based on the difference sequence.

In some embodiments of this application, the detection module 704 is also used to:

If the change trend of the depth information is decreasing, and the time period in the difference sequence that is less than or equal to the preset difference threshold is greater than the preset time, the target vehicle's entry and exit detection result is port arrival;

If the change trend of the depth information is increasing, and there is target depth information greater than the preset value in the corresponding depth information of the target vehicle in the sorted multiple transportation images, then the detection result of the target vehicle's entry and exit is port departure.

In the time period corresponding to multiple consecutive transportation images, compare the depth information in the transportation image corresponding to the target vehicle at each moment in the time period with the depth information in the transportation image corresponding to the next moment;

The adjustment module 706 is configured to obtain the first residence time of the target vehicle at the target site based on the time difference between the departure time and the entry time in the entry and exit detection results; obtain at least one second residence time, Wherein, the at least one second residence time is the residence time of the target vehicle at at least one subsequent site of the target site; determine the maximum residence time of the first residence time and the at least one second residence time. ; Adjust the transportation route and transportation volume of the target vehicle according to the maximum detention time, the number of the maximum detention time, and the venue corresponding to the maximum detention time.

In some embodiments of this application, the acquisition module 701 is used for:

Obtain the video image of the target site, extract the continuous frame images containing the target vehicle in the video image, and set the continuous frame images contained in the video image as multiple continuous transportation images of the target vehicle in the target site;

Perform target detection on each transportation image in multiple consecutive transportation images to obtain the bounding box coordinates of the target vehicle in the transportation image;

According to the bounding box coordinates, calculate the midpoint coordinates of the target vehicle's bounding box;

The corresponding midpoint coordinates of the target vehicle in each of the multiple continuous transportation images are set as the corresponding position information of the target vehicle in each of the multiple continuous transportation images.

Obtain the video image of the target site, perform vehicle detection on each frame of the video image, and obtain the detection result of the target vehicle in each frame of image; the detection result indicates whether the target vehicle exists in the frame image;

Taking the frame image in which the target vehicle exists as the starting frame, extract the continuous frame images containing the target vehicle in the video image.

If the average pixel value of the second area in the difference image corresponding to the second image whose pixel value is greater than the preset pixel value is greater than the preset average value, it is determined that the target vehicle exists in the video image.

The embodiment of the present application determines the depth information of the target vehicle through the position-depth relationship data associated with the target site and based on the position information of multiple consecutive transportation images in the target site, and determines the arrival and departure port detection results of the target vehicle based on the depth value. Reduce the amount of calculations in the detection process and improve detection efficiency; and obtain the depth information change trend of the target vehicle through the depth information of multiple consecutive transportation images of the target vehicle at the target site, and determine the entry and exit detection results of the target vehicle based on the depth change trend , adjust the inspection at the entry and exit ports from qualitative analysis to quantitative calculation, and improve the accuracy of the inspection at the entry and exit ports through data.

The embodiment of the present application also provides a vehicle entry and exit detection equipment, as shown in Figure 9, which shows a schematic structural diagram of the vehicle entry and exit detection equipment involved in the embodiment of the application. Specifically:

The vehicle entry and exit detection device may include components such as a processor 801 of one or more processing cores, a memory 802 of one or more computer-readable storage media, a power supply 803, an input unit 804 and other components. Those skilled in the art can understand that the structure of the vehicle entry and exit detection equipment shown in Figure 8 does not constitute a limitation on the vehicle entry and exit detection equipment. It may include more or fewer components than shown in the figure, or combine certain components. Or a different component arrangement. in:

The processor 801 is the control center of the vehicle entry and exit detection equipment, using various interfaces and lines to connect various parts of the entire vehicle entry and exit detection equipment, by running or executing software programs and/or modules stored in the memory 802, and calling The data stored in the memory 802 executes various functions of the vehicle entry and exit detection equipment and processes data, thereby overall monitoring the vehicle entry and exit detection equipment. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, where the application processor mainly processes operating systems, user interfaces, application programs, etc. , the modem processor mainly handles wireless communications. It can be understood that the above modem processor may not be integrated into the processor 801.

The memory 802 can be used to store software programs and modules, and the processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802 . The memory 802 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the storage data area may store data according to Data created by the use of vehicle entry and exit inspection equipment, etc. In addition, memory 802 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802 .

The vehicle entry and exit detection equipment also includes a power supply 803 that supplies power to various components. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, thereby realizing functions such as charging, discharging, and power consumption management through the power management system. The power supply 803 may also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

The vehicle entry and exit detection equipment may also include an input unit 804, which may be used to receive input numeric or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. .

Although not shown, the vehicle entry and exit detection device may also include a display unit, etc., which will not be described again here. Specifically, in this embodiment, the processor 801 in the vehicle entry and exit detection equipment will load the executable files corresponding to the processes of one or more application programs into the memory 802 according to the following instructions, and the processor 801 will Run the application program stored in the memory 802 to implement various functions, as follows:

Obtain multiple continuous transportation images of the target vehicle in the target site, and the corresponding position information of the target vehicle in each of the multiple continuous transportation images;

Determine the depth information of the target vehicle in the transportation image according to the position-depth relationship data and position information associated with the target site; the position-depth relationship data includes each position information in the target site and the depth information corresponding to the position information, and the depth information represents the transportation image When the point corresponding to the location information in is mapped to the target site, the distance between the point and the camera that collects the transportation image;

According to the corresponding depth information of multiple consecutive transportation images, the changing trend of the depth information of the target vehicle is determined;

According to the changing trend of the depth information, the entry and exit detection results of the target vehicle are obtained.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling relevant hardware through instructions. The instructions can be stored in a computer-readable storage medium, and loaded and executed by the processor.

To this end, embodiments of the present application provide a storage medium in which a plurality of instructions are stored, and the instructions can be loaded by the processor to execute the steps in any vehicle entry and exit detection method provided by the embodiments of the present application. For example, this command can perform the following steps:

For the specific implementation of each of the above operations, please refer to the previous embodiments and will not be described again here.

Among them, the storage medium may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

Since the instructions stored in the storage medium can execute the steps in any vehicle entry and exit detection method provided by the embodiments of this application, therefore, any vehicle entry and exit detection method provided by the embodiments of this application can be implemented. The beneficial effects that can be achieved are detailed in the previous embodiments and will not be described again here.

The above is a detailed introduction to the vehicle entry and exit detection method, device, equipment and storage medium provided by the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation modes of the present invention. The description of the above embodiments It is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, there will be changes in the specific implementation and application scope according to the idea of the present invention. To sum up, the present invention The content of the description should not be construed as limiting the invention.

Claims

A vehicle entry and exit detection method, characterized in that the method includes:

Obtaining multiple continuous transportation images of the target vehicle in the target site, and corresponding position information of the target vehicle in each of the multiple continuous transportation images;

Determine the depth information corresponding to the target vehicle in the transportation image according to the position-depth relationship data associated with the target site and the position information; the position-depth relationship data includes each position information in the target site and Depth information corresponding to each position information. The depth information represents the distance between the point corresponding to the position information in the transportation image and the camera that collects the transportation image when it is mapped to the target site;

According to the depth information corresponding to the multiple consecutive transportation images, the depth information change trend of the target vehicle is obtained;

According to the change trend of the depth information, the entry and exit detection results of the target vehicle are obtained.
The vehicle entry and exit detection method according to claim 1, characterized in that, based on the position depth relationship data associated with the target site and the position information, it is determined that the corresponding position of the target vehicle in the transportation image is Before depth information, also includes:

Collect a target image corresponding to the target site; the target image is an image in which no vehicle exists in the target site;

Perform depth estimation on the target image through a trained depth estimation network to obtain depth information corresponding to at least one pixel in the target image;

Obtain the position information of the at least one pixel point in the target image, associate the position information of the at least one pixel point with the depth information corresponding to the at least one pixel point, and obtain the position-depth relationship data.
The vehicle entry and exit detection method according to claim 2, characterized in that, during the depth estimation of the target image through the trained depth estimation network, the depth information corresponding to at least one pixel in the target image is obtained. Previously, this also included:

Set up multiple test points in the target site;

Collect sample images corresponding to the target site after setting the plurality of test points;

Input the sample image into a pre-trained depth estimation network to obtain test depth values corresponding to the multiple test points in the sample image;

If the error between the test depth value corresponding to one of the plurality of test points and the true distance corresponding to the one test point is greater than the preset error threshold, then according to the test depth value corresponding to the one test point The training loss value between the real distance corresponding to the one test point, adjusting the parameters of the pre-trained depth estimation network until the errors corresponding to the multiple test points are less than or equal to the preset error threshold, we get The trained depth estimation network; wherein the real distance represents the real distance between the test point and the camera.
The vehicle entry and exit detection method according to claims 1 to 3, characterized in that the depth information change trend of the target vehicle is obtained based on the depth information corresponding to the multiple consecutive transportation images, including:

Compare the depth information of the target vehicle in the multiple consecutive transportation images with the preset distance threshold respectively to obtain the change trend of the depth information of the target vehicle; or

According to the difference between the depth information of the target vehicle in the multiple consecutive transportation images, the change trend of the depth information of the target vehicle is obtained.
The vehicle entry and exit detection method according to claim 4, wherein the depth information of the target vehicle in the plurality of consecutive transportation images is compared with a preset distance threshold to obtain the The changing trend of the depth information of the target vehicle includes:

Sort the multiple consecutive transportation images according to the time order corresponding to the multiple consecutive transportation images, and obtain the multiple sorted transportation images and the position of the target vehicle in the sorted multiple transportation images. Corresponding depth information;

Compare the corresponding depth information of the target vehicle in the sorted plurality of transportation images with the preset distance threshold to obtain the change trend of the depth information of the target vehicle.
The vehicle entry and exit detection method according to claim 5, wherein the corresponding depth information of the target vehicle in the sorted plurality of transportation images is compared with the preset distance threshold, Obtain the depth information change trend of the target vehicle, including:

Difference the corresponding depth information of the target vehicle in the plurality of sorted transportation images with the preset distance threshold to obtain a difference sequence;

The depth information change trend of the target vehicle is obtained according to the difference sequence.
The vehicle entry and exit detection method according to claim 6, characterized in that, after obtaining the depth information change trend of the target vehicle according to the difference sequence, it further includes:

If the change trend of the depth information is decreasing, and the time period in the difference sequence that is less than or equal to the preset difference threshold is greater than the preset time length, then the entry and exit detection result of the target vehicle is port arrival;

If the change trend of the depth information is increasing, and the corresponding depth information of the target vehicle in the plurality of sorted transportation images contains target depth information greater than the preset value, then the entry and exit of the target vehicle The port test result is outbound.
The vehicle entry and exit detection method according to claim 4, wherein the depth of the target vehicle is obtained based on the difference between the depth information of the target vehicle in the multiple consecutive transportation images. Information trends include:

In the time period corresponding to the multiple consecutive transportation images, the depth information in the transportation image corresponding to the target vehicle at each moment in the time period is compared with the depth information in the transportation image corresponding to the next moment. information for comparison;

If the depth information corresponding to each moment is greater than the depth information corresponding to the next moment, it is determined that the change trend of the depth information of the target vehicle is increasing;

If the depth information corresponding to each moment is less than the depth information corresponding to the next moment, it is determined that the change trend of the depth information of the target vehicle is decreasing.
The vehicle entry and exit detection method according to any one of claims 1 to 8, characterized in that, after obtaining the entry and exit detection results of the target vehicle, it further includes:

According to the time difference between the departure time and the arrival time in the entry and exit detection results, the first residence time of the target vehicle at the target site is obtained;

Obtain at least one second residence time, wherein the at least one second residence time is the residence time of the target vehicle in at least one subsequent site of the target site;

Determining the maximum residence time among the first residence time and the at least one second residence time;

The transportation route and transportation volume of the target vehicle are adjusted according to the maximum detention time, the number of the maximum detention time, and the site corresponding to the maximum detention time.
The vehicle entry and exit detection method according to any one of claims 1 to 9, characterized in that the acquisition of multiple continuous transportation images of the target vehicle in the target site, and the target vehicle in the multiple continuous transportation images The corresponding location information in each transportation image in the transportation image includes:

Obtain the video image of the target site, extract the continuous frame images containing the target vehicle in the video image, and set the continuous frame images as multiple continuous transportation images of the target vehicle in the target site. ;

Perform target detection on each transportation image in the multiple consecutive transportation images to obtain the bounding box coordinates of the target vehicle in the transportation image;

Calculate the midpoint coordinates of the bounding box of the target vehicle according to the bounding box coordinates;

The midpoint coordinates of the target vehicle corresponding to each transportation image in the multiple continuous transportation images are set to the corresponding midpoint coordinates of the target vehicle in each transportation image of the multiple continuous transportation images. location information.
The vehicle entry and exit detection method according to claim 10, characterized in that said obtaining the video image of the target site and extracting the continuous frame images containing the target vehicle in the video image includes:

Obtain the video image of the target site, perform vehicle detection on each frame of the video image, and obtain the detection result of the target vehicle in each frame of image; the detection result represents whether the target exists in the frame image vehicle;

Taking the frame image in which the target vehicle exists as the starting frame, extract consecutive frame images containing the target vehicle in the video image.
The method of claim 11, wherein performing vehicle detection on each frame of the video image includes:

Set the collected image when there is no vehicle in the target site as the reference image;

Difference the first image in the video image and the reference image to obtain a difference image;

If there is a first region with a pixel value greater than the preset pixel value in the difference image, and the average pixel value of the first region is greater than the preset average value, then it is determined that the third image in the video image is located after the first image. The difference image corresponding to the two images;

If the average pixel value of the second area in the differential image corresponding to the second image is greater than the preset pixel value, it is determined that the target vehicle exists in the video image.
A vehicle entry and exit detection device, characterized in that the device includes:

An acquisition module, configured to acquire multiple continuous transportation images of the target vehicle in the target site, and the corresponding position information of the target vehicle in each of the multiple continuous transportation images;

Depth determination module, configured to determine the depth information corresponding to the target vehicle in the transportation image according to the position-depth relationship data associated with the target site and the location information; the position-depth relationship data includes the target site Each position information in the transportation image and the depth information corresponding to each position information. The depth information represents the distance between the point and the camera that collects the transportation image when the point corresponding to the position information in the transportation image is mapped to the target site. distance;

A depth trend determination module, configured to obtain the depth information change trend of the target vehicle based on the depth information corresponding to the multiple consecutive transportation images;

A detection module is used to obtain the entry and exit detection results of the target vehicle according to the change trend of the depth information.
A vehicle entry and exit detection equipment, characterized in that it includes a memory and a processor; the memory stores an application program, and the processor is used to run the application program in the memory to execute any one of claims 1 to 12 Operations in the vehicle entry and exit detection method.
A storage medium, characterized in that the storage medium stores a plurality of instructions, and the instructions are suitable for loading by the processor to execute the steps in the vehicle entry and exit detection method according to any one of claims 1 to 12 .