CN110738169B - Traffic flow monitoring method, device, equipment and computer readable storage medium - Google Patents

Abstract

The invention discloses a traffic flow monitoring method, a device, equipment and a computer readable storage medium, wherein the method comprises the following steps: collecting a first remote sensing image of a detection area, and collecting a second remote sensing image of the detection area after a preset interval time; extracting a first region of interest from the first remote sensing image and a second region of interest from the second remote sensing image through the trained semantic segmentation model; respectively carrying out target detection on the first region of interest and the second region of interest through a remote sensing target detection model which is completed through training so as to acquire first vehicle information in the first region of interest and second vehicle information in the second region of interest; and determining traffic flow information of the detection area in the preset interval time according to the first vehicle information and the second vehicle information. The invention improves the convenience and accuracy of traffic flow monitoring.

Description

Traffic flow monitoring method, device, equipment and computer-readable storage medium

Technical field

The invention relates to the field of intelligent transportation technology, and in particular to a traffic flow monitoring method, device, equipment and computer-readable storage medium.

Background technique

With the development of society and the advancement of science and technology, people's living standards have been greatly improved, car ownership has increased significantly, and traffic congestion has become increasingly serious. Therefore, how to effectively manage traffic is very important. Among them, traffic flow is Important traffic parameters in traffic management, traditional traffic flow monitoring technology, such as the ring coil method, requires the installation of coil sensors on the road surface, which causes damage to the road surface, is inconvenient in construction and installation, and can only sense the passing of the coil sensor installed Fixed-position vehicles are not only less convenient but also less accurate.

Contents of the invention

The main purpose of the present invention is to provide a traffic flow monitoring method, device, equipment and computer-readable storage medium, aiming to solve the technical problems of poor convenience and low accuracy of existing traffic flow monitoring technology.

To achieve the above objectives, the present invention provides a traffic flow monitoring method, which method includes the following steps:

Collect the first remote sensing image of the detection area, and collect the second remote sensing image of the detection area after a preset interval;

Using the trained semantic segmentation model, a first region of interest is extracted from the first remote sensing image and a second region of interest is extracted from the second remote sensing image;

Through the trained remote sensing target detection model, target detection is performed on the first area of interest and the second area of interest respectively to obtain the first vehicle information and the second area of interest in the first area of interest. Second vehicle information in the area of interest;

According to the first vehicle information and the second vehicle information, the traffic flow information of the detection area within the preset interval is determined.

Optionally, for the semantic segmentation model completed through training, the steps of respectively extracting the first region of interest from the first remote sensing image and extracting the second region of interest from the second remote sensing image include:

Input the first remote sensing image to the trained semantic segmentation model to identify the road skeleton from the first remote sensing image as the first region of interest;

The second remote sensing image is input to the trained semantic segmentation model to identify the road skeleton from the second remote sensing image as the second region of interest.

Optionally, the remote sensing target detection model completed through training performs target detection on the first area of interest and the second area of interest respectively to obtain the first vehicle in the first area of interest. The steps of obtaining information and second vehicle information in the second area of interest include:

The first region of interest is input into the trained remote sensing target detection model for target detection to identify the first vehicle and its type from the first region of interest, and the second region of interest is The area is input into the trained remote sensing target detection model for target detection to identify the second vehicle and its type from the second area of interest;

Determine the coordinate information of the first vehicle in the first remote sensing image, and count the total number of first vehicles, and determine the coordinate information of the second vehicle in the second remote sensing image, and count the total number of second vehicles;

The coordinate information and type of the first vehicle and the total number of first vehicles are determined as first vehicle information, and the coordinate information and type of the second vehicle and the total number of second vehicles are determined as second vehicle information.

Optionally, the step of determining the traffic flow information of the detection area within the preset interval based on the first vehicle information and the second vehicle information includes:

Calculate the center coordinates of the first vehicle in the first remote sensing image based on the coordinate information of the first vehicle, and calculate the center coordinates of the second vehicle in the second remote sensing image based on the coordinate information of the second vehicle coordinate;

Calculate the traffic speed of the detection area within the preset interval according to the center coordinate of the first vehicle and the center coordinate of the second vehicle;

Compare the type of the first vehicle with the type of the second vehicle, and the total number of the first vehicle with the total number of the second vehicle respectively to obtain the traffic flow change amount;

The traffic flow change amount and the calculated traffic flow speed are determined as the traffic flow information of the detection area within the preset interval.

Optionally, the step of calculating the traffic speed of the detection area within the preset interval based on the center coordinates of the first vehicle and the center coordinates of the second vehicle includes:

According to the center coordinates of the first vehicle, the overall center coordinates of all first vehicles in the first remote sensing image are calculated, and based on the center coordinates of the second vehicle, all second remote sensing images in the second remote sensing image are calculated. The overall center coordinates of the vehicle;

Obtain the zoom ratio of the first remote sensing image or the second remote sensing image;

According to the overall center coordinate of the first vehicle and the overall center coordinate of the second vehicle, and the scaling ratio, the traffic flow speed of the detection area within the preset interval is calculated.

Optionally, before the step of collecting the first remote sensing image of the detection area and collecting the second remote sensing image of the detection area after a preset interval, the step includes:

Train the semantic segmentation model to obtain the trained semantic segmentation model, and train the remote sensing target detection model to obtain the trained remote sensing target detection model.

Optionally, the traffic flow speed of the detection area within the preset interval is calculated based on the overall center coordinate of the first vehicle and the overall center coordinate of the second vehicle, and the scaling ratio. The steps include:

Calculate the traffic flow speed based on the remote sensing image according to the overall center coordinate of the first vehicle and the overall center coordinate of the second vehicle;

Calculate the product of the traffic flow speed based on the remote sensing image and the scaling ratio to obtain the traffic flow speed of the detection area within the preset interval.

In addition, to achieve the above object, the present invention also provides a vehicle flow monitoring device, which includes:

An acquisition module, configured to collect the first remote sensing image of the detection area, and collect the second remote sensing image of the detection area after a preset interval;

An extraction module, configured to extract a first region of interest from the first remote sensing image and a second region of interest from the second remote sensing image using the semantic segmentation model completed through training;

A detection module, configured to perform target detection on the first area of interest and the second area of interest through the trained remote sensing target detection model to obtain the first vehicle information in the first area of interest. and second vehicle information in the second area of interest;

A determining module, configured to determine the traffic flow information of the detection area within the preset interval according to the first vehicle information and the second vehicle information.

In addition, to achieve the above object, the present invention also provides a traffic flow monitoring device, which includes a processor, a memory, and visualization of traffic data stored on the memory and executed by the processor. Program, wherein when the traffic flow monitoring program is executed by the processor, the steps of the traffic flow monitoring method as described above are implemented.

In addition, in order to achieve the above object, the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a traffic flow monitoring program. When the traffic flow monitoring program is executed by a processor, the above-mentioned steps are implemented. Steps of the traffic flow monitoring method.

The invention provides a traffic flow monitoring method, device, equipment and computer-readable storage medium, which collects the first remote sensing image of the detection area, and collects the second remote sensing image of the detection area after a preset interval; it is completed through training The semantic segmentation model extracts the first region of interest from the first remote sensing image and the second region of interest from the second remote sensing image respectively; through the trained remote sensing target detection model, the first region of interest is extracted from the first remote sensing image and the second region of interest is extracted from the second remote sensing image respectively. performing target detection on a region of interest and the second region of interest to obtain first vehicle information in the first region of interest and second vehicle information in the second region of interest; according to the first One vehicle information and the second vehicle information determine the traffic flow information of the detection area within the preset interval. The present invention analyzes the remote sensing images of the detection area through the trained semantic segmentation model and remote sensing target detection model, provides a more detailed analysis basis for the monitoring of traffic flow, and improves the convenience and accuracy of traffic flow monitoring.

Description of the drawings

Figure 1 is a schematic diagram of the hardware structure of the traffic flow monitoring equipment involved in the embodiment of the present invention;

Figure 2 is a schematic flow chart of the first embodiment of the traffic flow monitoring method of the present invention;

Figure 3 is an example flow chart of the implementation of the first embodiment of the traffic flow monitoring method of the present invention;

Figure 4 is a functional module schematic diagram of the first embodiment of the traffic flow monitoring device of the present invention.

The realization of the purpose, functional features and advantages of the present invention will be further described with reference to the embodiments and the accompanying drawings.

Detailed ways

It should be understood that the specific embodiments described here are only used to explain the present invention and are not intended to limit the present invention.

The main solution of the embodiment of the present invention is to: collect the first remote sensing image of the detection area, and collect the second remote sensing image of the detection area after a preset interval; through the semantic segmentation model completed through training, respectively from the first remote sensing image of the detection area. Extract the first area of interest from a remote sensing image and extract the second area of interest from the second remote sensing image; through the trained remote sensing target detection model, respectively detect the first area of interest and the second sensing area. Perform target detection in the area of interest to obtain first vehicle information in the first area of interest and second vehicle information in the second area of interest; according to the first vehicle information and the second vehicle information , determine the traffic flow information of the detection area within the preset interval. To solve the technical problems of poor convenience and low accuracy of existing traffic flow monitoring technology.

As shown in Figure 1, Figure 1 is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present invention.

The traffic flow monitoring method involved in the embodiment of the present invention can be implemented by a traffic flow monitoring device. The traffic flow monitoring device can be a PC, a server, or other equipment with data processing functions.

Referring to Figure 1, Figure 1 is a schematic diagram of the hardware structure of the traffic flow monitoring equipment involved in the embodiment of the present invention. In the embodiment of the present invention, the traffic flow monitoring device may include a processor 1001 (such as a central processing unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize connection and communication between these components; the user interface 1003 can include a display screen (Display) and an input unit such as a keyboard (Keyboard); the network interface 1004 can optionally include a standard wired interface and a wireless interface. (such as WI-FI interface); the memory 1005 can be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 can optionally be a storage device independent of the aforementioned processor 1001 . Those skilled in the art can understand that the hardware structure shown in Figure 1 does not limit the present invention, and may include more or fewer components than shown, or combine certain components, or arrange different components.

Continuing to refer to FIG. 1 , the memory 1005 as a readable storage medium in FIG. 1 may include an operating system, a network communication module, and a traffic flow monitoring program. In Figure 1, the network communication module is mainly used to connect to the server and perform data communication with the server; and the processor 1001 can call the traffic flow monitoring program stored in the memory 1005 and execute the traffic flow monitoring method provided by the embodiment of the present invention.

The embodiment of the present invention provides a traffic flow monitoring method.

Referring to Figure 2, Figure 2 is a schematic flow chart of the first embodiment of the traffic flow monitoring method of the present invention.

In this embodiment, the traffic flow monitoring method is implemented by traffic flow monitoring equipment. The traffic flow monitoring equipment can be a terminal device such as a PC or a server, and can optionally be the equipment shown in Figure 1. The traffic flow monitoring equipment and the remote sensing equipment have an established relationship. Communication connection can control remote sensing equipment. The traffic flow monitoring method includes the following steps:

Step S10, collect the first remote sensing image of the detection area, and collect the second remote sensing image of the detection area after a preset interval;

Step S20: extract a first region of interest from the first remote sensing image and a second region of interest from the second remote sensing image through the trained semantic segmentation model;

Step S30: Using the trained remote sensing target detection model, perform target detection on the first area of interest and the second area of interest respectively to obtain the first vehicle information and the first vehicle information in the first area of interest. second vehicle information in the second area of interest;

Step S40: Determine the traffic volume information of the detection area within the preset interval based on the first vehicle information and the second vehicle information.

In this embodiment, the detection area can be any road section. Set up remote sensing equipment in the detection area in advance, and establish a communication connection between the traffic flow monitoring equipment and the remote sensing equipment. Then, the traffic flow monitoring equipment can control the remote sensing equipment to collect remote sensing images of the detection area at any time. Then, the traffic flow monitoring equipment can pass the training The completed semantic segmentation model and remote sensing target detection model analyze the collected remote sensing images to provide a more detailed analysis basis for traffic flow monitoring, thus improving the convenience and accuracy of traffic flow monitoring.

Step S10, collect the first remote sensing image of the detection area, and collect the second remote sensing image of the detection area after a preset interval;

Since in practice, in two frames of images with a short imaging time, the background is almost unchanged, and the changing part is caused by the moving vehicle. Therefore, this embodiment is based on physical kinematics and analyzes two frames separated by a preset time. Remote sensing images are used to obtain traffic flow information.

Specifically, the traffic flow monitoring device can send remote sensing image collection instructions to the remote sensing device in real time or regularly, control the remote sensing device to collect a frame of remote sensing image (defined as the first remote sensing image) of the detection area, and control the remote sensing device to collect a remote sensing image after a preset interval. Then collect another frame of remote sensing image of the detection area (defined as the second remote sensing image). The preset interval time can be flexibly set according to actual needs, and it can be shorter. The collection time of the first remote sensing image is defined as T1, and the collection time of the second remote sensing image is defined as T2. It can be understood that T1<T2.

Step S20: extract a first region of interest from the first remote sensing image and a second region of interest from the second remote sensing image through the trained semantic segmentation model;

Afterwards, through the trained semantic segmentation model, the first region of interest is extracted from the first remote sensing image and the second region of interest is extracted from the second remote sensing image. As an implementation manner, step S20 includes:

A. Input the first remote sensing image to the trained semantic segmentation model to identify the road skeleton from the first remote sensing image as the first region of interest;

B. Input the second remote sensing image to the trained semantic segmentation model to identify the road skeleton from the second remote sensing image as the second region of interest.

That is, the first remote sensing image collected at time T1 is input into the trained semantic segmentation model for analysis, so as to extract the road skeleton from the first remote sensing image collected at time T1 as the first area of interest, and, collect the first remote sensing image at time T2 The second remote sensing image is input into the semantic segmentation model for analysis to extract the road skeleton from the second remote sensing image collected at time T2 as the second region of interest.

Step S30: Using the trained remote sensing target detection model, perform target detection on the first area of interest and the second area of interest respectively to obtain the first vehicle information and the first vehicle information in the first area of interest. second vehicle information in the second area of interest;

After that, the trained remote sensing target detection model is used to perform target detection on the first area of interest and the second area of interest respectively to obtain the first vehicle information in the first area of interest and the first vehicle information in the second area of interest. Second vehicle information. Specifically, step S30 includes:

C. Input the first area of interest into the trained remote sensing target detection model for target detection to identify the first vehicle and its type from the first area of interest, and add the second The area of interest is input into the trained remote sensing target detection model for target detection to identify the second vehicle and its type from the second area of interest;

D. Determine the coordinate information of the first vehicle in the first remote sensing image, and count the total number of first vehicles, and determine the coordinate information of the second vehicle in the second remote sensing image, and count the number of the second vehicle. total;

E. Determine the coordinate information and type of the first vehicle and the total number of first vehicles as the first vehicle information, and determine the coordinate information and type of the second vehicle and the total number of second vehicles as the second vehicle information.

That is, the first area of interest in the first remote sensing image collected at time T1 and the second area of interest in the second remote sensing image collected at time T2 are respectively input to the trained remote sensing target detection model for target detection, so as to Vehicles and their corresponding types are respectively identified from the first area of interest and the second area of interest. The vehicle identified from the first area of interest is defined as the first vehicle, and the vehicle identified from the second area of interest is defined as the first vehicle. The identified vehicle is defined as the second vehicle. After that, a coordinate system is established in the first remote sensing image, thereby obtaining the coordinate information of each first vehicle in the first remote sensing image (the coordinates of the four points of the rectangular frame corresponding to each first vehicle), and, in the A coordinate system is established in the second remote sensing image to obtain the coordinate information of each second vehicle in the second remote sensing image (the coordinates of the four points of the rectangular frame corresponding to each second vehicle). In addition, the total number of first vehicles and the total number of second vehicles are also counted. The coordinate information and type of the first vehicle in the first remote sensing image and the total number of first vehicles are used as the first vehicle information, and the coordinate information and type of the second vehicle in the second remote sensing image and the total number of first vehicles are used as the second vehicle information.

Step S40: Determine the traffic volume information of the detection area within the preset interval based on the first vehicle information and the second vehicle information.

After that, based on the first vehicle information and the second vehicle information, the traffic flow information of the detection area within the preset interval is determined, that is, the traffic flow information of the detection area between T1 and T2 is determined. Specifically, step S40 includes:

F. Calculate the center coordinate of the first vehicle in the first remote sensing image based on the coordinate information of the first vehicle, and calculate the center coordinate of the second vehicle in the second remote sensing image based on the coordinate information of the second vehicle. center coordinates;

G. Calculate the traffic flow speed of the detection area within the preset interval according to the center coordinates of the first vehicle and the center coordinates of the second vehicle;

H. Compare the type of the first vehicle with the type of each second vehicle, and the total number of first vehicles with the total number of second vehicles, respectively, to obtain the traffic flow change;

1. Determine the traffic flow change amount and the calculated traffic flow speed as the traffic flow information of the detection area within the preset interval.

That is, based on the coordinate information of each first vehicle in the first remote sensing image, calculate the center coordinates of each first vehicle in the first remote sensing image, and based on the coordinate information of each second vehicle in the second remote sensing image , calculate the center coordinates of each second vehicle in the second remote sensing image. Taking the first remote sensing image at time T1 as an example, for any first vehicle in the remote sensing image at time T1, assume that the coordinate information in the first remote sensing image is (x1, y1), (x2, y2), (x3 , y3), (x4, y4), assuming its center coordinate in the remote sensing image at time T1 is (Px, Py), then the formula for calculating the center coordinate (Px, Py) of the vehicle is as follows:

P _x =(x1+x2+x3+x4)/4

P _y =(y1+y2+y3+y4)/4

By analogy, the center coordinates of each first vehicle in the first remote sensing image at time T1 and the center coordinates of each second vehicle in the second remote sensing image at time T2 can be obtained respectively. It can be seen from the above calculation formula that the center coordinates of each first vehicle refer to the center point coordinates of the rectangular frame corresponding to each first vehicle, and the center coordinates of each second vehicle refer to the center coordinates of each second vehicle corresponding to The coordinates of the center point of the rectangular box.

After that, the traffic flow speed of the detection area within the preset interval can be calculated based on the center coordinates of each first vehicle and the center coordinates of each second vehicle. Specifically, step H includes:

H1, based on the center coordinates of the first vehicle, calculate the overall center coordinates of all first vehicles in the first remote sensing image, and based on the center coordinates of the second vehicle, calculate the overall center coordinates of all first vehicles in the second remote sensing image The overall center coordinate of the second vehicle;

H2, obtain the zoom ratio of the first remote sensing image or the second remote sensing image;

H3: Calculate the traffic flow speed of the detection area within the preset interval according to the overall center coordinates of each first vehicle and the overall center coordinates of each second vehicle, and the scaling ratio.

That is, based on the center coordinates of each first vehicle, the overall center coordinates of all first vehicles in the first remote sensing image are calculated, and based on the center coordinates of each second vehicle, the overall center coordinates of all second vehicles in the second remote sensing image are calculated. The overall center coordinates are calculated as follows:

From this, we obtain the overall center coordinates of all first vehicles in the first remote sensing image at time T1 (Center_P _x1 , Center_P _y1 ), and the overall center coordinates of all second vehicles in the second remote sensing image at time T2 (Center_Px2, CentT2er_Py2).

According to physical kinematics, velocity is equal to displacement per unit time, that is:

Therefore, it can be calculated based on the overall center coordinates of all first vehicles in the first remote sensing image at time T1 and the overall center coordinates of all second vehicles in the second remote sensing image at time T2. In the remote sensing image, the difference between time T1 and The overall speed of the traffic flow per unit time between time T2 is calculated as follows:

Since there is a scaling relationship between remote sensing images and real scenes, it is also necessary to obtain the scaling ratio of the first remote sensing image or the scaling ratio of the second remote sensing image, that is, the actual scene of the first remote sensing image or the second remote sensing image relative to the detection area. If the scaling ratio is represented by N, then in the actual scenario, the overall speed of the traffic flow per unit time between T1 time and T2 time is vN.

The type of the first vehicle and the type of the second vehicle, and the total number of the first vehicle and the total number of the second vehicle are also compared respectively to obtain the traffic flow change amount, and the traffic flow change amount and the calculated traffic flow speed are used as the detection area in the predetermined area. Set the traffic flow information within the interval. In this way, the trained semantic segmentation model and remote sensing target detection model are used to analyze two frames of remote sensing images separated by a preset time, thereby realizing the monitoring of traffic flow information.

In more implementations, following step S, include:

J. Send the traffic flow information to the vehicle command and dispatch system, so that the vehicle command and dispatch system can publish the traffic flow information to vehicles within a predetermined distance.

That is, the traffic flow monitoring equipment can send the traffic flow information between TI time and T2 time to the vehicle command and dispatch system. After receiving the traffic flow information, the vehicle command and dispatch system publishes the traffic flow information to the vehicles within a predetermined distance. vehicles, such as sending it to vehicle-mounted terminals to meet traffic control needs.

In order to better understand this embodiment, the implementation process of this embodiment will be described below with reference to the flowchart shown in FIG. 3 .

As shown in Figure 3, the traffic flow monitoring equipment first acquires a remote sensing image of the detection area, and acquires another remote sensing image of the detection area after a preset time interval; then performs semantic segmentation processing on the two acquired remote sensing images. , to extract the road skeleton from the two remote sensing images respectively; then perform remote sensing target detection on the road skeleton extracted from the two remote sensing images to obtain traffic flow information such as vehicle speed, position, quantity, type, etc.; and then use the obtained The traffic flow information is sent to the vehicle command and dispatch system. After receiving the traffic flow information, the vehicle command and dispatch system can know the current road traffic flow status in real time, and then provide real-time notification of road conditions, that is, release the traffic flow information to a predetermined distance. The vehicle's on-board terminal allows vehicle drivers to avoid congested road sections and plan driving routes rationally, thereby ensuring smooth roads and meeting traffic control needs.

This embodiment provides a traffic flow monitoring method, which collects the first remote sensing image of the detection area, and collects the second remote sensing image of the detection area after a preset interval; through the trained semantic segmentation model, the Extract a first region of interest from the first remote sensing image and extract a second region of interest from the second remote sensing image; through the trained remote sensing target detection model, the first region of interest and the second region of interest are respectively Perform target detection in the area of interest to obtain first vehicle information in the first area of interest and second vehicle information in the second area of interest; according to the first vehicle information and the second vehicle information to determine the traffic flow information of the detection area within the preset interval. This embodiment uses the trained semantic segmentation model and remote sensing target detection model to analyze the remote sensing images of the detection area, providing a more detailed analysis basis for the monitoring of traffic flow, and improving the convenience and accuracy of traffic flow monitoring.

Furthermore, based on the above-mentioned first embodiment, a second embodiment of the traffic flow monitoring method of the present invention is proposed. The difference from the first embodiment is that before step S10, it includes:

Train the semantic segmentation model to obtain the trained semantic segmentation model, and train the remote sensing target detection model to obtain the trained remote sensing target detection model.

First, the process of training the semantic segmentation model (DeeplabV3) is as follows:

a. Establish training samples: adjust the initial images used to train the semantic segmentation model to training images that conform to the preset format and size, calibrate the roads in the training images, and uniformly assign them to the same initial category;

b. Multi-scale image resolution acquisition: Using the image pyramid method, the feature map is pooled at different scales to obtain rich contextual information;

c. Encoding-decoding architecture construction: In the encoding process, down-sampling is used to gradually reduce the resolution of the feature map to obtain high-level semantic information, and then encode the image information; in the decoding stage, up-sampling convolution transposition is used In this way, the image spatial information is gradually restored and the prediction results are obtained;

d. Error feedback adjustment process: By comparing the prediction result with the real label, the model loss function is calculated, and through the BP algorithm, the weight of each layer of neural network is adjusted through feedback, and repeated iterations are performed to optimize the semantic segmentation network model.

The process of training the remote sensing target detection model (R2CNN_Faster_RCNN network) is as follows:

e. Establish training samples: Obtain vehicle coordinate information from the images used to train the remote sensing target detection model, crop the images, normalize the size of the cropped images, take the average, and convert it to tfrecord format data;

f. Skeleton network selection: Based on ResNet101, the network is fine-tuned. In the first stage, the candidate frame is obtained through the RPN network. Since the vehicle in the remote sensing image is very small and has any direction, in the process of using R2CNN_Faster_RCNN for target detection, The anchor point scale needs to be changed to (4, 8, 16, 32) to facilitate the extraction of small targets; in order to obtain richer image information, the pooling size is modified to three sizes (7x7, 11x3, 3x11). The final feature map is connected to predict the target frame position. Since a target is often calibrated by multiple rectangular frames in target detection, and the vehicle on the road is tilted, it is necessary to use tilted NMS (non-maximum suppression algorithm, often used for multi-rectangle fusion) for post-processing to obtain the final target. Test results.

From this, the trained semantic segmentation model and remote sensing target detection model are obtained.

In addition, embodiments of the present invention also provide a vehicle flow monitoring device.

Referring to Figure 4, Figure 4 is a functional module schematic diagram of the first embodiment of the traffic flow monitoring device of the present invention.

In this embodiment, the traffic flow monitoring device includes:

The collection module 10 is used to collect the first remote sensing image of the detection area, and collect the second remote sensing image of the detection area after a preset interval;

The extraction module 20 is used to extract a first region of interest from the first remote sensing image and a second region of interest from the second remote sensing image through the trained semantic segmentation model;

The detection module 30 is configured to perform target detection on the first area of interest and the second area of interest through the trained remote sensing target detection model to obtain the first vehicle in the first area of interest. information and second vehicle information in the second area of interest;

The determination module 40 is configured to determine the traffic flow information of the detection area within the preset interval according to the first vehicle information and the second vehicle information.

Among them, each virtual function module of the above-mentioned traffic flow monitoring device is stored in the memory 1005 of the traffic flow monitoring device shown in Figure 1, and is used to realize all functions of the traffic flow monitoring program; when each module is executed by the processor 1001, it can be used for the vehicle. Traffic flow monitoring provides a more detailed analysis basis and improves the convenience and accuracy of traffic flow monitoring.

Further, the extraction module 20 includes:

A first recognition unit configured to input the first remote sensing image to the trained semantic segmentation model to identify the road skeleton from the first remote sensing image as the first region of interest;

The second recognition unit is configured to input the second remote sensing image to the trained semantic segmentation model to identify the road skeleton from the second remote sensing image as the second region of interest.

Further, the detection module 30 includes:

A third identification unit configured to input the first region of interest into the trained remote sensing target detection model for target detection, so as to identify each first vehicle and its type from the first region of interest, and , input the second region of interest into the trained remote sensing target detection model for target detection, so as to identify each second vehicle and its type from the second region of interest;

The fourth identification unit is used to determine the coordinate information of the first vehicle in the first remote sensing image, and count the total number of first vehicles, and determine the coordinate information of the second vehicle in the second remote sensing image, and Count the total number of second vehicles;

A first determining unit configured to determine the coordinate information and type of the first vehicle and the total number of first vehicles as first vehicle information, and to determine the coordinate information and type of the second vehicle and the total number of second vehicles. It is determined as the second vehicle information.

Further, the determination module 40 includes:

A first calculation unit configured to calculate the center coordinates of the first vehicle in the first remote sensing image based on the coordinate information of each first vehicle, and calculate the center coordinates of each second vehicle based on the coordinate information of the second vehicle. The center coordinates in the second remote sensing image;

A second calculation unit configured to calculate the traffic flow speed of the detection area within the preset interval based on the center coordinates of the first vehicle and the center coordinates of the second vehicle;

A comparison unit configured to compare the type of the first vehicle with the type of the second vehicle, and the total number of the first vehicle with the total number of the second vehicle, respectively, to obtain the traffic flow change amount;

The second determination unit is configured to determine the traffic flow change amount and the calculated traffic flow speed as the traffic flow information of the detection area within the preset interval.

Further, the second computing unit includes:

A first calculation subunit configured to calculate the overall center coordinates of all first vehicles in the first remote sensing image based on the center coordinates of the first vehicle, and calculate the overall center coordinates of all first vehicles in the first remote sensing image based on the center coordinates of the second vehicle. The overall center coordinates of all second vehicles in the second remote sensing image;

Obtaining subunit, used to obtain the scaling ratio of the first remote sensing image or the second remote sensing image;

A second calculation subunit configured to calculate the detection area within the preset interval based on the overall center coordinates of the first vehicle and the overall center coordinates of the second vehicle, and the scaling ratio. Traffic speed.

Further, the second calculation subunit includes:

A first calculation sub-unit, configured to calculate the traffic flow speed based on the remote sensing image according to the overall center coordinate of the first vehicle and the overall center coordinate of the second vehicle;

The second calculation sub-unit is used to calculate the product of the traffic speed based on the remote sensing image and the scaling ratio to obtain the traffic speed of the detection area within the preset interval.

Further, the traffic flow monitoring device also includes:

A sending module, configured to send the traffic flow information to a vehicle command and dispatch system, so that the vehicle command and dispatch system can publish the traffic flow information to vehicles within a predetermined distance.

Among them, the functional implementation of each module in the above-mentioned traffic flow monitoring device corresponds to each step in the embodiment of the above-mentioned traffic flow monitoring method, and its functions and implementation processes will not be described in detail here.

In addition, embodiments of the present invention also provide a computer-readable storage medium.

The computer-readable storage medium of the present invention stores a traffic flow monitoring program. When the traffic flow monitoring program is executed by a processor, the steps of the above-mentioned traffic flow monitoring method are implemented.

For the method implemented when the traffic flow monitoring program is executed, reference can be made to the various embodiments of the traffic flow monitoring method of the present invention, which will not be described again here.

It should be noted that, as used herein, the terms "include", "comprising" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or system that includes a list of elements not only includes those elements, but It also includes other elements not expressly listed or that are inherent to the process, method, article or system. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

The above serial numbers of the embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better. implementation. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence or the part that contributes to the existing technology. The computer software product is stored in a storage medium (such as ROM/RAM, disk, CD), including several instructions to cause a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present invention.

The above are only preferred embodiments of the present invention, and do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the description and drawings of the present invention may be directly or indirectly used in other related technical fields. , are all similarly included in the scope of patent protection of the present invention.

Claims (8)

1. A traffic flow monitoring method, characterized in that the method includes the following steps: Collect the first remote sensing image of the detection area, and collect the second remote sensing image of the detection area after a preset interval, wherein the background of the first remote sensing image and the second remote sensing image is unchanged and changes are caused in part by moving vehicles; Using the trained semantic segmentation model, a first region of interest is extracted from the first remote sensing image and a second region of interest is extracted from the second remote sensing image; Through the trained remote sensing target detection model, target detection is performed on the first area of interest and the second area of interest respectively to obtain the first vehicle information and the second area of interest in the first area of interest. Second vehicle information in the area of interest; Determine the traffic flow information of the detection area within the preset interval according to the first vehicle information and the second vehicle information; Wherein, the remote sensing target detection model completed through training performs target detection on the first area of interest and the second area of interest respectively to obtain the first vehicle information and information in the first area of interest. The steps of obtaining the second vehicle information in the second area of interest include: The first region of interest is input into the trained remote sensing target detection model for target detection to identify the first vehicle and its type from the first region of interest, and the second region of interest is The area is input into the trained remote sensing target detection model for target detection to identify the second vehicle and its type from the second area of interest; Determine the coordinate information of the first vehicle in the first remote sensing image, and count the total number of first vehicles, and determine the coordinate information of the second vehicle in the second remote sensing image, and count the total number of second vehicles; Determine the coordinate information and type of the first vehicle and the total number of first vehicles as first vehicle information, and determine the coordinate information and type of the second vehicle and the total number of second vehicles as second vehicle information; Wherein, the step of determining the traffic flow information of the detection area within the preset interval based on the first vehicle information and the second vehicle information includes: Calculate the center coordinates of the first vehicle in the first remote sensing image based on the coordinate information of the first vehicle, and calculate the center coordinates of the second vehicle in the second remote sensing image based on the coordinate information of the second vehicle coordinate; Calculate the traffic speed of the detection area within the preset interval according to the center coordinate of the first vehicle and the center coordinate of the second vehicle; Compare the type of the first vehicle with the type of the second vehicle, and the total number of the first vehicle with the total number of the second vehicle respectively to obtain the traffic flow change amount; The traffic flow change amount and the calculated traffic flow speed are determined as the traffic flow information of the detection area within the preset interval. 2. The traffic flow monitoring method according to claim 1, wherein the semantic segmentation model completed through training extracts a first region of interest from the first remote sensing image, and extracts a first region of interest from the second remote sensing image. The steps to extract the second region of interest in the image include: Input the first remote sensing image to the trained semantic segmentation model to identify the road skeleton from the first remote sensing image as the first region of interest; The second remote sensing image is input to the trained semantic segmentation model to identify the road skeleton from the second remote sensing image as the second region of interest. 3. The traffic flow monitoring method of claim 1, wherein the detection area is calculated at the preset interval based on the center coordinates of the first vehicle and the center coordinates of the second vehicle. The steps for traffic speed within time include: According to the center coordinates of the first vehicle, the overall center coordinates of all first vehicles in the first remote sensing image are calculated, and based on the center coordinates of the second vehicle, all second remote sensing images in the second remote sensing image are calculated. The overall center coordinates of the vehicle; Obtain the zoom ratio of the first remote sensing image or the second remote sensing image; According to the overall center coordinate of the first vehicle and the overall center coordinate of the second vehicle, and the scaling ratio, the traffic flow speed of the detection area within the preset interval is calculated. 4. The traffic flow monitoring method according to claim 3, characterized in that the calculation is based on the overall center coordinate of the first vehicle and the overall center coordinate of the second vehicle, and the scaling ratio. The step of detecting the traffic speed of the area within the preset interval includes: Calculate the traffic flow speed based on the remote sensing image according to the overall center coordinate of the first vehicle and the overall center coordinate of the second vehicle; Calculate the product of the traffic flow speed based on the remote sensing image and the scaling ratio to obtain the traffic flow speed of the detection area within the preset interval. 5. The traffic flow monitoring method according to claim 1, wherein the number of vehicles in the detection area within the preset interval is determined based on the first vehicle information and the second vehicle information. Following the steps for traffic information include: The traffic flow information is sent to the vehicle command and dispatch system, so that the vehicle command and dispatch system publishes the traffic flow information to vehicles within a predetermined distance. 6. A vehicle flow monitoring device, characterized in that the vehicle flow monitoring device includes: The collection module is used to collect the first remote sensing image of the detection area, and collect the second remote sensing image of the detection area after a preset interval, wherein the background of the first remote sensing image and the second remote sensing image is The constant, changing part is caused by the moving vehicle; An extraction module, configured to extract a first region of interest from the first remote sensing image and a second region of interest from the second remote sensing image using the semantic segmentation model completed through training; A detection module, configured to perform target detection on the first area of interest and the second area of interest through the trained remote sensing target detection model to obtain the first vehicle information in the first area of interest. and second vehicle information in the second area of interest; Determining module, configured to determine the traffic flow information of the detection area within the preset interval according to the first vehicle information and the second vehicle information; Among them, the detection module is specifically used for: The first region of interest is input into the trained remote sensing target detection model for target detection to identify the first vehicle and its type from the first region of interest, and the second region of interest is The area is input into the trained remote sensing target detection model for target detection to identify the second vehicle and its type from the second area of interest; Determine the coordinate information of the first vehicle in the first remote sensing image, and count the total number of first vehicles, and determine the coordinate information of the second vehicle in the second remote sensing image, and count the total number of second vehicles; Determine the coordinate information and type of the first vehicle and the total number of first vehicles as first vehicle information, and determine the coordinate information and type of the second vehicle and the total number of second vehicles as second vehicle information; Wherein, the determination module is specifically used for: Calculate the center coordinates of the first vehicle in the first remote sensing image based on the coordinate information of the first vehicle, and calculate the center coordinates of the second vehicle in the second remote sensing image based on the coordinate information of the second vehicle coordinate; Calculate the traffic speed of the detection area within the preset interval according to the center coordinate of the first vehicle and the center coordinate of the second vehicle; Compare the type of the first vehicle with the type of the second vehicle, and the total number of the first vehicle with the total number of the second vehicle respectively to obtain the traffic flow change amount; The traffic flow change amount and the calculated traffic flow speed are determined as the traffic flow information of the detection area within the preset interval. 7. A vehicle flow monitoring device, characterized in that the vehicle flow monitoring device includes a processor, a memory, and a traffic flow monitoring program stored on the memory and executable by the processor, wherein the vehicle flow monitoring device When the flow monitoring program is executed by the processor, the steps of the traffic flow monitoring method according to any one of claims 1 to 5 are implemented. 8. A computer-readable storage medium, characterized in that a traffic flow monitoring program is stored on the computer-readable storage medium, and when executed by a processor, the traffic flow monitoring program implements any one of claims 1 to 5 The steps of the traffic flow monitoring method described in the item.