CN110738169B - Traffic flow monitoring method, device, equipment and computer readable storage medium - Google Patents
Traffic flow monitoring method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN110738169B CN110738169B CN201910975456.2A CN201910975456A CN110738169B CN 110738169 B CN110738169 B CN 110738169B CN 201910975456 A CN201910975456 A CN 201910975456A CN 110738169 B CN110738169 B CN 110738169B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- remote sensing
- sensing image
- interest
- traffic flow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a traffic flow monitoring method, a device, equipment and a computer readable storage medium, wherein the method comprises the following steps: collecting a first remote sensing image of a detection area, and collecting a second remote sensing image of the detection area after a preset interval time; extracting a first region of interest from the first remote sensing image and a second region of interest from the second remote sensing image through the trained semantic segmentation model; respectively carrying out target detection on the first region of interest and the second region of interest through a remote sensing target detection model which is completed through training so as to acquire first vehicle information in the first region of interest and second vehicle information in the second region of interest; and determining traffic flow information of the detection area in the preset interval time according to the first vehicle information and the second vehicle information. The invention improves the convenience and accuracy of traffic flow monitoring.
Description
Technical Field
The present invention relates to the field of intelligent traffic technologies, and in particular, to a method, an apparatus, a device, and a computer readable storage medium for monitoring traffic flow.
Background
With the development of society and the progress of technology, the living standard of people is greatly improved, the automobile possession is greatly improved, and the traffic jam phenomenon is more serious, so how to efficiently manage traffic is very important, wherein the traffic flow is an important traffic parameter in traffic management, and the traditional traffic flow monitoring technology, such as a loop coil method, needs to install a coil sensor on a road surface, damages the road surface, is inconvenient to construct and install, and only can induce vehicles passing through a fixed position where the coil sensor is installed, so that the convenience is poor, and the accuracy is lower.
Disclosure of Invention
The invention mainly aims to provide a traffic flow monitoring method, a device, equipment and a computer readable storage medium, and aims to solve the technical problems of poor convenience and low accuracy of the existing traffic flow monitoring technology.
To achieve the above object, the present invention provides a vehicle flow monitoring method, comprising the steps of:
collecting a first remote sensing image of a detection area, and collecting a second remote sensing image of the detection area after a preset interval time;
extracting a first region of interest from the first remote sensing image and a second region of interest from the second remote sensing image through the trained semantic segmentation model;
Respectively carrying out target detection on the first region of interest and the second region of interest through a remote sensing target detection model which is completed through training so as to acquire first vehicle information in the first region of interest and second vehicle information in the second region of interest;
and determining traffic flow information of the detection area in the preset interval time according to the first vehicle information and the second vehicle information.
Optionally, the step of extracting the first region of interest from the first remote sensing image and the step of extracting the second region of interest from the second remote sensing image by using the semantic segmentation model after training includes:
inputting the first remote sensing image into a trained semantic segmentation model to identify a road skeleton from the first remote sensing image as a first region of interest;
and inputting the second remote sensing image into the trained semantic segmentation model to identify a road skeleton from the second remote sensing image as a second region of interest.
Optionally, the step of performing object detection on the first region of interest and the second region of interest by using the trained remote sensing object detection model to obtain the first vehicle information in the first region of interest and the second vehicle information in the second region of interest includes:
Inputting the first region of interest into a trained remote sensing target detection model for target detection to identify a first vehicle and a type thereof from the first region of interest, and inputting the second region of interest into a trained remote sensing target detection model for target detection to identify a second vehicle and a type thereof from the second region of interest;
determining coordinate information of a first vehicle in the first remote sensing image, counting the total number of the first vehicles, determining coordinate information of a second vehicle in the second remote sensing image, and counting the total number of the second vehicles;
the coordinate information and type of the first vehicle, the total number of the first vehicles are determined as first vehicle information, and the coordinate information and type of the second vehicles, the total number of the second vehicles are determined as second vehicle information.
Optionally, the step of determining the traffic flow information of the detection area in the preset interval time according to the first vehicle information and the second vehicle information includes:
calculating the center coordinates of a first vehicle in the first remote sensing image according to the coordinate information of the first vehicle, and calculating the center coordinates of a second vehicle in the second remote sensing image according to the coordinate information of the second vehicle;
Calculating the vehicle flow speed of the detection area in the preset interval time according to the center coordinates of the first vehicle and the center coordinates of the second vehicle;
respectively comparing the type of the first vehicle with the type of the second vehicle, and comparing the total number of the first vehicles with the total number of the second vehicles to obtain the traffic flow variation;
and determining the traffic flow variation and the calculated traffic flow speed as traffic flow information of the detection area in the preset interval time.
Optionally, the step of calculating the traffic speed of the detection area within the preset interval time according to the center coordinates of the first vehicle and the center coordinates of the second vehicle includes:
calculating the overall center coordinates of all the first vehicles in the first remote sensing image according to the center coordinates of the first vehicles, and calculating the overall center coordinates of all the second vehicles in the second remote sensing image according to the center coordinates of the second vehicles;
acquiring the scaling of the first remote sensing image or the second remote sensing image;
and calculating the vehicle flow speed of the detection area in the preset interval time according to the integral center coordinates of the first vehicle and the integral center coordinates of the second vehicle and the scaling.
Optionally, before the step of acquiring the first remote sensing image of the detection area and acquiring the second remote sensing image of the detection area after the preset interval time, the method includes:
training the semantic segmentation model to obtain a trained semantic segmentation model, and training the remote sensing target detection model to obtain a trained remote sensing target detection model.
Optionally, the step of calculating the traffic speed of the detection area in the preset interval time according to the global center coordinates of the first vehicle and the global center coordinates of the second vehicle and the scaling includes:
calculating a traffic speed based on a remote sensing image according to the overall center coordinates of the first vehicle and the overall center coordinates of the second vehicle;
and calculating the product of the traffic flow speed based on the remote sensing image and the scaling ratio to obtain the traffic flow speed of the detection area in the preset interval time.
In addition, in order to achieve the above object, the present invention also provides a traffic flow monitoring device including:
the acquisition module is used for acquiring a first remote sensing image of the detection area and acquiring a second remote sensing image of the detection area after a preset interval time;
The extraction module is used for respectively extracting a first region of interest from the first remote sensing image and extracting a second region of interest from the second remote sensing image through the trained semantic segmentation model;
the detection module is used for respectively carrying out target detection on the first region of interest and the second region of interest through a remote sensing target detection model which is completed through training so as to acquire first vehicle information in the first region of interest and second vehicle information in the second region of interest;
and the determining module is used for determining the traffic flow information of the detection area in the preset interval time according to the first vehicle information and the second vehicle information.
In addition, in order to achieve the above object, the present invention also provides a traffic flow monitoring device including a processor, a memory, and a visualization program of traffic data stored on the memory and executable by the processor, wherein the traffic flow monitoring program, when executed by the processor, implements the steps of the traffic flow monitoring method as described above.
In addition, in order to achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a traffic flow monitoring program which, when executed by a processor, implements the steps of the traffic flow monitoring method as described above.
The invention provides a traffic flow monitoring method, a device, equipment and a computer readable storage medium, wherein a first remote sensing image of a detection area is acquired, and a second remote sensing image of the detection area is acquired after a preset interval time; extracting a first region of interest from the first remote sensing image and a second region of interest from the second remote sensing image through the trained semantic segmentation model; respectively carrying out target detection on the first region of interest and the second region of interest through a remote sensing target detection model which is completed through training so as to acquire first vehicle information in the first region of interest and second vehicle information in the second region of interest; and determining traffic flow information of the detection area in the preset interval time according to the first vehicle information and the second vehicle information. According to the invention, the remote sensing image of the detection area is analyzed through the semantic segmentation model and the remote sensing target detection model which are completed through training, so that a more detailed analysis basis is provided for monitoring the traffic flow, and the convenience and accuracy of traffic flow monitoring are improved.
Drawings
FIG. 1 is a schematic hardware configuration of a traffic flow monitoring device according to an embodiment of the present invention;
FIG. 2 is a flow chart of a first embodiment of the traffic flow monitoring method of the present invention;
FIG. 3 is a flowchart illustrating an example implementation of a first embodiment of a traffic flow monitoring method of the present invention;
fig. 4 is a schematic functional block diagram of a first embodiment of the traffic flow monitoring device according to the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The main solutions of the embodiments of the present invention are: collecting a first remote sensing image of a detection area, and collecting a second remote sensing image of the detection area after a preset interval time; extracting a first region of interest from the first remote sensing image and a second region of interest from the second remote sensing image through the trained semantic segmentation model; respectively carrying out target detection on the first region of interest and the second region of interest through a remote sensing target detection model which is completed through training so as to acquire first vehicle information in the first region of interest and second vehicle information in the second region of interest; and determining traffic flow information of the detection area in the preset interval time according to the first vehicle information and the second vehicle information. The technical problems of poor convenience and low accuracy of the existing traffic flow monitoring technology are solved.
As shown in fig. 1, fig. 1 is a schematic diagram of a terminal structure of a hardware running environment according to an embodiment of the present invention.
The traffic flow monitoring method related to the embodiment of the invention can be realized by traffic flow monitoring equipment, and the traffic flow monitoring equipment can be equipment with data processing functions such as a PC (personal computer), a server and the like.
Referring to fig. 1, fig. 1 is a schematic hardware configuration of a traffic flow monitoring device according to an embodiment of the present invention. In an embodiment of the present invention, the traffic monitoring device may include a processor 1001 (e.g., central processing unit Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein the communication bus 1002 is used to enable connected communications between these components; the user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard); the network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface); the memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory, and the memory 1005 may alternatively be a storage device independent of the processor 1001. Those skilled in the art will appreciate that the hardware configuration shown in fig. 1 is not limiting of the invention and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
With continued reference to fig. 1, the memory 1005 of fig. 1, which is a readable storage medium, may include an operating system, a network communication module, and a traffic monitoring program. In fig. 1, the network communication module is mainly used for connecting with a server and performing data communication with the server; the processor 1001 may call the traffic flow monitoring program stored in the memory 1005 and execute the traffic flow monitoring method according to the embodiment of the present invention.
The embodiment of the invention provides a vehicle flow monitoring method.
Referring to fig. 2, fig. 2 is a flow chart of a first embodiment of the traffic flow monitoring method according to the present invention.
In this embodiment, the traffic flow monitoring method is implemented by a traffic flow monitoring device, where the traffic flow monitoring device may be a terminal device such as a PC, a server, or the like, and may be a device shown in fig. 1, where the traffic flow monitoring device is in communication connection with a remote sensing device, and may control the remote sensing device, and the traffic flow monitoring method includes the following steps:
step S10, collecting a first remote sensing image of a detection area, and collecting a second remote sensing image of the detection area after a preset interval time;
step S20, extracting a first region of interest from the first remote sensing image and a second region of interest from the second remote sensing image respectively through a trained semantic segmentation model;
Step S30, respectively carrying out target detection on the first region of interest and the second region of interest through a remote sensing target detection model which is completed through training so as to acquire first vehicle information in the first region of interest and second vehicle information in the second region of interest;
step S40, determining traffic flow information of the detection area within the preset interval time according to the first vehicle information and the second vehicle information.
In this embodiment, the detection area may be an arbitrary road section. The remote sensing equipment is built in the detection area in advance, and communication connection between the traffic flow monitoring equipment and the remote sensing equipment is built, so that the traffic flow monitoring equipment can control the remote sensing equipment to collect remote sensing images of the detection area at any time, and further, the traffic flow monitoring equipment can analyze the collected remote sensing images through the semantic segmentation model and the remote sensing target detection model which are completed through training, so that a more detailed analysis basis is provided for traffic flow monitoring, and convenience and accuracy of traffic flow monitoring are improved.
Step S10, collecting a first remote sensing image of a detection area, and collecting a second remote sensing image of the detection area after a preset interval time;
Since in practice, in two frames of images with shorter imaging time, the background is almost unchanged, and the changed part is caused by a moving vehicle, the embodiment obtains traffic flow information by analyzing two frames of remote sensing images with a preset time interval based on physical kinematics.
Specifically, the traffic flow monitoring device may send a remote sensing image acquisition instruction to the remote sensing device in real time or at regular time, control the remote sensing device to acquire one frame of remote sensing image (defined as a first remote sensing image) of the detection area, and control the remote sensing device to acquire another frame of remote sensing image (defined as a second remote sensing image) of the remote sensing image of the detection area after a preset interval time, where the preset interval time may be flexibly set according to actual needs, and is shorter. The acquisition time of the first remote sensing image is defined as T1, and the acquisition time of the second remote sensing image is defined as T2, and it can be understood that T1 is less than T2.
Step S20, extracting a first region of interest from the first remote sensing image and a second region of interest from the second remote sensing image respectively through a trained semantic segmentation model;
and then, respectively extracting a first region of interest from the first remote sensing image and extracting a second region of interest from the second remote sensing image through the trained semantic segmentation model. As one embodiment, step S20 includes:
A. Inputting the first remote sensing image into a trained semantic segmentation model to identify a road skeleton from the first remote sensing image as a first region of interest;
B. and inputting the second remote sensing image into the trained semantic segmentation model to identify a road skeleton from the second remote sensing image as a second region of interest.
Namely, a first remote sensing image acquired at the time of T1 is input into a training completion semantic segmentation model for analysis to extract a road skeleton from the first remote sensing image acquired at the time of T1 as a first region of interest, and a second remote sensing image acquired at the time of T2 is input into the semantic segmentation model for analysis to extract the road skeleton from the second remote sensing image acquired at the time of T2 as a second region of interest.
Step S30, respectively carrying out target detection on the first region of interest and the second region of interest through a remote sensing target detection model which is completed through training so as to acquire first vehicle information in the first region of interest and second vehicle information in the second region of interest;
and then, respectively carrying out target detection on the first region of interest and the second region of interest by adopting a remote sensing target detection model which is completed through training so as to acquire first vehicle information in the first region of interest and second vehicle information in the second region of interest. Specifically, step S30 includes:
C. Inputting the first region of interest into a trained remote sensing target detection model for target detection to identify a first vehicle and a type thereof from the first region of interest, and inputting the second region of interest into a trained remote sensing target detection model for target detection to identify a second vehicle and a type thereof from the second region of interest;
D. determining coordinate information of a first vehicle in the first remote sensing image, counting the total number of the first vehicles, determining coordinate information of a second vehicle in the second remote sensing image, and counting the total number of the second vehicles;
E. the coordinate information and type of the first vehicle, the total number of the first vehicles are determined as first vehicle information, and the coordinate information and type of the second vehicles, the total number of the second vehicles are determined as second vehicle information.
Namely, a first region of interest in a first remote sensing image acquired at the time T1 and a second region of interest in a second remote sensing image acquired at the time T2 are respectively input into a trained remote sensing target detection model to perform target detection so as to respectively identify vehicles and corresponding types thereof from the first region of interest and the second region of interest, wherein the vehicles identified from the first region of interest are defined as first vehicles, and the vehicles identified from the second region of interest are defined as second vehicles. Then, a coordinate system is established in the first remote sensing image, so that coordinate information of each first vehicle in the first remote sensing image (coordinates of four points of a rectangular frame corresponding to each first vehicle) is obtained, and a coordinate system is established in the second remote sensing image, so that coordinate information of each second vehicle in the second remote sensing image (coordinates of four points of a rectangular frame corresponding to each second vehicle) is obtained. In addition, the total number of first vehicles and the total number of second vehicles are counted. The coordinate information and the type of the first vehicle in the first remote sensing image and the total number of the first vehicles are taken as first vehicle information, and the coordinate information and the type of the second vehicle in the second remote sensing image and the total number of the first vehicles are taken as second vehicle information.
Step S40, determining traffic flow information of the detection area within the preset interval time according to the first vehicle information and the second vehicle information.
Then, according to the first vehicle information and the second vehicle information, the vehicle flow information of the detection area in the preset interval time is determined, namely, the vehicle flow information of the detection area between T1 and T2 is determined. Specifically, step S40 includes:
F. calculating the center coordinates of a first vehicle in the first remote sensing image according to the coordinate information of the first vehicle, and calculating the center coordinates of a second vehicle in the second remote sensing image according to the coordinate information of the second vehicle;
G. calculating the vehicle flow speed of the detection area in the preset interval time according to the center coordinates of the first vehicle and the center coordinates of the second vehicle;
H. respectively comparing the type of the first vehicle with the type of each second vehicle, and respectively comparing the total number of the first vehicles with the total number of the second vehicles to obtain the traffic flow variation;
I. and determining the traffic flow variation and the calculated traffic flow speed as traffic flow information of the detection area in the preset interval time.
That is, the center coordinates of each first vehicle in the first remote sensing image are calculated according to the coordinate information of each first vehicle in the first remote sensing image, and the center coordinates of each second vehicle in the second remote sensing image are calculated according to the coordinate information of each second vehicle in the second remote sensing image. Taking the first remote sensing image at the time T1 as an example, for any first vehicle in the remote sensing image at the time T1, the coordinate information in the first remote sensing image is (x 1, y 1), (x 2, y 2), (x 3, y 3), (x 4, y 4), and the central coordinate in the remote sensing image at the time T1 is (Px, py), and then the formula for calculating the central coordinate (Px, py) of the vehicle is as follows:
P x ＝(x1+x2+x3+x4)/4
P y ＝(y1+y2+y3+y4)/4
and the center coordinates of each first vehicle in the first remote sensing image at the moment T1 and the center coordinates of each second vehicle in the second remote sensing image at the moment T2 can be obtained respectively. As can be seen from the above calculation formula, the center coordinates of each first vehicle refer to the center point coordinates of the rectangular frame corresponding to each first vehicle, and the center coordinates of each second vehicle refer to the center point coordinates of the rectangular frame corresponding to each second vehicle.
And then, calculating the vehicle flow speed of the detection area in the preset interval time according to the center coordinates of the first vehicles and the center coordinates of the second vehicles. Specifically, step H includes:
H1, calculating the overall center coordinates of all first vehicles in the first remote sensing image according to the center coordinates of the first vehicles, and calculating the overall center coordinates of all second vehicles in the second remote sensing image according to the center coordinates of the second vehicles;
h2, obtaining the scaling of the first remote sensing image or the second remote sensing image;
and H3, calculating the vehicle flow speed of the detection area in the preset interval time according to the overall center coordinates of each first vehicle and the overall center coordinates of each second vehicle and the scaling.
That is, the overall center coordinates of all the first vehicles in the first remote sensing image are calculated according to the center coordinates of each first vehicle, and the overall center coordinates of all the second vehicles in the second remote sensing image are calculated according to the center coordinates of each second vehicle, and the calculation formula is as follows:
thereby obtaining the global Center coordinates (center_P) of all the first vehicles in the first remote sensing image at the time T1 x1 ，Center_P y1 ) And the global Center coordinates (center_px2, cent2er_py2) of all second vehicles in the second remote sensing image at the time T2.
From physical kinematics, the velocity is equal to the displacement per unit time, namely:
Therefore, the overall speed of the vehicle flow in the unit time between the time T1 and the time T2 in the remote sensing image can be calculated according to the overall center coordinates of all the first vehicles in the first remote sensing image at the time T1 and the overall center coordinates of all the second vehicles in the second remote sensing image at the time T2, and the calculation formula is as follows:
because the remote sensing image and the real scene have a scaling relationship, the scaling of the first remote sensing image or the scaling of the second remote sensing image, that is, the scaling of the first remote sensing image or the scaling of the second remote sensing image relative to the actual scene of the detection area, if the scaling is represented by N, the overall speed of the vehicle flow in unit time between the time T1 and the time T2 in the actual scene is vN.
And respectively comparing the type of the first vehicle with the type of the second vehicle, and comparing the total number of the first vehicles with the total number of the second vehicles to obtain the traffic flow variation, and taking the traffic flow variation and the calculated traffic flow speed as traffic flow information of the detection area in the preset interval time. Therefore, the two frames of remote sensing images with preset time intervals are analyzed by using the semantic segmentation model and the remote sensing target detection model which are completed through training, and the monitoring of traffic flow information is realized.
In further implementations, after step S, comprising:
J. and sending the traffic flow information to a vehicle commanding and dispatching system so that the vehicle commanding and dispatching system can issue the traffic flow information to vehicles within a preset distance.
That is, the traffic flow monitoring device may send traffic flow information between the TI time and the T2 time to the vehicle commanding and dispatching system, and after receiving the traffic flow information, the vehicle commanding and dispatching system issues the traffic flow information to the vehicle within a predetermined distance, for example, to the vehicle-mounted terminal, so as to meet traffic control requirements.
For a better understanding of the present embodiment, the implementation procedure of the present embodiment is described below with reference to the flowchart shown in fig. 3.
As shown in fig. 3, the traffic flow monitoring device first acquires one remote sensing image of the detection area, and acquires another remote sensing image of the detection area after a preset time interval; then, semantic segmentation processing is carried out on the two acquired remote sensing images respectively so as to extract road frameworks from the two remote sensing images respectively; respectively carrying out remote sensing target detection on road frameworks extracted from the two remote sensing images to obtain vehicle flow information such as vehicle speed, position, quantity, type and the like; and then the obtained traffic flow information is sent to a vehicle commanding and dispatching system, and after the traffic flow information is received by the vehicle commanding and dispatching system, the current road surface traffic flow condition can be obtained in real time, and then the road condition is notified in real time, namely the traffic flow information is issued to the vehicle-mounted terminal of the vehicle in a preset distance, so that a vehicle driver can avoid a congestion road section, reasonably plan a driving route, further ensure that the road is smooth, and meet traffic control requirements.
The embodiment provides a traffic flow monitoring method, which comprises the steps of collecting a first remote sensing image of a detection area, and collecting a second remote sensing image of the detection area after a preset interval time; extracting a first region of interest from the first remote sensing image and a second region of interest from the second remote sensing image through the trained semantic segmentation model; respectively carrying out target detection on the first region of interest and the second region of interest through a remote sensing target detection model which is completed through training so as to acquire first vehicle information in the first region of interest and second vehicle information in the second region of interest; and determining traffic flow information of the detection area in the preset interval time according to the first vehicle information and the second vehicle information. According to the method, the remote sensing image of the detection area is analyzed through the semantic segmentation model and the remote sensing target detection model which are completed through training, a more detailed analysis basis is provided for monitoring the traffic flow, and convenience and accuracy of traffic flow monitoring are improved.
Further, based on the first embodiment, a second embodiment of the traffic flow monitoring method according to the present invention is proposed, which is different from the first embodiment in that before step S10, the method includes:
Training the semantic segmentation model to obtain a trained semantic segmentation model, and training the remote sensing target detection model to obtain a trained remote sensing target detection model.
First, the process of training the semantic segmentation model (deep labv 3) is as follows:
a. building training samples: the method comprises the steps of adjusting an initial image for training a semantic segmentation model into a training image conforming to a preset format and size, calibrating roads in the training image, and uniformly distributing the roads into the same initial category;
b. multi-scale image resolution acquisition: different-scale pooling operation is carried out on the feature images in an image pyramid mode so as to obtain rich upper and lower text information;
c. coding-decoding architecture construction: in the encoding process, a downsampling mode is adopted, and the resolution of the feature images is gradually reduced to obtain advanced semantic information, so that the image information is encoded; in the decoding stage, gradually recovering image space information in an up-sampling convolution transposition mode to obtain a prediction result;
d. error feedback adjustment process: and comparing the real labels according to the prediction result, calculating a model loss function, and feeding back and adjusting the weight of each layer of neural network according to a BP algorithm, and iterating repeatedly to ensure that the semantic segmentation network model is optimal.
The process of training the remote sensing target detection model (r2cnn_fast_rcnn network) is as follows:
e. building training samples: acquiring vehicle coordinate information from an image for training a remote sensing target detection model, cutting the image, normalizing the size of the cut image, taking an average value, and converting the image into tfreeord format data;
f. selecting a skeleton network: fine tuning is carried out on the network based on ResNet101, in the first stage, candidate frames are obtained through an RPN network, and because vehicles in a remote sensing image are small and the direction is random, anchor points are required to be changed into (4, 8,16 and 32) in the process of adopting R2 CNN_fast_RCNN as target detection, and the extraction of small targets is facilitated; to obtain richer image information, the pooling size is modified to be (7 x7, 11x3,3x 11) three sizes, and the final feature map is connected to predict the target frame position. Because a situation that one target is marked by a plurality of rectangular frames frequently occurs in target detection, and meanwhile, a tilting NMS (non-maximum suppression algorithm, which is commonly used for multi-rectangular fusion) is required to be adopted for post-processing due to the tilting of vehicles in a road, so that a final target detection result is obtained.
Thus, a semantic segmentation model and a remote sensing target detection model which are trained are obtained.
In addition, the embodiment of the invention also provides a vehicle flow monitoring device.
Referring to fig. 4, fig. 4 is a schematic functional block diagram of a first embodiment of a traffic flow monitoring device according to the present invention.
In this embodiment, the traffic flow monitoring device includes:
the acquisition module 10 is used for acquiring a first remote sensing image of a detection area and acquiring a second remote sensing image of the detection area after a preset interval time;
the extracting module 20 is configured to extract a first region of interest from the first remote sensing image and extract a second region of interest from the second remote sensing image through the trained semantic segmentation model;
the detection module 30 is configured to perform target detection on the first region of interest and the second region of interest through a trained remote sensing target detection model, so as to obtain first vehicle information in the first region of interest and second vehicle information in the second region of interest;
a determining module 40, configured to determine traffic flow information of the detection area within the preset interval time according to the first vehicle information and the second vehicle information.
Wherein, each virtual function module of the traffic flow monitoring device is stored in the memory 1005 of the traffic flow monitoring device shown in fig. 1, and is used for implementing all functions of the traffic flow monitoring program; when each module is executed by the processor 1001, a more detailed analysis basis can be provided for monitoring the traffic flow, and convenience and accuracy of traffic flow monitoring are improved.
Further, the extracting module 20 includes:
the first recognition unit is used for inputting the first remote sensing image into the trained semantic segmentation model so as to recognize a road skeleton from the first remote sensing image as a first region of interest;
the second recognition unit is used for inputting the second remote sensing image into the trained semantic segmentation model so as to recognize a road skeleton from the second remote sensing image as a second region of interest.
Further, the detection module 30 includes:
the third recognition unit is used for inputting the first region of interest into a trained remote sensing target detection model to perform target detection so as to recognize each first vehicle and each type thereof from the first region of interest, and inputting the second region of interest into the trained remote sensing target detection model to perform target detection so as to recognize each second vehicle and each type thereof from the second region of interest;
a fourth identifying unit, configured to determine coordinate information of a first vehicle in the first remote sensing image, and count the total number of the first vehicles, and determine coordinate information of a second vehicle in the second remote sensing image, and count the total number of the second vehicles;
A first determining unit configured to determine the coordinate information and type of the first vehicle, a total number of the first vehicles as first vehicle information, and the coordinate information and type of the second vehicle, a total number of the second vehicles as second vehicle information.
Further, the determining module 40 includes:
a first calculation unit configured to calculate a center coordinate of each first vehicle in the first remote sensing image according to the coordinate information of each first vehicle, and calculate a center coordinate of each second vehicle in the second remote sensing image according to the coordinate information of each second vehicle;
a second calculation unit configured to calculate a vehicle flow speed of the detection area within the preset interval time according to the center coordinates of the first vehicle and the center coordinates of the second vehicle;
the comparison unit is used for respectively comparing the type of the first vehicle with the type of the second vehicle, the total number of the first vehicles and the total number of the second vehicles to obtain the traffic flow variation;
and the second determining unit is used for determining the traffic flow variation and the calculated traffic flow speed as traffic flow information of the detection area in the preset interval time.
Further, the second calculation unit includes:
the first calculating subunit is used for calculating the overall center coordinates of all the first vehicles in the first remote sensing image according to the center coordinates of the first vehicles, and calculating the overall center coordinates of all the second vehicles in the second remote sensing image according to the center coordinates of the second vehicles;
an acquisition subunit, configured to acquire a scaling ratio of the first remote sensing image or the second remote sensing image;
and the second calculating subunit is used for calculating the vehicle flow speed of the detection area in the preset interval time according to the integral center coordinates of the first vehicle and the integral center coordinates of the second vehicle and the scaling.
Further, the second computing subunit includes:
a first calculation subunit, configured to calculate a traffic speed based on a remote sensing image according to the global center coordinate of the first vehicle and the global center coordinate of the second vehicle;
and the second calculation subunit is used for calculating the product of the traffic speed based on the remote sensing image and the scaling to obtain the traffic speed of the detection area in the preset interval time.
Further, the traffic flow monitoring device further includes:
and the sending module is used for sending the traffic flow information to a vehicle commanding and dispatching system so that the vehicle commanding and dispatching system can issue the traffic flow information to vehicles within a preset distance.
The function implementation of each module in the traffic flow monitoring device corresponds to each step in the traffic flow monitoring method embodiment, and the function and implementation process of each module are not described here in detail.
In addition, the embodiment of the invention also provides a computer readable storage medium.
The computer readable storage medium of the present invention stores a traffic flow monitoring program, wherein the traffic flow monitoring program, when executed by a processor, implements the steps of the traffic flow monitoring method as described above.
The method implemented when the traffic flow monitoring program is executed may refer to various embodiments of the traffic flow monitoring method of the present invention, and will not be described herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (8)
1. A method of traffic flow monitoring, the method comprising the steps of:
collecting a first remote sensing image of a detection area, and collecting a second remote sensing image of the detection area after a preset interval time, wherein the first remote sensing image and the second remote sensing image are unchanged in background, and the changed part is caused by a moving vehicle;
extracting a first region of interest from the first remote sensing image and a second region of interest from the second remote sensing image through the trained semantic segmentation model;
respectively carrying out target detection on the first region of interest and the second region of interest through a remote sensing target detection model which is completed through training so as to acquire first vehicle information in the first region of interest and second vehicle information in the second region of interest;
determining traffic flow information of the detection area within the preset interval time according to the first vehicle information and the second vehicle information;
the step of respectively performing object detection on the first region of interest and the second region of interest by the remote sensing object detection model after training to obtain first vehicle information in the first region of interest and second vehicle information in the second region of interest includes:
Inputting the first region of interest into a trained remote sensing target detection model for target detection to identify a first vehicle and a type thereof from the first region of interest, and inputting the second region of interest into a trained remote sensing target detection model for target detection to identify a second vehicle and a type thereof from the second region of interest;
determining coordinate information of a first vehicle in the first remote sensing image, counting the total number of the first vehicles, determining coordinate information of a second vehicle in the second remote sensing image, and counting the total number of the second vehicles;
determining the coordinate information and type of the first vehicle and the total number of the first vehicles as first vehicle information, and determining the coordinate information and type of the second vehicle and the total number of the second vehicles as second vehicle information;
wherein the step of determining traffic flow information of the detection area within the preset interval time according to the first vehicle information and the second vehicle information includes:
calculating the center coordinates of a first vehicle in the first remote sensing image according to the coordinate information of the first vehicle, and calculating the center coordinates of a second vehicle in the second remote sensing image according to the coordinate information of the second vehicle;
Calculating the vehicle flow speed of the detection area in the preset interval time according to the center coordinates of the first vehicle and the center coordinates of the second vehicle;
respectively comparing the type of the first vehicle with the type of the second vehicle, and comparing the total number of the first vehicles with the total number of the second vehicles to obtain the traffic flow variation;
and determining the traffic flow variation and the calculated traffic flow speed as traffic flow information of the detection area in the preset interval time.
2. The traffic flow monitoring method according to claim 1, wherein the steps of extracting a first region of interest from the first remote sensing image and extracting a second region of interest from the second remote sensing image by the trained semantic segmentation model respectively include:
inputting the first remote sensing image into a trained semantic segmentation model to identify a road skeleton from the first remote sensing image as a first region of interest;
and inputting the second remote sensing image into the trained semantic segmentation model to identify a road skeleton from the second remote sensing image as a second region of interest.
3. The traffic flow monitoring method according to claim 1, wherein the step of calculating the traffic flow speed of the detection area within the preset interval time from the center coordinates of the first vehicle and the center coordinates of the second vehicle includes:
Calculating the overall center coordinates of all the first vehicles in the first remote sensing image according to the center coordinates of the first vehicles, and calculating the overall center coordinates of all the second vehicles in the second remote sensing image according to the center coordinates of the second vehicles;
acquiring the scaling of the first remote sensing image or the second remote sensing image;
and calculating the vehicle flow speed of the detection area in the preset interval time according to the integral center coordinates of the first vehicle and the integral center coordinates of the second vehicle and the scaling.
4. The traffic flow monitoring method according to claim 3, wherein the step of calculating the traffic flow speed of the detection region within the preset interval time based on the global center coordinates of the first vehicle and the global center coordinates of the second vehicle, and the scaling ratio includes:
calculating a traffic speed based on a remote sensing image according to the overall center coordinates of the first vehicle and the overall center coordinates of the second vehicle;
and calculating the product of the traffic flow speed based on the remote sensing image and the scaling ratio to obtain the traffic flow speed of the detection area in the preset interval time.
5. The traffic flow monitoring method according to claim 1, wherein after the step of determining traffic flow information of the detection area within the preset interval time based on the first vehicle information and the second vehicle information, comprising:
and sending the traffic flow information to a vehicle commanding and dispatching system so that the vehicle commanding and dispatching system can issue the traffic flow information to vehicles within a preset distance.
6. A traffic flow monitoring device, characterized in that the traffic flow monitoring device comprises:
the acquisition module is used for acquiring a first remote sensing image of a detection area and acquiring a second remote sensing image of the detection area after a preset interval time, wherein the first remote sensing image and the second remote sensing image are unchanged in background, and the changed part is caused by a moving vehicle;
the extraction module is used for respectively extracting a first region of interest from the first remote sensing image and extracting a second region of interest from the second remote sensing image through the trained semantic segmentation model;
the detection module is used for respectively carrying out target detection on the first region of interest and the second region of interest through a remote sensing target detection model which is completed through training so as to acquire first vehicle information in the first region of interest and second vehicle information in the second region of interest;
The determining module is used for determining traffic flow information of the detection area in the preset interval time according to the first vehicle information and the second vehicle information;
wherein, the detection module is specifically used for:
inputting the first region of interest into a trained remote sensing target detection model for target detection to identify a first vehicle and a type thereof from the first region of interest, and inputting the second region of interest into a trained remote sensing target detection model for target detection to identify a second vehicle and a type thereof from the second region of interest;
determining coordinate information of a first vehicle in the first remote sensing image, counting the total number of the first vehicles, determining coordinate information of a second vehicle in the second remote sensing image, and counting the total number of the second vehicles;
determining the coordinate information and type of the first vehicle and the total number of the first vehicles as first vehicle information, and determining the coordinate information and type of the second vehicle and the total number of the second vehicles as second vehicle information;
the determining module is specifically configured to:
calculating the center coordinates of a first vehicle in the first remote sensing image according to the coordinate information of the first vehicle, and calculating the center coordinates of a second vehicle in the second remote sensing image according to the coordinate information of the second vehicle;
Calculating the vehicle flow speed of the detection area in the preset interval time according to the center coordinates of the first vehicle and the center coordinates of the second vehicle;
respectively comparing the type of the first vehicle with the type of the second vehicle, and comparing the total number of the first vehicles with the total number of the second vehicles to obtain the traffic flow variation;
and determining the traffic flow variation and the calculated traffic flow speed as traffic flow information of the detection area in the preset interval time.
7. A traffic flow monitoring device comprising a processor, a memory, and a traffic flow monitoring program stored on the memory and executable by the processor, wherein the traffic flow monitoring program when executed by the processor implements the steps of the traffic flow monitoring method according to any one of claims 1 to 5.
8. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a traffic flow monitoring program which, when executed by a processor, implements the steps of the traffic flow monitoring method according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910975456.2A CN110738169B (en) | 2019-10-11 | 2019-10-11 | Traffic flow monitoring method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910975456.2A CN110738169B (en) | 2019-10-11 | 2019-10-11 | Traffic flow monitoring method, device, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110738169A CN110738169A (en) | 2020-01-31 |
| CN110738169B true CN110738169B (en) | 2023-09-08 |
Family
ID=69268925
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910975456.2A Active CN110738169B (en) | 2019-10-11 | 2019-10-11 | Traffic flow monitoring method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110738169B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112101167B (en) * | 2020-09-08 | 2021-05-14 | 广东新禾道信息科技有限公司 | Remote sensing image data processing method and device |
| CN113505638B (en) * | 2021-05-27 | 2024-04-02 | 中国科学院深圳先进技术研究院 | Method and device for monitoring traffic flow and computer readable storage medium |
| CN113984767A (en) * | 2021-11-24 | 2022-01-28 | 牧原肉食品有限公司 | System, method, apparatus and computer storage medium for livestock carcass quality detection |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101968924A (en) * | 2010-10-19 | 2011-02-09 | 北方工业大学 | High-resolution image-based real-time traffic flow detection method |
| CN106780603A (en) * | 2016-12-09 | 2017-05-31 | 宇龙计算机通信科技(深圳)有限公司 | Vehicle checking method, device and electronic equipment |
| CN108986465A (en) * | 2018-07-27 | 2018-12-11 | 深圳大学 | A kind of method of vehicle Flow Detection, system and terminal device |
| CN109492588A (en) * | 2018-11-12 | 2019-03-19 | 广西交通科学研究院有限公司 | A kind of rapid vehicle detection and classification method based on artificial intelligence |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6970102B2 (en) * | 2003-05-05 | 2005-11-29 | Transol Pty Ltd | Traffic violation detection, recording and evidence processing system |
-
2019
- 2019-10-11 CN CN201910975456.2A patent/CN110738169B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101968924A (en) * | 2010-10-19 | 2011-02-09 | 北方工业大学 | High-resolution image-based real-time traffic flow detection method |
| CN106780603A (en) * | 2016-12-09 | 2017-05-31 | 宇龙计算机通信科技(深圳)有限公司 | Vehicle checking method, device and electronic equipment |
| CN108986465A (en) * | 2018-07-27 | 2018-12-11 | 深圳大学 | A kind of method of vehicle Flow Detection, system and terminal device |
| CN109492588A (en) * | 2018-11-12 | 2019-03-19 | 广西交通科学研究院有限公司 | A kind of rapid vehicle detection and classification method based on artificial intelligence |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110738169A (en) | 2020-01-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110738169B (en) | Traffic flow monitoring method, device, equipment and computer readable storage medium | |
| CN109947793B (en) | Method and device for analyzing accompanying relationship and storage medium | |
| CN108550259B (en) | Road congestion judging method, terminal device and computer readable storage medium | |
| JP6741107B2 (en) | Real-time detection of traffic conditions | |
| KR102143108B1 (en) | Lane recognition modeling method, device, storage medium and device, and recognition method, device, storage medium and device | |
| US20230144209A1 (en) | Lane line detection method and related device | |
| US11967132B2 (en) | Lane marking detecting method, apparatus, electronic device, storage medium, and vehicle | |
| CN104167109A (en) | Detection method and detection apparatus for vehicle position | |
| CN112078571B (en) | Automatic parking method, automatic parking equipment, storage medium and automatic parking device | |
| US11837084B2 (en) | Traffic flow estimation apparatus, traffic flow estimation method, traffic flow estimation program, and storage medium storing traffic flow estimation program | |
| CN111126209B (en) | Lane line detection method and related equipment | |
| CN111898491A (en) | Method and device for identifying reverse driving of vehicle and electronic equipment | |
| JP5578986B2 (en) | Weather radar observation information providing system and weather radar observation information providing method | |
| CN110838230A (en) | Mobile video monitoring method, monitoring center and system | |
| CN111931683A (en) | Image recognition method, image recognition device and computer-readable storage medium | |
| CN113221756A (en) | Traffic sign detection method and related equipment | |
| CN110793531B (en) | Road matching method and device and readable storage medium | |
| CN110969864A (en) | Vehicle speed detection method, vehicle driving event detection method and electronic equipment | |
| CN110019608B (en) | Information acquisition method, device and system and storage equipment | |
| JP2022544348A (en) | Methods and systems for identifying objects | |
| CN112419542B (en) | Road error correction reporting method, device, equipment and storage medium | |
| CN104567887A (en) | Path matching method and device | |
| CN112036268B (en) | Component identification method and related device | |
| CN114166234A (en) | System, method, device, processor and computer storage medium for selecting navigation route and road damage identification early warning based on road damage measurement | |
| CN108010319B (en) | Road state identification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |