CN114822043B - Road congestion detection method and device and electronic equipment - Google Patents
Road congestion detection method and device and electronic equipment Download PDFInfo
- Publication number
- CN114822043B CN114822043B CN202210747704.XA CN202210747704A CN114822043B CN 114822043 B CN114822043 B CN 114822043B CN 202210747704 A CN202210747704 A CN 202210747704A CN 114822043 B CN114822043 B CN 114822043B
- Authority
- CN
- China
- Prior art keywords
- congestion
- coordinates
- road
- vehicle
- target vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0125—Traffic data processing
- G08G1/0129—Traffic data processing for creating historical data or processing based on historical data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0137—Measuring and analyzing of parameters relative to traffic conditions for specific applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30236—Traffic on road, railway or crossing
Abstract
The invention provides a road congestion detection method and device and electronic equipment. According to the method, historical data are adopted to be fitted on a three-dimensional space to obtain a curved surface feature matrix corresponding to each congestion level, the comparison speed is determined according to the curved surface feature matrix, then the congestion level is determined according to the comparison result of the comparison speed and the current pixel speed, and compared with the method that congestion level judgment is carried out through a fixed threshold value which is manually set, the accuracy of road congestion detection can be improved.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a road congestion detection method and device and electronic equipment.
Background
Under the condition that the existing expressway monitoring cameras are numerous, the road vehicle speed cannot be detected in a mode of presetting an interest area due to the fact that the positions, heights and angles of different cameras are different, and under different inclination angles, the front vehicle and the rear vehicle are shielded, the distance between the vehicles cannot be estimated, and road congestion cannot be accurately detected.
Disclosure of Invention
The invention aims to provide a road congestion detection method, a road congestion detection device and electronic equipment, which can improve the accuracy of road congestion detection.
In order to achieve the above object, the present invention provides a road congestion detection method, comprising the steps of:
step S1, collecting multiple groups of historical image data corresponding to different congestion levels of a first road, wherein each group of historical image data comprises multiple three-dimensional discrete points, the coordinates of the x axis and the y axis of each three-dimensional discrete point are the coordinates of the center point of a detection frame of a vehicle in the image, and the coordinates of the z axis of each three-dimensional discrete point are the pixel speed of the vehicle in the image;
step S2, performing surface fitting on the multiple sets of historical image data, and calculating to obtain a surface feature matrix of the congestion level corresponding to each historical image data;
step S3, inputting the current image of the first road into the trained vehicle detection model to obtain the coordinates of the center point of the detection frame of the target vehicle in the nth frame picture and the coordinates of the center point of the detection frame in the mth frame picture, wherein n and m are positive integers;
step S4, calculating the current pixel speed of the target vehicle according to the coordinates of the center point of the detection frame of the target vehicle in the nth frame picture, the coordinates of the center point of the detection frame in the mth frame picture and the time difference between the nth frame picture and the mth frame picture;
step S5, calculating a plurality of comparison speeds corresponding to a plurality of different congestion levels of the target vehicle in the mth frame picture according to the coordinates of the central point of the detection frame of the target vehicle in the mth frame picture and the curved surface feature matrixes corresponding to the different congestion levels;
step S6, comparing the current pixel speed of the target vehicle with the plurality of comparison speeds, determining the comparison speed with the smallest difference from the current pixel speed of the target vehicle, and determining that the congestion level of the area corresponding to the center point coordinate of the detection frame of the target vehicle in the mth frame picture is the congestion level corresponding to the comparison speed with the smallest difference.
Optionally, performing surface fitting on the multiple sets of historical image data, and calculating a surface feature matrix of the congestion level corresponding to each set of historical image data specifically includes:
setting a quadric surface equation corresponding to a curved surface to be fitted as follows:
v=A*(x,y)；
v is the pixel speed of the vehicle in the image, x and y are coordinates of the central point of a detection frame of the vehicle in the image, and A is a coefficient matrix of a quadric surface equation;
and writing a plurality of three-dimensional discrete points in the current historical image data into the quadric surface equation to perform surface fitting, solving to obtain a coefficient matrix of the quadric surface equation, and taking the coefficient matrix as a surface characteristic matrix of the congestion level corresponding to the current historical image data.
Optionally, the formula for calculating the current pixel speed of the target vehicle is:
wherein the content of the first and second substances,
) The coordinates of the center point of the detection frame in the nth frame of picture,
) The coordinates of the center point of the frame are detected in the m-th frame picture for the target vehicle,
is the current pixel speed of the target vehicle.
Optionally, the method further comprises:
carrying out perspective transformation on the current image of the first road to obtain a top view corresponding to the current image of the first road;
carrying out lane line identification in a plan view to obtain the area of a road area;
acquiring the length and width of the detection frame of each vehicle in the mth frame of picture, and calculating the area of the detection frame of each vehicle;
and calculating the ratio of the sum of the areas of the detection frames of each vehicle to the area of the road area to obtain the occupancy rate of the current road.
Optionally, the method further includes verifying the congestion level determined in step S6 according to the occupancy of the current road, and determining whether the congestion level determined in step S6 is accurate.
Optionally, the plurality of sets of historical image data corresponding to different congestion levels includes four sets of historical image data corresponding to four congestion levels, namely severe congestion, moderate congestion, light congestion and smooth traffic.
Optionally, the trained vehicle recognition model firstly adopts the yolov5 network model to perform target recognition of the vehicle, then adopts the fastreid network model to perform feature extraction on each recognized vehicle, and then utilizes the extracted features to perform target tracking.
The invention also provides a road congestion level detection device, comprising:
the collection module is used for collecting multiple groups of historical image data corresponding to different congestion levels of a first road, each group of historical image data comprises multiple three-dimensional discrete points, coordinates of an x axis and a y axis of each three-dimensional discrete point are coordinates of a central point of a detection frame of a vehicle in the image, and coordinates of a z axis of each three-dimensional discrete point are pixel speeds of the vehicle in the image;
the fitting module is used for performing surface fitting on the multiple groups of historical image data and calculating to obtain a surface feature matrix of the congestion level corresponding to each group of historical image data;
the identification module is used for inputting the current image of the first road into the trained vehicle detection model to obtain the coordinates of the central point of the detection frame of the target vehicle in the nth frame of picture and the coordinates of the central point of the detection frame in the mth frame of picture, wherein n and m are positive integers;
the first calculation module is used for calculating the current pixel speed of the target vehicle according to the coordinate of the central point of the detection frame of the target vehicle in the nth frame image, the coordinate of the central point of the detection frame in the mth frame image and the time difference between the nth frame image and the mth frame image;
the second calculation module is used for calculating and obtaining a plurality of comparison speeds of the target vehicle corresponding to a plurality of different congestion levels in the mth frame picture according to the coordinates of the central point of the detection frame of the target vehicle in the mth frame picture and the curved surface feature matrixes corresponding to the different congestion levels;
and the comparison module is used for comparing the current pixel speed of the target vehicle with the plurality of comparison speeds, determining the comparison speed with the minimum difference between the current pixel speed of the target vehicle and the current pixel speed of the target vehicle, and judging that the congestion level of the area corresponding to the center point coordinate of the detection frame of the target vehicle in the mth frame picture is the congestion level corresponding to the comparison speed with the minimum difference.
The present invention also provides an electronic device comprising: a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the above method.
The invention has the beneficial effects that: the invention provides a road congestion detection method, which comprises the following steps: step S1, collecting multiple groups of historical image data corresponding to different congestion levels of a first road, wherein each group of historical image data comprises multiple three-dimensional discrete points, the coordinates of the x axis and the y axis of each three-dimensional discrete point are the coordinates of the center point of a detection frame of a vehicle in the image, and the coordinates of the z axis of each three-dimensional discrete point are the pixel speed of the vehicle in the image; step S2, performing surface fitting on the multiple groups of historical image data, and calculating to obtain a surface feature matrix of the congestion level corresponding to each group of historical image data; step S3, inputting the current image of the first road into the trained vehicle detection model to obtain the coordinates of the center point of the detection frame of the target vehicle in the nth frame picture and the coordinates of the center point of the detection frame in the mth frame picture, wherein n and m are positive integers; step S4, calculating the current pixel speed of the target vehicle according to the coordinates of the center point of the detection frame of the target vehicle in the nth frame picture, the coordinates of the center point of the detection frame in the mth frame picture and the time difference between the nth frame picture and the mth frame picture; step S5, calculating a plurality of comparison speeds respectively corresponding to a plurality of different congestion levels of the target vehicle in the mth frame picture according to the coordinates of the central point of the detection frame of the target vehicle in the mth frame picture and the curved surface feature matrixes corresponding to the different congestion levels; step S6, comparing the current pixel speed of the target vehicle with the plurality of comparison speeds, determining the comparison speed with the minimum difference between the current pixel speed of the target vehicle and the current pixel speed of the target vehicle, determining that the congestion level of the area corresponding to the center point coordinate of the detection frame of the target vehicle in the mth frame picture is the congestion level corresponding to the comparison speed with the minimum difference, performing congestion level determination by using the comparison speed obtained by fitting the historical data in the three-dimensional space, and improving the accuracy of road congestion detection compared with the determination by manually setting a fixed threshold.
Drawings
For a better understanding of the nature and technical aspects of the present invention, reference should be made to the following detailed description of the invention, taken in conjunction with the accompanying drawings, which are provided for purposes of illustration and description and are not intended to limit the invention.
In the drawings, there is shown in the drawings,
FIG. 1 is a flow chart of a road congestion detection method of the present invention;
FIG. 2 is a schematic diagram of a fitted surface according to an embodiment of the road congestion detection method of the present invention;
FIG. 3 is an image before perspective transformation in an embodiment of a method for detecting road congestion according to the present invention;
FIG. 4 is a perspective transformed image in an embodiment of the road congestion detection method of the present invention;
fig. 5 is a flowchart of calculating the occupancy of the current road in the road congestion detection method of the present invention;
FIG. 6 is an image before perspective transformation in another embodiment of the road congestion detection method of the present invention;
FIG. 7 is a perspective transformed image in another embodiment of the road congestion detection method of the present invention;
fig. 8 is a schematic diagram illustrating the calculation of the occupancy rate of the current road in another embodiment of the road congestion detection method according to the present invention;
FIG. 9 is a schematic view of a road congestion detection apparatus of the present invention;
fig. 10 is a schematic diagram of an electronic device of the present invention.
Detailed Description
To further illustrate the technical means and effects of the present invention, the following detailed description is given with reference to the preferred embodiments of the present invention and the accompanying drawings.
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the disclosure.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
Referring to fig. 1, an embodiment of the invention provides a method for detecting road congestion, including the following steps:
step S1, collecting multiple groups of historical image data corresponding to different congestion levels of the first road, wherein each group of historical image data comprises multiple three-dimensional discrete points, coordinates of an x axis and a y axis of each three-dimensional discrete point are coordinates of a center point of a detection frame of the vehicle in the image, and coordinates of a z axis of each three-dimensional discrete point are pixel speeds of the vehicle in the image.
In particular, each set of historical image data includes a plurality of three-dimensional discrete points, which may be represented, for example, in some embodiments of the invention as the following data sets:
{(
),(
),(
).....(
),(
)}；
wherein the content of the first and second substances,
coordinates of a center point of a detection frame representing the m-th vehicle,
showing the m-th vehicle at coordinates: (
) Pixel velocity of the vehicle acquired at the location.
It should be noted that the plurality of sets of historical image data corresponding to different congestion levels include four sets of historical image data corresponding to four congestion levels, namely, severe congestion, moderate congestion, mild congestion and smooth congestion.
And step S2, performing surface fitting on the multiple sets of historical image data, and calculating to obtain a surface feature matrix of the congestion level corresponding to each set of historical image data.
Specifically, performing surface fitting on the multiple sets of historical image data, and calculating a surface feature matrix of the congestion level corresponding to each set of historical image data includes:
setting a quadric surface equation corresponding to the curved surface to be fitted as follows:
v=A*(x,y)；
wherein v is the pixel speed of the vehicle in the image, x and y are the coordinates of the central point of the detection frame of the vehicle in the image, and A is the coefficient matrix of the quadric equation;
and writing a plurality of three-dimensional discrete points in the current historical image data into the quadric surface equation for surface fitting, solving to obtain a coefficient matrix of the quadric surface equation, and taking the coefficient matrix as a surface characteristic matrix of the congestion level corresponding to the current historical image data.
Further, in some embodiments of the present invention, the multiple sets of historical image data corresponding to different congestion levels include four sets of historical image data corresponding to four congestion levels, namely, severe congestion, moderate congestion, light congestion and clear congestion, and the corresponding calculated curved surface feature matrix also includes four sets, namely, a1, a2, A3 and a4, and a1, a2, A3 and a4 respectively correspond to four congestion levels, namely, severe congestion, moderate congestion, light congestion and clear congestion.
Taking one of the curved surface feature matrices as an example, the calculation process of the curved surface feature matrix is explained as follows:
assuming a quadratic surface equation of
And (x, y, v) is a three-dimensional discrete point, and the quadric surface equation is rewritten as follows:
the curved surface obtained by fitting is shown in fig. 2, and a matrix a can be obtained by solving in combination with the historical image data and the equation, where the matrix a is a curved surface feature matrix corresponding to the historical image data (congestion level), and by analogy, four curved surface feature matrices a1, a2, A3, and a4 corresponding to four congestion levels of severe congestion, moderate congestion, mild congestion, and clear congestion are obtained.
Step S3, inputting the current image of the first road into the trained vehicle detection model to obtain the coordinates of the center point of the detection frame of the target vehicle in the nth frame picture and the coordinates of the center point of the detection frame in the mth frame picture, wherein n and m are positive integers.
Specifically, the trained vehicle identification model firstly adopts the yolov5 network model to identify the target of the vehicle, then adopts the fastreid network model to extract the characteristics of each identified vehicle, and then utilizes the extracted characteristics to track the target, wherein the characteristics of each vehicle are 512-dimensional characteristic vectors, and the fastreid model can well extract the appearance characteristics and local characteristics of the vehicle, so that the better tracking effect is obtained for the vehicle of each frame in the video.
And step S4, calculating the current pixel speed of the target vehicle according to the coordinates of the center point of the detection frame of the target vehicle in the nth frame picture, the coordinates of the center point of the detection frame in the mth frame picture and the time difference between the nth frame picture and the mth frame picture.
Specifically, the formula for calculating the current pixel speed of the target vehicle is as follows:
wherein, the first and the second end of the pipe are connected with each other,
) The coordinates of the center point of the detection frame in the nth frame of picture,
) The coordinates of the center point of the frame are detected in the m-th frame picture for the target vehicle,
is the current pixel speed of the target vehicle.
It should be noted that, the pixel speed of the vehicle under the current lens is obtained by using the pixel displacement between the two frames, and since the pixel speed of the vehicle changes nonlinearly in the process of the vehicle from near to far in the image, the pixel speeds of the vehicle at different positions are different, and the congestion level cannot be determined directly by comparing the pixel speed with a predetermined fixed threshold, and the threshold (comparison speed) for comparing with the pixel speed should be able to change with the position of the vehicle cabin in the image, that is, the determination of the congestion level requires the position of the vehicle in the image and the pixel speed to be integrated for determination.
And step S5, calculating a plurality of comparison speeds corresponding to a plurality of different congestion levels of the target vehicle in the mth frame picture according to the coordinates of the central point of the detection frame of the target vehicle in the mth frame picture and the curved surface feature matrixes corresponding to the different congestion levels.
Specifically, in an embodiment of the present invention, the position coordinates (coordinates of the center point of the detection frame) of the ith vehicle in the image are (xi, yi), and the pixel speed is vi, and then the position coordinates (xi, yi) of the ith vehicle in the image are respectively substituted into the formulas
Then four comparison speeds respectively corresponding to the four congestion levels of severe congestion, moderate congestion, light congestion and smooth traffic can be obtainedSuch as: comparison speed in severe congestion
Comparison speed in the case of moderate congestion
Comparison speed in light traffic congestion
Comparison speed in case of unblocked state
。
Therefore, the comparison speed in the method of the invention is dynamically changed along with the position of the vehicle in the image, and is not a fixed threshold, so that the error caused by comparison of the fixed threshold can be reduced.
Step S6, comparing the current pixel speed of the target vehicle with the plurality of comparison speeds, determining the comparison speed with the minimum difference from the current pixel speed of the target vehicle, and determining that the congestion level of the area corresponding to the center point coordinate of the detection frame of the target vehicle in the mth frame picture is the congestion level corresponding to the comparison speed with the minimum difference.
In particular, in some embodiments of the invention, the current velocity v is calculated separately i The distances d of the four comparison speeds are respectively:
absolute value of the comparison speed difference with severe congestion:
；
absolute value of comparison speed difference with medium congestion:
；
absolute value of comparison speed difference with light congestion:
；
absolute value of the comparison difference with clear speed:
；
according to 4 speed difference minimum min
,
,
,
Determine the current congestion level, e.g. d, corresponding thereto 3 And when the congestion level is the minimum value, determining that the congestion level at the moment is light congestion.
Further, as shown in fig. 5, in order to further improve the accuracy of congestion level detection, the present invention further detects the occupancy of the current road as an aid to congestion level determination, wherein due to the inconsistency of the camera mounting angles on the road, substantially all the cameras are at a certain angle with the horizontal plane, the mounting angles in different scenes are not fixed, the vehicles are from near to far in the road, and the vehicles close to the lens side can block the vehicles in front to cause the distance between the vehicles to be covered, thereby affecting the accuracy of occupancy calculation of the vehicles in the road, and therefore, the present invention proposes the following method to calculate the occupancy of the current road:
first, the current image of the first road shown in fig. 3 or 6 is subjected to perspective transformation to obtain a top view corresponding to the current image of the first road shown in fig. 4 or 7, specifically, four vertexes of a quadrangle are set along the lane lines on both sides of the road in the current image of the first road
,
,
,
Then four target coordinate points corresponding to the twisted image are obtained
,
,
,
And obtaining a perspective transformation matrix M through a perspective transformation function, and then obtaining a corresponding top view after performing perspective transformation on the current image of the first road through the obtained torsion matrix M.
Next, as shown in fig. 8, lane line recognition is performed in a plan view to obtain an area S of a road region;
acquiring the length w and the width h of the detection frame of each vehicle in the mth frame of picture, and calculating the area w x h of the detection frame of each vehicle;
calculating the ratio of the sum of the areas of the detection frames of each vehicle to the area of the road area to obtain the occupancy rate of the current road
Where ρ is the occupancy rate of the current road and j is the number of vehicles on the current road.
Finally, the method also comprises the steps of verifying the congestion level determined in the step S6 according to the current road occupancy rate, and judging whether the congestion level determined in the step S6 is accurate, namely the method outputs the congestion level obtained by comparing pixel speeds and the current road occupancy rate which are required to be comprehensively provided for judging the road congestion level, so that the misjudgment caused by only meeting one condition can be effectively solved, and the method has higher robustness and higher accuracy.
Referring to fig. 9, the present invention further provides a road congestion level detection apparatus, including:
the collecting module 10 is configured to collect multiple sets of historical image data corresponding to different congestion levels of a first road, where each set of historical image data includes multiple three-dimensional discrete points, coordinates of an x axis and a y axis of each three-dimensional discrete point are coordinates of a center point of a detection frame of a vehicle in an image, and coordinates of a z axis of each three-dimensional discrete point are pixel speeds of the vehicle in the image;
the fitting module 20 is configured to perform surface fitting on the multiple sets of historical image data, and calculate a surface feature matrix of a congestion level corresponding to each historical image data;
the recognition module 30 is configured to input the current image of the first road into the trained vehicle detection model, so as to obtain a center point coordinate of the detection frame of the target vehicle in the nth frame of picture and a center point coordinate of the detection frame in the mth frame of picture, where n and m are positive integers;
the first calculating module 40 is configured to calculate a current pixel speed of the target vehicle according to a center point coordinate of the detection frame of the target vehicle in the nth frame picture, a center point coordinate of the detection frame in the mth frame picture, and a time difference between the nth frame picture and the mth frame picture;
the second calculation module 50 is configured to calculate, according to the coordinates of the central point of the detection frame of the target vehicle in the mth frame image and the curved surface feature matrices corresponding to different congestion levels, a plurality of comparison speeds corresponding to a plurality of different congestion levels of the target vehicle in the mth frame image;
and a comparing module 60, configured to compare the current pixel speed of the target vehicle with the plurality of comparison speeds, determine a comparison speed with the smallest difference from the current pixel speed of the target vehicle, and determine that the congestion level of the area corresponding to the center point coordinate of the detection frame of the m-th frame of the target vehicle is the congestion level corresponding to the comparison speed with the smallest difference.
Referring to fig. 10, the present invention further provides an electronic device, including: a memory 100 and a processor 200, the memory 100 storing a computer program which, when executed by the processor 200, causes the processor 200 to perform the steps of the above-described method.
In summary, the present invention provides a road congestion detection method, including the following steps: step S1, collecting multiple groups of historical image data corresponding to different congestion levels of a first road, wherein each group of historical image data comprises multiple three-dimensional discrete points, the coordinates of the x axis and the y axis of each three-dimensional discrete point are the coordinates of the center point of a detection frame of a vehicle in the image, and the coordinates of the z axis of each three-dimensional discrete point are the pixel speed of the vehicle in the image; step S2, performing surface fitting on the multiple sets of historical image data, and calculating to obtain a surface feature matrix of the congestion level corresponding to each historical image data; step S3, inputting the current image of the first road into a trained vehicle detection model to obtain the center point coordinate of a detection frame of a target vehicle in the nth frame picture and the center point coordinate of the detection frame in the mth frame picture, wherein n and m are positive integers; step S4, calculating the current pixel speed of the target vehicle according to the coordinates of the center point of the detection frame of the target vehicle in the nth frame picture, the coordinates of the center point of the detection frame in the mth frame picture and the time difference between the nth frame picture and the mth frame picture; step S5, calculating a plurality of comparison speeds respectively corresponding to a plurality of different congestion levels of the target vehicle in the mth frame picture according to the coordinates of the central point of the detection frame of the target vehicle in the mth frame picture and the curved surface feature matrixes corresponding to the different congestion levels; step S6, comparing the current pixel speed of the target vehicle with the plurality of comparison speeds, determining the comparison speed with the minimum difference between the current pixel speed of the target vehicle and the current pixel speed of the target vehicle, determining that the congestion level of the area corresponding to the center point coordinate of the detection frame of the target vehicle in the mth frame picture is the congestion level corresponding to the comparison speed with the minimum difference, performing congestion level determination by using the comparison speed obtained by fitting the historical data in the three-dimensional space, and improving the accuracy of road congestion detection compared with the determination by manually setting a fixed threshold.
As described above, it will be apparent to those skilled in the art that other various changes and modifications may be made based on the technical solution and concept of the present invention, and all such changes and modifications are intended to fall within the scope of the appended claims.
Claims (9)
1. A road congestion detection method is characterized by comprising the following steps:
step S1, collecting multiple groups of historical image data corresponding to different congestion levels of a first road, wherein each group of historical image data comprises multiple three-dimensional discrete points, the coordinates of the x axis and the y axis of each three-dimensional discrete point are the coordinates of the center point of a detection frame of a vehicle in the image, and the coordinates of the z axis of each three-dimensional discrete point are the pixel speed of the vehicle in the image;
step S2, performing surface fitting on the multiple sets of historical image data, and calculating to obtain a surface feature matrix of the congestion level corresponding to each historical image data;
step S3, inputting the current image of the first road into the trained vehicle detection model to obtain the coordinates of the center point of the detection frame of the target vehicle in the nth frame picture and the coordinates of the center point of the detection frame in the mth frame picture, wherein n and m are positive integers;
step S4, calculating the current pixel speed of the target vehicle according to the coordinates of the center point of the detection frame of the target vehicle in the nth frame picture, the coordinates of the center point of the detection frame in the mth frame picture and the time difference between the nth frame picture and the mth frame picture;
step S5, calculating a plurality of comparison speeds respectively corresponding to a plurality of different congestion levels of the target vehicle in the mth frame picture according to the coordinates of the central point of the detection frame of the target vehicle in the mth frame picture and the curved surface feature matrixes corresponding to the different congestion levels;
step S6, comparing the current pixel speed of the target vehicle with the plurality of comparison speeds, determining the comparison speed with the minimum difference from the current pixel speed of the target vehicle, and determining that the congestion level of the area corresponding to the center point coordinate of the detection frame of the target vehicle in the mth frame picture is the congestion level corresponding to the comparison speed with the minimum difference.
2. The method for detecting road congestion according to claim 1, wherein performing surface fitting on the plurality of sets of historical image data, and calculating a surface feature matrix of the congestion level corresponding to each set of historical image data specifically includes:
setting a quadric surface equation corresponding to the curved surface to be fitted as follows:
v=A*(x,y)；
wherein v is the pixel speed of the vehicle in the image, x and y are the coordinates of the central point of the detection frame of the vehicle in the image, and A is the coefficient matrix of the quadric equation;
and writing a plurality of three-dimensional discrete points in the current historical image data into the quadric surface equation for surface fitting, solving to obtain a coefficient matrix of the quadric surface equation, and taking the coefficient matrix as a surface characteristic matrix of the congestion level corresponding to the current historical image data.
3. The road congestion detection method according to claim 1, wherein the formula for calculating the current pixel speed of the target vehicle is:
wherein the content of the first and second substances,
) The coordinates of the center point of the detection frame in the nth frame of picture,
) The coordinates of the center point of the frame are detected in the m-th frame picture for the target vehicle,
is the current pixel speed of the target vehicle.
4. The road congestion detection method according to claim 1, further comprising:
carrying out perspective transformation on the current image of the first road to obtain a top view corresponding to the current image of the first road;
carrying out lane line identification in a plan view to obtain the area of a road area;
acquiring the length and width of the detection frame of each vehicle in the mth frame picture, and calculating the area of the detection frame of each vehicle;
and calculating the ratio of the sum of the areas of the detection frames of all the vehicles to the area of the road region to obtain the occupancy rate of the current road.
5. The road congestion detection method according to claim 4, further comprising verifying the congestion level determined in step S6 according to the occupancy of the current road, and determining whether the congestion level determined in step S6 is accurate.
6. The road congestion detection method according to claim 1, wherein the plurality of sets of historical image data corresponding to different congestion levels include four sets of historical image data corresponding to four congestion levels of severe congestion, moderate congestion, mild congestion, and clear congestion.
7. The road congestion detection method according to claim 1, wherein the trained vehicle recognition model firstly adopts yolov5 network model to perform target recognition of vehicles, then adopts fastreid network model to perform feature extraction on each recognized vehicle, and then uses the extracted features to perform target tracking.
8. A road congestion level detection device, comprising:
the collection module is used for collecting multiple groups of historical image data corresponding to different congestion levels of a first road, each group of historical image data comprises multiple three-dimensional discrete points, coordinates of an x axis and a y axis of each three-dimensional discrete point are coordinates of a central point of a detection frame of a vehicle in the image, and coordinates of a z axis of each three-dimensional discrete point are pixel speeds of the vehicle in the image;
the fitting module is used for performing surface fitting on the multiple groups of historical image data and calculating to obtain a surface feature matrix of the congestion level corresponding to each group of historical image data;
the identification module is used for inputting the current image of the first road into the trained vehicle detection model to obtain the coordinates of the central point of the detection frame of the target vehicle in the nth frame of picture and the coordinates of the central point of the detection frame in the mth frame of picture, wherein n and m are positive integers;
the first calculation module is used for calculating the current pixel speed of the target vehicle according to the coordinates of the central point of the detection frame of the target vehicle in the nth frame picture, the coordinates of the central point of the detection frame in the mth frame picture and the time difference between the nth frame picture and the mth frame picture;
the second calculation module is used for calculating and obtaining a plurality of comparison speeds of the target vehicle corresponding to a plurality of different congestion levels in the mth frame picture according to the coordinates of the central point of the detection frame of the target vehicle in the mth frame picture and the curved surface feature matrixes corresponding to the different congestion levels;
and the comparison module is used for comparing the current pixel speed of the target vehicle with the plurality of comparison speeds, determining the comparison speed with the minimum difference between the current pixel speed of the target vehicle and the current pixel speed of the target vehicle, and judging that the congestion level of the area corresponding to the center point coordinate of the detection frame of the target vehicle in the mth frame picture is the congestion level corresponding to the comparison speed with the minimum difference.
9. An electronic device, comprising: a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210747704.XA CN114822043B (en) | 2022-06-29 | 2022-06-29 | Road congestion detection method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210747704.XA CN114822043B (en) | 2022-06-29 | 2022-06-29 | Road congestion detection method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114822043A CN114822043A (en) | 2022-07-29 |
| CN114822043B true CN114822043B (en) | 2022-09-13 |
Family
ID=82523537
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210747704.XA Active CN114822043B (en) | 2022-06-29 | 2022-06-29 | Road congestion detection method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114822043B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH07253476A (en) * | 1994-01-26 | 1995-10-03 | Matsushita Electric Works Ltd | Image type sensor |
| JP2005004426A (en) * | 2003-06-11 | 2005-01-06 | Toshiba Corp | Instrument and system for automatic traffic volume measurement |
| CN1707544A (en) * | 2005-05-26 | 2005-12-14 | 上海交通大学 | Method for estimating city road network traffic flow state |
| CN104282020A (en) * | 2014-09-22 | 2015-01-14 | 中海网络科技股份有限公司 | Vehicle speed detection method based on target motion track |
| CN105844328A (en) * | 2015-01-15 | 2016-08-10 | 开利公司 | Method applied to automatic commissioning personnel counting system and automatic commissioning personnel counting system |
| CN107301369A (en) * | 2017-09-04 | 2017-10-27 | 南京航空航天大学 | Road traffic congestion analysis method based on Aerial Images |
| CN109147331A (en) * | 2018-10-11 | 2019-01-04 | 青岛大学 | A kind of congestion in road condition detection method based on computer vision |
| CN110992693A (en) * | 2019-12-04 | 2020-04-10 | 浙江工业大学 | Deep learning-based traffic congestion degree multi-dimensional analysis method |
| CN111027447A (en) * | 2019-12-04 | 2020-04-17 | 浙江工业大学 | Road overflow real-time detection method based on deep learning |
| CN111605481A (en) * | 2019-08-27 | 2020-09-01 | 北京纵目安驰智能科技有限公司 | Congestion car following system and terminal based on look around |
| CN111724591A (en) * | 2020-06-12 | 2020-09-29 | 重庆大学 | Three-dimensional McMaster method for detecting traffic jam of section of highway |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6542808B2 (en) * | 1999-03-08 | 2003-04-01 | Josef Mintz | Method and system for mapping traffic congestion |
| WO2018017793A1 (en) * | 2016-07-21 | 2018-01-25 | Intelligent Technologies International, Inc. | System and method for creating, updating, and using maps generated by probe vehicles |
-
2022
- 2022-06-29 CN CN202210747704.XA patent/CN114822043B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH07253476A (en) * | 1994-01-26 | 1995-10-03 | Matsushita Electric Works Ltd | Image type sensor |
| JP2005004426A (en) * | 2003-06-11 | 2005-01-06 | Toshiba Corp | Instrument and system for automatic traffic volume measurement |
| CN1707544A (en) * | 2005-05-26 | 2005-12-14 | 上海交通大学 | Method for estimating city road network traffic flow state |
| CN104282020A (en) * | 2014-09-22 | 2015-01-14 | 中海网络科技股份有限公司 | Vehicle speed detection method based on target motion track |
| CN105844328A (en) * | 2015-01-15 | 2016-08-10 | 开利公司 | Method applied to automatic commissioning personnel counting system and automatic commissioning personnel counting system |
| CN107301369A (en) * | 2017-09-04 | 2017-10-27 | 南京航空航天大学 | Road traffic congestion analysis method based on Aerial Images |
| CN109147331A (en) * | 2018-10-11 | 2019-01-04 | 青岛大学 | A kind of congestion in road condition detection method based on computer vision |
| CN111605481A (en) * | 2019-08-27 | 2020-09-01 | 北京纵目安驰智能科技有限公司 | Congestion car following system and terminal based on look around |
| CN110992693A (en) * | 2019-12-04 | 2020-04-10 | 浙江工业大学 | Deep learning-based traffic congestion degree multi-dimensional analysis method |
| CN111027447A (en) * | 2019-12-04 | 2020-04-17 | 浙江工业大学 | Road overflow real-time detection method based on deep learning |
| CN111724591A (en) * | 2020-06-12 | 2020-09-29 | 重庆大学 | Three-dimensional McMaster method for detecting traffic jam of section of highway |
Non-Patent Citations (2)
| Title |
|---|
| THE APPLICATION OF THE BINOCULAR CAMERA TO CROWD DENSITY ESTIMATION;SEN GUO，SAN-FENG CHEN;《Proceedings of the 2012 International Conference on Machine Learning and Cybernetics》;20120731;全文 * |
| Video-Based Traffic Flow Monitoring Algorithm;Yongmei Zhang 等;《2020 IEEE 3rd International Conference on Computer and Communication Engineering Technology》;20201231;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114822043A (en) | 2022-07-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109345593B (en) | Camera posture detection method and device | |
| US9292750B2 (en) | Method and apparatus for detecting traffic monitoring video | |
| CN102609953B (en) | Many objects outward appearance of video camera and range-sensor data strengthens and merges | |
| US8184859B2 (en) | Road marking recognition apparatus and method | |
| US11398051B2 (en) | Vehicle camera calibration apparatus and method | |
| US20160019683A1 (en) | Object detection method and device | |
| CN109543493B (en) | Lane line detection method and device and electronic equipment | |
| US20020042676A1 (en) | Road lane marker recognition | |
| CN107750364A (en) | Detected using the road vertically profiling of stable coordinate system | |
| US20030190058A1 (en) | Apparatus and method for measuring queue length of vehicles | |
| CN112396122B (en) | Method and system for multiple optimization of target detector based on vertex distance and cross-over ratio | |
| CN107480646B (en) | Binocular vision-based vehicle-mounted video abnormal motion detection method | |
| US10235579B2 (en) | Vanishing point correction apparatus and method | |
| CN106599918B (en) | vehicle tracking method and system | |
| CN111898491A (en) | Method and device for identifying reverse driving of vehicle and electronic equipment | |
| CN106228531B (en) | Automatic vanishing point calibration method and system based on horizon line search | |
| CN111738071B (en) | Inverse perspective transformation method based on motion change of monocular camera | |
| WO2020146980A1 (en) | Lane line recognizing method, lane line recognizing device, and nonvolatile storage medium | |
| EP3726421A2 (en) | Recognition method and apparatus for false detection of an abandoned object and image processing device | |
| CN112347817B (en) | Video target detection and tracking method and device | |
| CN114822043B (en) | Road congestion detection method and device and electronic equipment | |
| CN113449632A (en) | Vision and radar perception algorithm optimization method and system based on fusion perception and automobile | |
| CN114037977B (en) | Road vanishing point detection method, device, equipment and storage medium | |
| CN115507815A (en) | Target ranging method and device and vehicle | |
| CN211293945U (en) | Simulated lane of Android platform lane departure early warning method of simulation robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |