CN115482474B

CN115482474B - Bridge deck vehicle load identification method and system based on aerial image

Info

Publication number: CN115482474B
Application number: CN202211021016.1A
Authority: CN
Inventors: 沈明燕; 党浩鹏; 舒小娟; 李贞贤; 孙华
Original assignee: Hunan University of Science and Technology
Current assignee: Hunan University of Science and Technology
Priority date: 2022-08-24
Filing date: 2022-08-24
Publication date: 2023-07-04
Anticipated expiration: 2042-08-24
Also published as: CN115482474A

Abstract

The invention provides a bridge deck vehicle load identification method and system based on aerial images, and belongs to the field of bridge deck operation safety. The method comprises the steps of collecting bridge deck vehicle images of aerial photography at high altitude, and constructing and calibrating a vehicle type and vehicle weight comparison database; preprocessing a bridge deck vehicle image, performing vehicle shadow filtering, and inputting the filtered image into a vehicle recognition model; identifying the vehicle through the vehicle identification model, and outputting the identified vehicle parameters; the vehicle parameters include: vehicle type, vehicle direction, vehicle number and vehicle distance; and obtaining the bridge deck vehicle load distribution through a statistical method according to the constructed vehicle comparison database and the identified vehicle parameters. The invention not only reduces the calculated amount of bridge deck vehicle load identification, but also improves the identification efficiency and the identification accuracy, and is suitable for instantaneous/long-term vehicle load monitoring of various bridges.

Description

Bridge deck vehicle load identification method and system based on aerial image

Technical Field

The invention belongs to the field of bridge operation safety, and particularly relates to a bridge deck vehicle load identification method and system based on aerial images.

Background

The vehicle load distribution of the bridge deck is an important influencing factor for evaluating the safety level during the operation of the bridge and is also an important basis for bridge design, state evaluation, maintenance and reinforcement.

In the prior art, the statistical analysis of the bridge deck vehicle load is mainly carried out in a manual statistical mode and in a video acquisition mode of installing a camera on a monitoring rod. The method based on the manual statistics has the defects of long time consumption, large workload, low working efficiency and the like; the traditional video acquisition mode is generally to acquire three-dimensional data of a vehicle by installing a plurality of groups of cameras on a monitoring rod or a bridge upright rod, because of the limitation of the height of traffic auxiliary facilities, the cameras have limited observation view in the bridge length direction, the overall distribution condition of the bridge deck vehicle at a certain moment is difficult to acquire, the vehicle weight information of the vehicle cannot be acquired, and the distribution condition of the bridge deck vehicle, the speed of the vehicle and other parameters cannot be accurately measured. In addition, traffic monitoring tasks are usually completed by a plurality of groups of cameras in a coordinated manner, and once one of the cameras fails, vehicle information can be lost, so that the statistical result of traffic is affected.

Disclosure of Invention

In view of the above, the embodiment of the invention provides a bridge deck vehicle load identification method and system based on aerial images, which can reduce the calculated amount and improve the identification efficiency and the identification accuracy.

In order to achieve the above purpose, the embodiment of the present invention adopts the following technical scheme:

in a first aspect, an embodiment of the present invention provides a bridge deck vehicle load identification method based on aerial images, including the following steps:

a bridge deck vehicle load identification method based on aerial images is characterized by comprising the following steps:

constructing a vehicle type and vehicle weight comparison database;

acquiring a bridge deck vehicle image of aerial photography at high altitude;

preprocessing the received bridge deck vehicle image, and then performing vehicle shadow filtration to obtain a clean bridge deck vehicle image;

constructing a vehicle identification model, inputting the clean bridge deck vehicle image into the vehicle identification model to identify the bridge deck vehicle, and outputting the identified vehicle parameters; the vehicle parameters include: vehicle type, vehicle direction, vehicle number and vehicle distance;

and obtaining the bridge deck vehicle load distribution through a statistical method according to the constructed vehicle comparison database and the identified vehicle parameters.

As a preferred embodiment of the invention, the acquisition of the aerial bridge deck vehicle image comprises

S21, arranging an unmanned aerial vehicle above a bridge deck, wherein the unmanned aerial vehicle carries a camera device through a cradle head, and the visual field range of the camera device is the whole bridge deck;

step S22, adjusting flight parameters, enabling the camera to perform nodding on the bridge deck, collecting each vehicle image of the whole bridge deck, and transmitting the vehicle images of the bridge deck to the upper computer.

As a preferred embodiment of the present invention, the height of the nodding is 50-200 m.

As a preferred embodiment of the present invention, the vehicle shadow filtering includes:

step S31, obtaining a three-color channel red R, green G and blue B brightness value matrix of the image according to the bridge deck vehicle image, wherein the three-color channel brightness value of the nth row number and the nth column pixel point is expressed as R _(m,n) ,G _(m,n) ,B _(m,n) ；

Step S32, counting the brightness values of three-color channels of the bridge deck, vehicles with various colors and the shadow parts of the vehicles in the aerial photograph to obtain maximum and minimum values as representative values, wherein the shadow parts are expressed as

Step S33, judging whether each pixel point in the image meets the requirement simultaneously

If the two types of the images are satisfied at the same time, judging that the images are shadows;

step S34, the pixel three-color channel brightness value (R _shadow ,G _shadow ,B _shadow ) Replaced by bridge deck three-color channel brightness value or brightness average value

Shadow filtering is completed.

As a preferred embodiment of the present invention, the vehicle recognition model is constructed using a YOLO-V3 network structure.

As a preferred embodiment of the present invention, the YOLO-V3 network structure does not have a pooling layer, and in terms of output tensors, 3 feature maps of different scales are output.

As a preferred embodiment of the present invention, the 3 different scale feature maps are implemented by dividing the original image with 3 different grids, including 16 x 16 grids for large objects, 26 x 26 grids for medium objects, and 52 x 52 grids for small objects.

As a preferred embodiment of the present invention, the identifying the bridge deck vehicle includes the steps of:

step S411, extracting first features based on the appearance and the outline of the vehicle, primarily classifying the vehicles in the image based on the first features, recognizing the position of the vehicle in the short-time difference by adopting vehicle types with obvious features and less quantity, and judging the running direction of the vehicle flow according to the position of the vehicle in the photo;

step S412, determining the width of the lanes by two vehicles with the farthest distance according to each traffic flow traveling direction, correspondingly dividing different lanes according to the lane widths, dividing each vehicle into respective lanes, and identifying the inter-vehicle distance in each lane;

step S413, extracting a second feature based on the aspect ratio of the vehicle, identifying the model of each vehicle in the image, and calculating the number of vehicles of each model.

As a preferred embodiment of the present invention, the vehicle distance parameter, according to the principle of small hole imaging, relates the size of the object U, the distance W of the small hole Φ from the camera, and the distance X from the camera, and the relationship is as follows:

and the vehicle recognition model obtains the inter-vehicle distance in the image according to the principle and the received bridge deck vehicle image.

In a second aspect, an embodiment of the present invention further provides a bridge deck vehicle load recognition system based on aerial images, where the system includes: the system comprises a vehicle comparison database module, an image acquisition module, an image processing module, a vehicle identification module and a vehicle data analysis module; wherein, the liquid crystal display device comprises a liquid crystal display device,

the image acquisition module is used for acquiring bridge deck vehicle images of aerial photography at high altitude;

the vehicle comparison database module is used for constructing a vehicle type and vehicle weight comparison database;

the image processing module is used for preprocessing the received bridge deck vehicle image and then filtering the vehicle shadow to obtain a clean bridge deck vehicle image;

the vehicle identification module is used for constructing a vehicle identification model, inputting the clean bridge deck vehicle image into the vehicle identification model to identify the bridge deck vehicle, and outputting the identified vehicle parameters; the vehicle parameters include: vehicle type, vehicle direction, vehicle number and vehicle distance;

the vehicle data analysis module is used for obtaining bridge deck vehicle load distribution through a statistical method according to the constructed vehicle comparison database and the identified vehicle parameters.

The technical scheme provided by the embodiment of the invention has the following beneficial effects:

according to the bridge deck vehicle load identification method and system based on the aerial image of the high altitude, firstly, shadow filtration is carried out on the acquired aerial image of the bridge deck vehicle, and vehicle identification errors caused by vehicle shadows are eliminated on the premise that the height of the vehicle is ignored, so that the calculated amount is reduced, the identification efficiency and the identification accuracy are improved, and the bridge deck vehicle load identification method and system based on the aerial image of the high altitude are suitable for instantaneous/long-term vehicle load monitoring of various bridges. Compared with a fixed camera, the vehicle load identification method based on the aerial image provided by the invention has higher flexibility, and the distribution condition of all vehicles on the whole bridge deck at a certain moment can be observed.

Of course, it is not necessary for any one product or method of practicing the invention to achieve all of the advantages set forth above at the same time.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a bridge deck vehicle load identification method based on aerial images in an embodiment of the invention;

fig. 2 is a schematic structural diagram of a bridge deck vehicle load recognition system based on aerial images according to an embodiment of the present invention;

FIG. 3 is a statistical diagram of RGB values of a shadow area according to an embodiment of the present invention;

FIG. 4 is a statistical chart of RGB values of a bridge surface background area according to an embodiment of the present invention;

FIG. 5 is a training result of vehicle type recognition in an embodiment of the present invention;

FIG. 6 is a graph showing statistics of the number of vehicles in a first period according to an embodiment of the present invention;

FIG. 7 is a chart showing statistics of the number of vehicles in the second period according to the embodiment of the present invention;

FIG. 8 is a graph showing statistics of the number of vehicles in a third period according to an embodiment of the present invention;

FIG. 9 is a graph showing the comparison of vehicle spacing between different vehicle models in an embodiment of the present invention;

Detailed Description

After finding the above problems, the present inventors have conducted intensive studies on a statistical method for load distribution of a bridge surface vehicle in the prior art. The research finds that the machine vision statistical method has a plurality of advantages far exceeding the manual statistical method, and with the rapid development and application of the machine vision technology, a plurality of traffic recognition results based on the image processing technology exist, but the image based on bridge deck shooting can not meet the requirements of bridge deck vehicle load analysis.

Unmanned aerial vehicle has advantages such as small and mobility are strong because of it, can overcome traditional image acquisition system's defect, and more is applied to intelligent transportation system. The main task of traffic monitoring is the identification of the number of vehicles, the height information of the vehicles can be ignored, the two-dimensional characteristic information of the vehicles obtained through aerial images can meet the task requirements of traffic monitoring, the operand is greatly reduced, and the working efficiency of a vehicle identification model is improved. However, in the aerial image collected at high altitude under natural illumination, a part of shadow images of the vehicle are adhered to the vehicle image, and the vehicle shadow may be identified as a vehicle, so that the accuracy of model identification is reduced, and the error of traffic statistics becomes large.

It should be noted that the above prior art solutions have all the drawbacks that the inventors have obtained after practice and careful study, and thus the discovery process of the above problems and the solutions presented below by the embodiments of the present invention for the above problems should be all contributions to the present invention by the inventors during the present invention.

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. It should be noted that, in the case of no conflict, the embodiments of the present invention and features in the embodiments may also be combined with each other.

It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures. In the description of the present invention, the terms "first," "second," "third," "fourth," and the like are used merely to distinguish between descriptions and are not to be construed as indicating or implying relative importance.

After the above deep analysis, the embodiment of the invention provides a bridge deck vehicle load identification method and system based on high-altitude aerial images, which omits the height of the vehicle based on the acquired aerial top view images, reduces the calculated amount on the premise of meeting the identification requirement, overcomes the problem of limited visual field height in the traditional identification means, and improves the identification efficiency; in the recognition process, the error of vehicle shadow recognition is eliminated by utilizing the difference between the vehicle shadow and the vehicle and bridge floor background pigment values, so that the rapid recognition of bridge floor vehicle load is realized, the detection recognition accuracy is effectively improved, and the bridge floor vehicle load detection method is suitable for instantaneous/long-term vehicle load monitoring of various bridges.

Referring to fig. 1, the bridge deck vehicle load identification method based on aerial images provided by the embodiment of the invention comprises the following steps:

and S1, constructing a vehicle type and vehicle weight comparison database.

In the step, a vehicle type and vehicle weight comparison database is constructed, and the vehicle types and the vehicle weights are paired one by one. The vehicle types are classified by aspect ratio of the vehicles. Preferably, the vehicle type and vehicle weight comparison database contains seven types of vehicle types, the vehicle types are shown in table 1, and the vehicle axle weight distribution is shown in table 2.

TABLE 1

TABLE 2

Vehicle model code	Number of axes	Total weight/KN	Axle weight/kN	Wheelbase/mm
					V1	2	19	9+10	2749
V2	2	60	21+39	4320
					V3	2	130	60+70	4000
V4	2	140	65+75	4300
					V5	2	160	55+105	4000
V6	3	250	50+100+100	4100+1360
					V7	4	310	45+95+85+85	1860+3560+1350
V8	5	420	40+125+85+85+85	3600+5010+1310+1310
					V9	6	490	30+95+95+90+90+90	3270+1350+6230+1310+1310

And S2, acquiring a bridge deck vehicle image of aerial photography at high altitude.

In this step, specifically, the method includes:

step S21, unmanned aerial vehicle is laid in the bridge floor upper air, and unmanned aerial vehicle carries camera device through the cloud platform, camera device' S field of vision scope is whole bridge floor.

In this step, cloud platform and the camera device that unmanned aerial vehicle carried can be noded to whole bridge floor. Through the angle of the point position and the camera device of the unmanned aerial vehicle of adjustment simultaneously, no matter which point position is in the bridge floor sky, the shooting of the whole bridge floor can be realized.

In the step, the image pickup device can ignore details of the vehicle, but needs to carry out complete shooting on the contour line of the vehicle; meanwhile, real-time vehicle shadows need to be collected at the same time. The adjustment flight parameters are expressed by the ratio of the length of the vehicle to the flight height, and clear images containing the full-bridge vehicle are obtained by adjusting the flight parameters. The nodding height is 50-200 m. Through unmanned aerial vehicle's high altitude shooting, neglect the influence of vehicle height, only contain the length and the width information of vehicle, can reduce the calculated amount under the prerequisite that satisfies the detection requirement, improve detection efficiency.

And S3, preprocessing the received bridge deck vehicle image, and then filtering the vehicle shadow to obtain a clean bridge deck vehicle image.

In this step, the vehicle shadow filtering includes two steps of shadow detection and elimination, and the vehicle shadow detection is realized by the difference of bridge deck, vehicle and vehicle shadow RGB values. Through vehicle shadow filtering, the influence of vehicle shadow on the recognition result is reduced, and the vehicle recognition accuracy is improved. The method specifically comprises the following steps:

step S31, obtaining three-color channel red (R), green (G) and blue (B) brightness value matrixes of the image according to the bridge deck vehicle image, wherein the three-color channel brightness value of the pixel point in the nth row number and the nth column is expressed as R _(m,n) ,G _(m,n) ,B _(m,n) ；

In this step, the three-color channel brightness values of the vehicles of each color are all provided with threshold values, and the threshold values are listed in table 3.

TABLE 3 Table 3

Step S33, judging whether each pixel point in the image is full at the same timeFoot support

If the two types of the images are satisfied at the same time, judging that the images are shadow.

Shadow filtering is completed.

S4, constructing a vehicle identification model, inputting the clean bridge deck vehicle image into the vehicle identification model to identify the bridge deck vehicle, and outputting the identified vehicle parameters; the vehicle parameters include: vehicle type, vehicle direction, vehicle number and vehicle distance.

In the step, the vehicle identification model is constructed by adopting a YOLO-V3 network structure. Preferably, the YOLO-V3 network structure does not have a pooling layer, and the YOLO-V3 network structure outputs 3 feature graphs with different scales, namely y, in terms of output tensor ₁ 、y ₂ And y ₃ . In this embodiment, multiple scales are used to detect different targets, and the finer the grid unit, the finer the object that can be detected. Preferably, in this embodiment, the division of the original image is implemented by using 3 different grids, 16×16 is aimed at a large object, 26×26 is aimed at a medium object, 52×52 is aimed at a small object, and the division is the finest of the three grids.

The YOLO model loss function is determined by the characteristics of each, and end-to-end training is achieved through a loss function.

The vehicle type and the vehicle direction parameters are used for identifying the advancing direction of the traffic flow of the bridge deck vehicle image, the vehicle identification model is used for analyzing the length-width ratio of the vehicle, so that the purpose of identifying the advancing direction of the traffic flow is achieved, the vehicle type and the vehicle direction parameters are the first step of model identification, and the problem that the identification effect of the image cannot meet the requirement due to different directions can be avoided. The different lanes and the distinction of a single lane are determined by vehicles with different distances, the width of the lane is determined by two vehicles with the farthest distances, and the different lanes are correspondingly distinguished according to the width of the lane, so that the purpose of distinguishing each vehicle into each lane is achieved. The vehicle model parameters include the length-to-width ratio (L/B) of the vehicle. The types of vehicles in the images are distinguished from different vehicle types after training according to the length-width ratio of each vehicle in the images.

Specifically, the method for identifying the bridge deck vehicle comprises the following steps:

step S412, determining the width of the lanes by two vehicles with the farthest distance for each traffic traveling direction, correspondingly dividing different lanes according to the lane widths, dividing each vehicle to respective lanes, identifying the inter-vehicle distance in each lane, and the identified parameters can also include the vehicle speed;

The vehicle number parameter YOLO adopts a regression-method-based deep learning detection algorithm to identify the vehicle number in the image. The vehicle number identification is to call a test-detector function in a detector, modify batch processing, count the number of detection targets returned to each picture, increase a return value, and finally recompile a dark net, wherein the detected images can display the detected object classification and number. The idea of the YOLO algorithm is to acquire the range of the target object in the picture by using an excellent classifier in the network structure, and then to accurately achieve a certain position by up-sampling and continuous iteration of a loss function. For the vehicle number detection, the counting is relatively simple, and the counting is completed according to the position relation between each judgment target and the detection line.

The vehicle distance parameter is related to the size of an object (U), the distance (W) between the object and the camera and the distance (X) between the object and the camera according to the principle of small hole imaging, and the relationship is as follows:

In this embodiment, an image set with fixed interval time is adopted, a certain vehicle is fixed in the image set, the image set is shot with the same time interval and the same shooting height, the shot image is identified, the center of the vehicle is marked as a characteristic point through the positions of the two image vehicles, and the time interval is added through the position difference of the characteristic point, so that the speed of the vehicle when the vehicle is in use is obtained.

And S5, obtaining the bridge deck vehicle load distribution through a statistical method according to the constructed vehicle comparison database and the identified vehicle parameters.

In the step, the identified vehicle parameters are matched with a calibrated vehicle type and vehicle weight comparison database, the identified vehicles are converted into corresponding vehicle loads, and the bridge deck vehicle load distribution situation is obtained through a statistical method by combining vehicle characteristic information such as the number of vehicles, the vehicle distance and the like.

Based on the same thought, the embodiment of the invention also provides a bridge deck vehicle load identification system based on the aerial image, as shown in fig. 2, the system comprises: the system comprises a vehicle comparison database module, an image acquisition module, an image processing module, a vehicle identification module and a vehicle data analysis module.

The image acquisition module is used for acquiring bridge deck vehicle images of aerial photography. Preferably, the image acquisition module comprises: unmanned aerial vehicle, cloud platform and camera device;

the image processing module is used for preprocessing the received bridge deck vehicle image, then performing vehicle shadow filtering to obtain a clean bridge deck vehicle image and sending the clean bridge deck vehicle image to the vehicle identification module;

the vehicle recognition module is used for constructing a vehicle recognition model, inputting the clean bridge deck vehicle image into the vehicle recognition model to recognize the bridge deck vehicle, and outputting recognized vehicle parameters; the vehicle parameters include: vehicle type, vehicle direction, vehicle number and vehicle distance;

The modules in this embodiment are implemented by a processor, and the memory is appropriately increased when storage is required. The processor may be, but is not limited to, a microprocessor MPU, a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), other programmable logic devices, discrete gates, transistor logic devices, discrete hardware components, or the like. The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

It should be noted that, the bridge deck vehicle load recognition system and method based on aerial images in this embodiment correspond to each other, and description and limitation of the system are also applicable to the method, and are not repeated here.

For the purpose of facilitating understanding of the embodiments of the present invention, specific examples are set forth to illustrate, but are not to be construed as limiting the scope of the invention.

And acquiring relevant image information by using an unmanned aerial vehicle aiming at the Xiangtan four-bridge and analyzing the load distribution of the bridge deck vehicle by taking the Xiangtan four-bridge as an engineering background. The method comprises the following steps:

step S1, a vehicle type and vehicle weight comparison database is constructed, and the data in the table 1 and the table 2 are adopted in the embodiment.

And S2, arranging an unmanned aerial vehicle above the Xiangtan four-bridge, wherein the unmanned aerial vehicle carries a camera through a cradle head. The camera and pan/tilt parameters are shown in tables 4 and 5, respectively.

TABLE 4 Table 4

TABLE 5

Adjusting flight parameters, lifting the unmanned plane to 200m flight altitude, and performing bridge deck aerial photographing for acquiring bridge deck vehicle images in different time periods (7:30-7:50, 8:00-8:20, 9:30-9:50, 10:00-10:20, 11:30-11:50, 12:00-12:20, 14:30-14:50, 16:30-16:50, 17:00-17:20, 18:30-18:50 and 19:00-19:20).

In this example, pictures of 50m, 100m, 150m, 200m, and 300m flying heights were tested, and the recognition rates at the respective flying heights are shown in table 6.

TABLE 6

Fly height/m	Recognition rate
			50	99％
100	97％
		150	90％
200	89％
		300	5％

Analysis of the recognition results of images photographed at different flying heights of 50m, 100m, 150m, 200m, and 300m shows that the recognition of the photographed images is substantially ineffective when the flying height reaches 300 m. The image recognition effect shot below 300m can basically meet the research requirement, the minimum accuracy rate also reaches 89%, and the model has good robustness. Considering the whole bridge research and the reasons of image breadth and model recognition accuracy, the image acquisition is carried out at the height of 200m by comparing images shot at different flying heights.

Step S3, after preprocessing the received bridge deck vehicle image, performing vehicle shadow filtering, selecting 35 shadow areas in the shadow image as shadow RGB value samples, randomly selecting 3 points in each area as samples, taking the average value of the 3 points in each area as the representative value of the area, and carrying out statistics, wherein the statistical result is shown in figure 3. Similarly, the bridge deck RGB values were statistically analyzed, and the results are shown in fig. 4.

As can be seen from FIG. 3, the range of R values for the shadow area of the bridge deck vehicle is 80-100, the range of G values is 105-125, and the range of B values is 125-145. The results of fig. 4 show that the R value, G value, and B value of the bridge floor background area basically fluctuate in the range of 160-185 under different brightness conditions, and the average value of the R value, G value, and B value of the bridge floor background area is calculated to obtain an average value of 169, an average value of 172, and an average value of 168.

The common vehicles with different colors are subjected to point selection and identification, and the R value, G value and B value selection intervals of the vehicles with different colors are shown in table 7.

From the statistics in fig. 3-4 and table 7, it can be seen that the shade of the vehicle is smaller in the position close to the vehicle than in the position of the vehicle, the image is shown as darker, whereas the lighter the color, i.e. the gradient of the shade. According to the shadow gradient principle, the detection of the shadow is to detect the R value, the G value and the B value in the region, and only the R value, the G value and the B value of a certain region in the image accord with the value range of the R value, the G value and the B value of the shadow part can the shadow be identified. The corresponding MATLAB program is programmed to implement the shadow identification and replacement process, with the shadow identification and replacement pseudocode shown in table 7.

TABLE 7

S4, constructing a vehicle recognition model, inputting the clean bridge deck vehicle image into the vehicle recognition model to recognize the bridge deck vehicle, and outputting recognized vehicle parameters; the vehicle parameters include: vehicle type, vehicle direction, vehicle number and vehicle distance.

The vehicle type is identified as follows:

the acquired aerial image is input as an input end of the recognition model, the YOLO V3 network is utilized for recognizing the type of the bridge deck vehicle, and the training effect of the type recognition is shown in figure 5.

The number of vehicles is identified as follows:

because the single cruising time of the unmanned aerial vehicle is 20 minutes, the flight time is limited, images acquired in the periods of 9:40-9:50, 11:50-12:00 and 18:50-19:00 are selected as the basis, the data of the vehicle type and the vehicle number are identified through a YOLO model, the data detected by the model are compared with the data manually acquired in the same period, the feasibility of identifying the vehicle flow based on aerial images is verified, and experimental results are shown in figures 6-8.

As can be seen from fig. 6 to 8, for each period of time, the vehicle count value based on the unmanned aerial vehicle is smaller than the value based on the manual count, the minimum error is 2.96%, the maximum error is 5.11%, but the overall vehicle count error is 3.86%, within the error allowable range, compared with the manual count based method.

The inter-vehicle distances are identified as follows:

vehicle data in a bridge length of 200m are analyzed, images with dense traffic flow in collected pictures are used as research data sets, vehicle distance information is obtained by means of identification of a YOLO model, collected data are data of a collecting period of 5 days, 3000 images are taken as research samples, 600 images are taken for each period to be detected, the periods are 7:30-8:30, 9:30-10:30, 11:30-12:30, 16:30-17:30 and 18:30-19:30, and the selection of the periods basically comprises early peaks, late peaks and traffic flow in a normal running state during bridge operation, so that analysis of the samples is reasonable. The average processing for the vehicle spacing in the image is performed according to formula (1).

Average spacing:

wherein L is a collection bridge long region; Σn is the total number of vehicles;

is the average spacing.

For Xiangtan four-bridge, the method is used for obtaining the data of the number of vehicles, the types of vehicles and the distance between the vehicles among different lanes in each period, and counting the total data. Statistical analysis is carried out on the information of the inter-vehicle distances in the acquired 3000 images, the average distance is obtained through the inter-vehicle distances of the single images, the average inter-vehicle distances in the single images are obtained, the data processing is carried out on the 3000 images in the sample by adopting the same method, the average value processing is carried out on 3000 groups of data of each lane, the 95% confidence level is ensured, the average value of the inter-vehicle distances is obtained, and the result is shown in figure 9.

The vehicle speed is identified as follows:

taking the morning 7 in the time period of vehicle speed collection in the peak period of the Xiangtan four-bridge: 30 to 9:30 time period, night 5:30 to 7: and 30, processing the acquired images, establishing a data set for speed identification, listing the data set in a table 8, and identifying images of the flight height of 200m and a fixed time period through an image identification model so as to obtain the speed of the same vehicle.

TABLE 8

Vehicle model	One-type vehicle	Two-type vehicle	Three-type vehicle	Four-wheel vehicle	Five-type vehicle	Six-type vehicle	Seven-type vehicle
								Number of vehicles/vehicle	2368	97	261	71	24	4	94
Average speed km/h	79.5	66.5	70.2	63.4	70.6	68.3	69.4

Step S5, vehicle load distribution statistics: the traffic jam running state has the least adverse effect on the bridge structure, so that the subsequent analysis is the traffic under the dense running state. For the automobile load research of the bridge, a regression analysis method is adopted, the types, the numbers and the distances between vehicles in the images at the moment are identified through a model, the vehicle weight information of various vehicles in a vehicle type and vehicle weight comparison database is analyzed and researched in the images acquired at different time intervals, a sample with sample capacity of 1500 images is established, and the load calculation is researched by adopting a method for calculating the load concentration, wherein the main formula is as follows:

the duty ratio of each vehicle type can be calculated according to the following formula:

the load concentration calculation formula is as follows:

in the formulas (2) and (3): t is t _i The vehicle weight average value of each vehicle type; l is the image acquisition length; z is a sampleNumber of pieces; m is the total number of vehicles.

Fitting is carried out on the weight and the distribution of each vehicle model, and the fitting is in a form of normal distribution, so that the distribution function of the weight and the distribution of each vehicle model is as follows:

carrying out lane division statistics on 1500 samples to obtain the vehicle type and vehicle number data of each lane, and obtaining the vehicle type ratio of each bridge by taking the vehicle type and the vehicle number data as the basis of calculation. The method comprises the steps of calculating the load concentration of each bridge, establishing samples with relatively small sample capacity, and mainly selecting a dense state during bridge operation for analysis, wherein the selection of small samples is required to follow the principle of vehicle occupation, and analyzing working conditions with different acquisition lengths, so that the load concentration during bridge operation is obtained.

When the four-axle concentration is calculated, as the four-axle traffic flow forms a plurality of vehicle types including one type of vehicle to seven types of vehicles, after the vehicle weight data are referenced, the concentration of the one type of vehicle to the four types of vehicles is calculated by adopting uniform load, and the five type of vehicle to the seven type of equal weight vehicles are calculated as concentrated load, and the intensive state traffic flow under the fixed collection length is selected for concentration calculation by adopting the conditions of small collection length and intensive traffic flow. The load concentration calculation values for each lane of the four bridges are shown in table 9.

TABLE 9

The heavy vehicle load research is carried out based on four-bridge heavy vehicles, 1500 samples established for the previous section to four-bridge are taken as research objects, load calculation is carried out on heavy vehicles with different acquisition lengths, and dense vehicle flows with different acquisition lengths are counted and concentrated load conversion is carried out. The number of heavy vehicles and the types of the vehicles in each lane in 1500 samples were counted as shown in table 10.

Table 10

The research on the heavy vehicle load is a vehicle flow load effect in a dense running state, and the four-bridge heavy vehicle load is used as a concentrated force for loading research in view of the fact that the four-bridge belongs to a large-span bridge. According to the method, the load of the heavy vehicle is researched, uniform load fitting of the single vehicle is needed according to the wheelbase and axle weight distribution of the heavy vehicle, so that statistics is needed for data such as the wheelbase and axle weight of each vehicle type, and the wheelbase and axle weight distribution of the heavy vehicle type are shown in a table 11.

TABLE 11

Vehicle model	Total weight/KN	Wheelbase/mm	Axle weight/KN
				Five-type vehicle	321	1860+3560+1350	49+100+86+86
Six-type vehicle	439	3600+5010+1310+1310	43+132+88+88+88
				Seven-type vehicle	495	3270+1350+6230+1310+1310	30+99+99+89+89+89

And (3) counting the lane splitting of the heavy vehicles in the counted sample, analyzing the working condition of the heavy vehicles in the dense state, and after the data are arranged, obtaining the image of the most dense heavy vehicles in the single lane, wherein the number of the vehicles is shown in a table 12.

Table 12

By combining the wheel base and axle weight data of each vehicle type, uniform load values of the single vehicle of each vehicle type are obtained, as shown in table 13:

TABLE 13

According to the bridge deck vehicle load identification method and system based on the aerial image in the high altitude, shadow filtering is carried out on the collected aerial bridge deck vehicle image, errors of vehicle shadow identification are eliminated on the premise that the height of the vehicle is ignored, calculated amount is reduced, identification efficiency and identification accuracy are improved, and the bridge deck vehicle load identification method and system based on the aerial image in the high altitude are suitable for instantaneous/long-term vehicle load monitoring of various bridges.

The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any changes or substitutions easily contemplated by those skilled in the art within the scope of the present invention should be included in the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.

Claims

1. A bridge deck vehicle load identification method based on aerial images is characterized by comprising the following steps:

constructing a vehicle type and vehicle weight comparison database;

acquiring a bridge deck vehicle image of aerial photography at high altitude;

constructing a vehicle identification model, inputting the clean bridge deck vehicle image into the vehicle identification model to identify the bridge deck vehicle, and outputting the identified vehicle parameters; the vehicle parameters include: the vehicle type, the vehicle direction, the vehicle number and the vehicle distance are constructed by adopting a YOLO-V3 network structure;

obtaining bridge deck vehicle load distribution through a statistical method according to the constructed vehicle comparison database and the identified vehicle parameters;

the vehicle shadow filtering includes:

Completing shadow filtration;

the method for identifying the bridge deck vehicle comprises the following steps:

2. The method for identifying the bridge deck vehicle load based on the aerial image of claim 1, wherein the step of acquiring the aerial image of the bridge deck vehicle comprises the steps of

3. The method for recognizing a bridge floor vehicle load based on aerial images according to claim 2, wherein the height of the nodding is 50-200 m.

4. The method for identifying the bridge deck vehicle load based on the aerial image according to claim 1, wherein the YOLO-V3 network structure does not have a pooling layer, and 3 feature maps with different scales are output in terms of output tensors.

5. The method for recognizing bridge floor vehicle load based on aerial images according to claim 4, wherein the 3 feature maps of different scales are realized by dividing an original image by 3 different grids, including 16 x 16 grids for large objects, 26 x 26 grids for medium objects, and 52 x 52 grids for small objects.

6. The bridge deck vehicle load identification method based on aerial images of claim 1, wherein the vehicle distance parameter is related to the size of the object U, the distance W of the object from the camera to the size Φ of the small hole and the distance X from the camera according to the principle of small hole imaging, and the relationship is as follows:

7. Bridge deck vehicle load identification system based on aerial images, characterized in that it comprises: the system comprises a vehicle comparison database module, an image acquisition module, an image processing module, a vehicle identification module and a vehicle data analysis module; wherein, the liquid crystal display device comprises a liquid crystal display device,

the vehicle identification module is used for constructing a vehicle identification model, inputting the clean bridge deck vehicle image into the vehicle identification model to identify the bridge deck vehicle, and outputting the identified vehicle parameters; the vehicle parameters include: the vehicle type, the vehicle direction, the vehicle number and the vehicle distance are constructed by adopting a YOLO-V3 network structure;

the vehicle data analysis module is used for obtaining bridge deck vehicle load distribution through a statistical method according to the constructed vehicle comparison database and the identified vehicle parameters;

the image processing module is specifically configured to:

obtaining a three-color channel red R, green G and blue B brightness value matrix of an image according to a bridge deck vehicle image, wherein the three-color channel brightness values of the m-th row number and the n-th column pixel point are expressed as R _(m,n) ,G _(m,n) ,B _(m,n) ；

Counting the brightness values of three-color channels of bridge deck, vehicles with various colors and shadow parts of the vehicles in the aerial photo to obtain maximum and minimum values as representative values, wherein the shadow parts are expressed as

Judging whether each pixel point in the image meets the requirement simultaneously

pixel three-color channel luminance values (R _shadow ,G _shadow ,B _shadow ) Replaced by bridge deck three-color channel brightness value or brightness average value

Completing shadow filtration;

the vehicle identification module is specifically configured to:

extracting first characteristics based on the appearance and the outline of the vehicle, primarily classifying the vehicles in the image based on the first characteristics, recognizing and reading the position of the vehicle in the short-time difference by adopting vehicle types with obvious characteristics and less quantity, and judging the running direction of the vehicle flow according to the position of the vehicle in the photo;

determining the width of a lane by two vehicles with the farthest distance according to the advancing direction of each traffic flow, correspondingly dividing different lanes according to the lane width, dividing each vehicle into respective lanes, and identifying the inter-vehicle distance in each lane;

and extracting a second characteristic based on the aspect ratio of the vehicle, identifying the vehicle type of each vehicle in the image, and calculating the number of vehicles of each vehicle type.