CN115035251B - Bridge deck vehicle real-time tracking method based on field enhanced synthetic data set - Google Patents
Bridge deck vehicle real-time tracking method based on field enhanced synthetic data set Download PDFInfo
- Publication number
- CN115035251B CN115035251B CN202210680866.6A CN202210680866A CN115035251B CN 115035251 B CN115035251 B CN 115035251B CN 202210680866 A CN202210680866 A CN 202210680866A CN 115035251 B CN115035251 B CN 115035251B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- camera
- real
- data set
- image data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 26
- 238000001514 detection method Methods 0.000 claims abstract description 23
- 238000005070 sampling Methods 0.000 claims abstract description 20
- 239000002131 composite material Substances 0.000 claims abstract description 6
- 238000012549 training Methods 0.000 claims description 34
- 238000004088 simulation Methods 0.000 claims description 20
- 230000006399 behavior Effects 0.000 claims description 17
- 238000002372 labelling Methods 0.000 claims description 11
- 238000010586 diagram Methods 0.000 claims description 9
- 238000009826 distribution Methods 0.000 claims description 9
- 230000004927 fusion Effects 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 8
- 238000005516 engineering process Methods 0.000 claims description 7
- 238000010276 construction Methods 0.000 claims description 5
- 238000009434 installation Methods 0.000 claims description 5
- 238000013528 artificial neural network Methods 0.000 claims description 4
- 238000004220 aggregation Methods 0.000 claims description 3
- 230000002776 aggregation Effects 0.000 claims description 3
- 230000001427 coherent effect Effects 0.000 claims description 3
- 238000005520 cutting process Methods 0.000 claims description 3
- 238000013079 data visualisation Methods 0.000 claims description 3
- 230000003247 decreasing effect Effects 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000009877 rendering Methods 0.000 claims description 3
- 239000007787 solid Substances 0.000 claims description 3
- 230000002194 synthesizing effect Effects 0.000 claims description 3
- 238000012805 post-processing Methods 0.000 abstract description 2
- 238000013480 data collection Methods 0.000 abstract 1
- 238000012360 testing method Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000003153 chemical reaction reagent Substances 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000001308 synthesis method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30256—Lane; Road marking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computational Linguistics (AREA)
- Computer Graphics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Geometry (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a bridge deck vehicle real-time tracking method based on a field enhanced synthetic data set, which comprises the steps of establishing a full-size three-dimensional model according to real data of a target bridge, and generating different weather scenes through software rendering; according to the real traffic rules, an independently-running virtual vehicle model is generated, simulated image data under different road conditions are collected through virtual cameras arranged on the bridge model, the simulated image data are used as a part of a composite data set, the composite data set is constructed through a random sampling data enhancement method, a target detection network is trained together with the real image data, and then the target detection network can be used for rapid identification of targets. And simulating different mounting positions of the cameras in the three-dimensional model by using a camera arrangement algorithm, and searching an optimal solution of camera arrangement. The invention provides a safe and efficient virtual tool for collecting image data under specific vehicle condition road conditions, and simultaneously, the unique camera arrangement algorithm ensures that the invention has the advantage of high efficiency in data collection and post-processing.
Description
Technical Field
The invention relates to the technical field of target detection. More particularly, the invention relates to a real-time tracking method for bridge deck vehicles based on a field enhanced synthetic data set.
Background
The target detection algorithm is an algorithm for determining the position of a target by searching for an image characteristic rule from a large number of training samples. In the implementation process of the target detection algorithm based on deep learning, a large amount of sample collection and labeling work are needed to be performed manually.
The size and type of the training sample in the target detection algorithm largely determine the effect of the target detection algorithm. However, as the data dimension explodes, visual judgment must be extended to more complex problems-learning and sample complexity, computational efficiency, class imbalance, etc. At this point, the balance between experimental flexibility and dataset needs to be taken into account to find a truly large dataset to train the algorithm. But this is still a data set with a fixed number of samples, a fixed potential regularity, and a fixed degree of class separation between positive and negative samples. The data set therefore also needs to take into account deeper issues such as how the selected test and training data proportions affect the performance and robustness of the algorithm, how to trade off bias and variance in the face of different levels of class imbalance, how to test and find defects in the deep learning algorithm under various noise signal disturbances, how to test and find defects in the algorithm in the training and test data (i.e., noise in the tag and feature set), and so on. It is difficult to solve the above problem by only one real dataset. Thus, sufficient random synthetic data must be used to achieve uncertainty in the real dataset, while the data must be sufficiently controllable to help scientifically investigate certain advantages and disadvantages. The synthetic data set has shown obvious advantages in various fields at present, and a great deal of labor and fund cost is saved for target detection tasks in the fields of medical imaging, constructional engineering, mining industry, mines and the like. However, there is no effective data set synthesis method in the field of bridge vehicle tracking, so how to achieve a method for quickly and inexpensively obtaining a data set meeting the training requirements of a vehicle target tracking model is an important problem.
Disclosure of Invention
The invention aims to provide a bridge deck vehicle real-time tracking method based on a field enhanced synthetic data set, which solves the problem that real traffic image data is difficult to collect, and provides a safe and efficient virtual tool for collecting image data under specific vehicle condition road conditions. Meanwhile, the unique camera arrangement algorithm ensures that the invention has the advantage of high efficiency in data acquisition and post-processing.
To achieve these objects and other advantages and in accordance with the purpose of the invention, a method for real-time tracking of a deck vehicle based on a domain-enhanced synthetic data set is provided, comprising the steps of:
step 1, in 3D modeling software, full-size digital model modeling is carried out according to the actual size of a bridge, different types of vehicle models are arranged on a bridge deck, a vehicle behavior model is built, a three-dimensional vehicle condition road condition scene is generated through rendering, the motion state of a vehicle on the bridge deck is obtained through an arranged virtual camera, and analog image data are generated;
step 2, performing generation type countermeasure learning by using the simulated image data and the real image data, generating synthesized simulated image data, and labeling the synthesized simulated image data and the real image data to complete the construction of a synthesized data set;
step 3, training a target detection network by adopting a synthetic data set as a training set and a random sampling data enhancement method based on a YOLO V4 algorithm, synthesizing a training diagram containing various target characteristics, and identifying target vehicle related information;
step 4, in the full-size digital model in the step 1, fitting a lane curve according to a training diagram based on a curve-fitting lane line detection principle, and splicing a plurality of coherent camera views into a complete road lane through view stitching;
and 5, the relevant information of the target vehicle tracked in each camera is mapped through a coordinate system of the stitching vision field, so that the recognition and tracking of the vehicle across the vision field are realized.
Preferably, the step 1 specifically includes:
step 1.1: constructing an actual bridge full-size digital model in 3D modeling software Blender by adopting a solid modeling method, manufacturing three-dimensional vehicle model libraries of various types of vehicles, and inputting the three-dimensional vehicle model libraries into the Blender model libraries;
step 1.2: establishing a multi-type vehicle behavior model conforming to actual traffic rules;
step 1.3: different types of vehicle models are imported according to actual engineering requirements, and specific vehicle behavior models are given to the vehicle models, so that the vehicle models run in a full-size digital model according to set paths and behavior modes, and real traffic conditions are simulated;
step 1.4: adjusting parameters of a simulation system, and simulating partial shielding conditions of cameras under different weather conditions and dynamic traffic flow situations to obtain vehicle running states under different situations;
step 1.5: setting a virtual camera, collecting the motion state of a vehicle in a bridge deck lane in the running process of a model, deriving the motion state into a video format, and converting the motion state into a picture format frame by frame to generate analog image data.
Preferably, the step 2 specifically includes:
step 2.1: firstly, identifying and classifying vehicles to be tracked, importing real pictures of different types of vehicles into a model as a real image data set to serve as a primary data source, and combining simulation image data obtained by vehicle traffic simulation in the model to serve as a secondary data source;
step 2.2: generating a random noise set { z } and a sample set { x } by performing a generative countermeasure learning based on the analog image data, i.e., the secondary data source, and the real image data, i.e., the primary data source;
step 2.3: randomly sampling m samples from a gaussian noise distribution, i.e., a random noise set { z }; randomly sampling m samples from a real data distribution, namely a random sample set { x };
step 2.4: updating the discriminant S by means of a random gradient descent SGD method w Parameters of (2);
step 2.5: randomly sampling another m samples from Gaussian noise distribution, namely a random noise set { z };
step 2.6: updating the arbiter G by a random gradient descent SGD method θ Parameters of (2);
step 2.7: obtaining generated simulated image data, and using the generated simulated image data as a data set for training a neural network;
step 2.8: and labeling the obtained simulated image data set and the real image data set to complete the construction of the synthetic data set.
Preferably, the step 3 specifically includes:
step 3.1: dividing the synthetic data set into n batches for batch processing, taking out one batch, randomly selecting 4 pictures, cutting and splicing the random positions into new pictures, and repeating the batch_size for times to obtain a new fusion data set;
step 3.2: after the image is subjected to the random sampling data enhancement processing in the step 3.1, the image is subjected to the pretreatment of the cmBN and SAT self-countermeasure training;
step 3.3: the method comprises the steps of optimizing a cross-stage gradient information repetition problem by using a cross-stage local network CSPNet, adopting a ResNet residual error structure optimization model integral frame, dividing a feature map of a base layer into two parts, and merging the two parts through a cross-stage hierarchical structure;
step 3.4: sending the feature images with different sizes obtained in the step 3.3 into an SPP network, and carrying out pooling operation with fixed sizes on the feature images to obtain feature images with fixed dimensions;
step 3.5: performing parameter aggregation on the feature graphs with different sizes in the step 3.4 by using an FPN+PAN structure to obtain a fusion feature graph;
step 3.6: and (3) inputting the fusion feature map obtained in the step (3.5) into a Head output layer to obtain a prediction frame output result, and obtaining relevant information for identifying the vehicle.
Preferably, the real image data in step 1 is a vehicle picture extracted from other bridge monitoring videos acquired from the network, and the vehicle picture is provided with corresponding identification data, such as a model number, a size of a vehicle and pixel coordinates of the vehicle in the picture.
Preferably, the labeling in the step 2 is to use a labeling tool to mark the vehicle position on the obtained image data in the form of a rectangular frame and mark the tag information in the form of a file name.
Preferably, the information about the target vehicle identified in the step 3 includes a vehicle position, a center coordinate, and a target vehicle pixel size.
Preferably, the specific method for obtaining the bridge deck camera layout in the step 4 includes:
a) Selecting a camera product with corresponding parameters according to the existing highway parameters;
b) Obtaining the interval and the number of the longitudinal cameras according to the visible distance of the cameras and the lengths of the road surface and the bridge deck;
c) Obtaining the number of the transverse cameras according to the view angles of the cameras, pixels and road parameters;
d) Obtaining the basic layout of the cameras on the highway according to the mountable camera positions on the highway and the data of the step b) and the step c), and checking the basic layout of the cameras through the virtual cameras arranged in the digital model established in the step 1;
e) The camera basic layout is combined, the actual camera layout is corrected according to the actual mountable positions of the cameras on the road surface and the bridge deck as recursive parameters, and the corrected actual camera layout is checked through the virtual cameras arranged in the digital model established in the step 1;
f) Determining the effective recognition distance of the vehicle in the actual camera layout according to the frequency of the target vehicle in the camera view area after the related information of the target vehicle is recognized in the step 3;
g) Correcting the actual camera layout in the step e) by taking the effective recognition distance as a parameter, namely, increasing or decreasing the camera interval to ensure that the effective recognition distances of the adjacent cameras can connect the covered pavement, so as to obtain the optimal coverage rate;
h) And further correcting the layout of the actual cameras according to the coverage rate to obtain the number and the installation positions of the required cameras.
Preferably, the lane curve fitting in step 4 specifically includes:
step 4.1: selecting seed points, namely searching a center point of a lane line, performing lane line fitting on a set of lane line center points, obtaining a gray level image through binarization based on lane line images in a training image, distinguishing the outline of a road surface and a road mark by using gray level saturation, extracting a lane mark position by using outline features, and finally obtaining the lane center point and classifying by using consistency features of the marks;
step 4.2: after the seed points, namely the center points of the lanes, are classified, under the condition of better road conditions, the near-sight lane lines are clearer, and the seed points with more sufficient number of lane lines are obtained by scanning and classifying from bottom to top;
step 4.3: after classifying the seed points of the lane lines and when the number of the seed points is sufficient, selecting an optimal subset by using an LMedsquare curve fitting technology to remove redundant noise, and then performing linear fitting in the optimal subset by using a least square method to obtain a lane curve.
Preferably, the vehicle cross-view identification in step 5 is performed, and the specific method of tracking is as follows:
a) The target vehicle identified by the camera identifies the lane where the target vehicle is located in the camera view according to the lane curve data fitted in the step 4;
b) According to the camera layout in the step 4, the target vehicle can appear in two adjacent camera vision fields at the same time;
c) And obtaining the lane position of the target vehicle through the fitted lane curve data, and judging whether the camera is the same vehicle or not by combining the real-time data of the target vehicle obtained by the camera and the position of the target vehicle in the camera identification area, so as to realize the tracking of the vehicle across the vision.
Preferably, the specific method for mapping the coordinate system after the camera in the step 5 is spliced by the vision field includes:
step 5.1: acquiring a camera coordinate system, which is a coordinate system of measuring an object by a camera standing on the angle of the camera;
step 5.2: a road image coordinate system is acquired, which is a coordinate system established based on a two-dimensional photograph taken by a camera and used for specifying the position of a vehicle in the photograph, wherein (x, y) is a continuous image coordinate or a space image coordinate, and (u, v) is a discrete image coordinate system or a pixel image coordinate system,
step 5.3: the road image coordinate system is converted into a pixel coordinate system, the coordinates can be directly applied to a camera picture as mark coordinates for identifying vehicles, and can be mapped onto a road simulation system as parameters for data visualization,
the invention at least comprises the following beneficial effects:
1. because the bridge deck vehicle running condition is complex, the specific vehicle condition road condition reproduction frequency is low, and the time cost and the economic cost for acquiring the data set in a large scale in the real world are high, the invention constructs the simulation world by using intelligent accurate modeling software to simulate the real vehicle condition road, carries out image data annotation to construct a composite image data set, and carries out subsequent target training.
2. Because the difference exists between the marked image data obtained in the simulated world and the real world image data, the method for generating the data set by using the generated type contrast learning to perform data enhancement and data fusion can quickly obtain the marked data set of any target at low cost, increase the richness of the data set and provide sufficient marked data for large-scale deep learning.
3. Because the composite data set contains various vehicle condition road conditions, such as extreme weather, complex road conditions and the like, the marked data features under the conditions are not obvious, so that in order to realize the accurate recognition of the data features, the invention adopts an improved YOLO V4 algorithm, namely a method based on random sampling data enhancement, to carry out target detection on input data, combines 4 training images into one training image in a random sampling data enhancement mode to train and synthesize the training image containing various target features, and improves the recognition efficiency of model recognition.
4. The invention provides a method for selecting an optimal subset based on LMedsquare on the basis of a lane line detection principle based on curve fitting, and vehicle identification and track tracking are realized in a full-size model.
5. The invention adopts a splicing technology of a vision coordinate system, feeds the coordinate system back to a road simulation system, realizes the tracking processing of the vehicle across cameras, and designs a set of brand-new camera arrangement algorithm for improving the coverage rate of cameras.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention.
Drawings
FIG. 1 is a schematic illustration of an exemplary simulated image generated by the model of the present invention;
FIG. 2 is a block diagram of the CSPResNe (X) t algorithm of the present invention;
FIG. 3 is a unitary frame diagram of the present invention;
FIG. 4 is a diagram of the overall frame of the YOLO v4 algorithm of the present invention;
FIG. 5 is an experimental verification result of the method of the present invention;
FIG. 6 is a schematic diagram of a random sample data enhancement step according to the present invention;
fig. 7 is a flowchart of a camera layout method according to the present invention.
Detailed Description
The present invention is described in further detail below with reference to the drawings to enable those skilled in the art to practice the invention by referring to the description.
It should be noted that the experimental methods described in the following embodiments, unless otherwise specified, are all conventional methods, and the reagents and materials, unless otherwise specified, are all commercially available; in the description of the present invention, the terms "transverse", "longitudinal", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus are not to be construed as limiting the present invention.
The invention discloses a bridge deck vehicle real-time identification and tracking algorithm based on bridge full-size modeling, composite data set and target detection. Establishing a full-size three-dimensional model in Blender software according to real data of a target bridge, and generating different weather scenes by a software rendering method; according to the real traffic rules, an independently-running virtual vehicle model is automatically generated, simulated image data under different vehicle condition road conditions are collected through virtual cameras arranged on the bridge model, the image data are used as a part of a synthetic data set, the synthetic data set is constructed through a random sampling data enhancement method, a target detection network is trained together with the real image data collected in reality, and the target detection network can be used for rapid identification after training is completed. And (3) simulating different placement positions of cameras in a full-size three-dimensional model by utilizing an independently developed camera placement algorithm, and searching an optimal solution of camera placement.
As shown in fig. 1 to 6, the invention provides a bridge deck vehicle real-time tracking method based on a field enhanced synthetic data set, which comprises the following steps:
step 1, in 3D modeling software Blender, full-size digital model modeling is carried out according to the actual size of a bridge, different types of vehicle models are arranged on a bridge deck, a three-dimensional vehicle condition road condition scene is generated through rendering, and simulated world image data required by training a neural network is acquired through an arranged virtual camera. The invention has the innovation that the full-size model is used for simulating the real bridge, a simulated vehicle behavior system is built in the model, and the simulated data is maximally close to the real world by setting different vehicle behaviors and different environments. The method comprises the following five substeps:
step 1.1: constructing an actual bridge full-size digital model in Blender software by adopting a solid modeling method, manufacturing a three-dimensional model library of various types of vehicles, and sending the three-dimensional model library into the Blender model library, as shown in FIG. 1;
step 1.2: establishing a vehicle behavior model conforming to an actual traffic rule, wherein the vehicle behavior model comprises a single vehicle behavior model and a multi-vehicle behavior model; traffic rules include constraints on vehicle speed, lanes, vehicle types, etc., and decisions on illegal behavior of the vehicle, such as overspeed, lane change with solid lines, etc.;
step 1.3: different types of vehicle models are imported according to actual engineering requirements, and specific vehicle behavior models are given to the vehicle models, so that the vehicle models run in full-size models according to set paths and behavior modes, and real traffic conditions are simulated;
step 1.4: adjusting parameters of a simulation system, and simulating different weather conditions such as sunny days, rainy days, foggy days, snowy days and the like and partial shielding conditions of the cameras under the situation of dynamic traffic flow to obtain vehicle running states under different situations;
step 1.5: setting a virtual camera, collecting the motion state of a vehicle in a bridge deck lane in the running process of a model, deriving the motion state into a video format, converting the video format into a picture format frame by frame, and storing the derived picture into a bitmap format which can be stored into a png format, namely generating the simulated image data.
And 2, generating a simulation image by a data enhancement method based on the generation of the countermeasure learning, and marking the simulation image data and actual real image data (the actual real image is a picture extracted from other bridge monitoring videos). The method comprises the steps of manually marking the position, the environment condition and the type of the vehicle in the obtained image, and completing the construction of a data set. The method specifically comprises the following substeps:
step 2.1: first, the vehicles to be detected are identified and classified. According to the category, importing real pictures of different types of vehicles into a model as a data set, taking a network gallery (namely, each vehicle picture is provided with corresponding identification data such as the model number, the size and pixel coordinates in the picture) as a primary data source, and taking image data recorded and exported by combining videos obtained by vehicle traffic simulation in a simulator in a model system as a secondary data source;
step 2.2: performing a generative countermeasure learning using the simulated image data and the real image data, generating a random noise set { z } and a sample set { x };
step 2.3: randomly sampling m samples from a gaussian noise distribution, i.e., a random noise set { z }; randomly sampling m samples from a real data distribution, namely a random sample set { x };
step 2.4: updating the discriminant S by a random gradient descent (SGD) method w Parameters of (2);
step 2.5: randomly sampling another m samples from Gaussian noise distribution, namely a random noise set { z };
step 2.6: updating the discriminant G by a random gradient descent (SGD) method θ Parameters of (2);
step 2.7: obtaining a generated simulation image (namely a simulation image data set integrated by m samples extracted in the step) and using the generated simulation image as a data set for training a neural network;
step 2.8: and labeling the obtained simulation data set and the real data set. The vehicle position is indicated in the form of a frame line on the obtained image using an labeling tool, and the tag information is indicated in the form of a file name.
And 3, performing target detection on the input data by using a method based on random sampling data enhancement. According to the invention, an improved YOLO V4 algorithm is adopted to carry out target detection tasks, as shown in fig. 4, a training result is shown in fig. 5, the transverse coordinates in the figure are the number of training sets and the number of iterations (batch), the higher the number is, the higher the theoretically obtained accuracy is, and meanwhile, more time is required. The ordinate is loss (loss), and the smaller the value, the smaller the error rate. The broken line in the figure represents the minimum accuracy (mAP) representing the accuracy of the recognition of the object after training. 4 training images are combined into one training image in a random sampling data enhancement mode, and the recognition efficiency of model recognition is improved by synthesizing one training image containing multiple target features.
Step 3.1: the dataset (the above-described simulated dataset and the real image dataset) was divided into n batches for batch processing. The random sample data enhancement step is shown in fig. 6. Taking out a batch, randomly selecting 4 pictures, cutting and splicing random positions into a new picture, and repeating the batch_size for times to obtain a new fusion data set;
step 3.2: after the image is subjected to random sampling data enhancement processing, the image is subjected to pretreatment by cmBN and SAT self-countermeasure training
Step 3.3: the problem of cross-stage gradient information repetition is optimized by using a cross-stage local network CSPNet, and a ResNet residual structure optimization model overall framework is adopted, as shown in figure 2. Firstly, dividing the feature mapping of the base layer into two parts, and then merging the two parts through a cross-stage hierarchical structure, so that the accuracy can be ensured while the calculated amount is reduced;
step 3.4: sending the feature graphs with different sizes obtained in the step 3.3 into an SPP network, and carrying out pooling operation with fixed sizes on the feature graphs to obtain features with fixed dimensions;
step 3.5: performing parameter aggregation on the feature graphs with different sizes in the step 3.4 by using an FPN+PAN structure to obtain a fused feature graph, and obtaining better feature expression;
step 3.6: and inputting the fusion feature map into a Head output layer to obtain a prediction frame output result, and obtaining the prediction frame, so that the relevant information of the identified vehicle, such as the vehicle position, the center coordinate, the target pixel size and the like, can be accurately known.
Step 4, in the full-size digital model in the step 1, fitting a lane curve according to a training diagram based on a curve-fitting lane line detection principle, and splicing a plurality of coherent camera views into a complete pavement lane through view stitching; vehicle identification and trajectory tracking are implemented in a full-scale model. The invention provides a method for selecting an optimal subset based on LMedsquare on the basis of a lane line detection principle based on curve fitting. The lane line detection algorithm comprises the following steps:
step 4.1: and (5) selecting seed points. To accurately obtain the lane line coordinate equation, the center point of the lane line needs to be found first, and then lane line fitting is performed on the set of the center points of the lane line. Therefore, it is necessary to find the lane line center point. The lane line center point can be found out according to the characteristics of the lane lines after binarization of the lane line image, specifically, a gray level image is obtained through binarization, the outline of a road pavement and a road mark is distinguished by gray level saturation, the lane mark position is extracted by using the outline characteristics, and finally, the lane center point is obtained through the consistency characteristics of the marks, so that preparation work is carried out for the next step of lane line parameter fitting;
step 4.2: and (5) detecting a lane marking line. After classifying the seed points, judging whether the seed points to be fitted are sufficient, under the condition of good road conditions, the near-sight lane lines are clearer, and the number of the seed points of the lane lines which are sufficient is generally obtained by scanning and classifying from bottom to top, so that the lane line detection algorithm of the optimal subset can be selected by directly utilizing the LMedsquare to perform parameter fitting;
step 4.3: a least squares straight line fit based on the optimal subset. The fitting curve by using the least square method is a common fitting algorithm in the intelligent navigation technology, but has the biggest defects of being sensitive to noise, and obviously, solving the influence of the noise is the premise and the basis for accurately fitting out the lane line parameters. Therefore, when the number of the seed points is sufficient after the seed points of the lane lines are classified, an optimal subset can be selected by utilizing the thought of the LMedsquare curve fitting technology, so that after redundant noise is removed, the least square method is utilized to perform linear fitting in the optimal subset, a good effect can be obtained, and the fitted subset is obtained to synthesize the lane curve, so that reference data is provided for the following cross-view tracking;
in the system, an optimal camera layout scheme is made according to road conditions through a camera layout algorithm, wherein regional overlapping is an important data acquisition point for detecting vehicle regression. The vehicles are tracked at the same time in the overlapping area, and the data of the same lane is determined to be stitched into a uniformly-identified vehicle tracking data stream by using a lane recognition technology, and the uniformly-identified vehicle tracking data stream is used as a parameter of the next regression. The overall framework through the camera layout algorithm is shown in fig. 7, and the specific method is as follows:
a) Selecting a camera product with corresponding parameters according to the existing highway parameters;
b) Obtaining the interval and the number of the longitudinal cameras according to the visible distance of the cameras and the lengths of the road surface and the bridge deck;
c) Obtaining the number of the transverse cameras according to the view angles of the cameras, pixels and road parameters;
d) Obtaining the basic layout of the cameras on the highway according to the mountable camera positions on the highway and the data of the step b) and the step c), and checking the basic layout of the cameras through the virtual cameras arranged in the digital model established in the step 1;
e) The camera basic layout is combined, the actual camera layout is corrected according to the actual mountable positions of the cameras on the road surface and the bridge deck as recursive parameters, and the corrected actual camera layout is checked through the virtual cameras arranged in the digital model established in the step 1; the basic layout data provides a reference for actually installing the camera, and the position data of the mountable camera changes according to the actual installation condition (depending on factors such as whether additional facilities can be erected on a road surface to meet the installation position of the camera or the camera can only be installed on a specific position);
f) Determining the effective recognition distance of the vehicle in the actual camera layout according to the frequency of the target vehicle in the camera view area after the related information of the target vehicle is recognized in the step 3;
g) Correcting the actual camera layout in the step e) by taking the effective recognition distance as a parameter, namely, increasing or decreasing the camera interval to ensure that the effective recognition distance of the adjacent cameras can connect the covered road surfaces, so as to achieve the optimal coverage rate (the difference between the visual radii of the two cameras/the length of the covered road surfaces) and meet the condition that the visual fields of the adjacent cameras are mutually covered;
h) And further correcting the layout of the actual cameras according to the coverage rate to obtain the number and the installation positions of the required cameras.
And 5, the relevant information of the target vehicle tracked in each camera is mapped through a coordinate system of the stitching vision field, so that the recognition and tracking of the vehicle across the vision field are realized.
In step 5, the vehicle is identified across the viewing area, and the specific method of tracking is as follows:
a) The target vehicle identified by the camera identifies the lane where the target vehicle is located in the camera view according to the lane curve data fitted in the step 4;
b) According to the camera layout in the step 4, the target vehicle can appear in two adjacent camera vision fields at the same time;
c) And obtaining the lane position of the target vehicle through the fitted lane curve data, and judging whether the camera is the same vehicle or not by combining the real-time data of the target vehicle obtained by the camera and the position of the target vehicle in the camera identification area, so as to realize the tracking of the vehicle across the vision.
And 5, cross-camera association. The invention adopts the splicing technology of the vision coordinate system, and feeds the coordinate system back to the road simulation system to realize the tracking processing of the vehicle across cameras. The efficiency of vehicle cross-view tracking is improved, and an optimal coverage rate is obtained through calculation.
Step 5.1: a camera coordinate system is acquired. The coordinate system is that the camera stands on the angle of the camera to measure the object, the object under the world coordinate system needs to go through rigid body change to be transferred to the camera coordinate system, and then the relationship with the road image coordinate system occurs.
Step 5.2: and acquiring a road image coordinate system. The coordinate system is established based on the two-dimensional photograph taken by the camera. For specifying the position of the object in the photograph. (x, y) is a continuous image coordinate or a space image coordinate, and (u, v) is a discrete image coordinate system or a pixel image coordinate system
Step 5.3: converting road coordinate system into pixel coordinate system, which can be directly applied to camera picture as mark coordinate for identifying object, and can be mapped onto road simulation system as important parameter for data visualization
Although embodiments of the present invention have been disclosed above, it is not limited to the details and embodiments shown and described, it is well suited to various fields of use for which the invention would be readily apparent to those skilled in the art, and accordingly, the invention is not limited to the specific details and illustrations shown and described herein, without departing from the general concepts defined in the claims and their equivalents.
Claims (9)
1. A bridge deck vehicle real-time tracking method based on a field enhanced synthetic data set is characterized by comprising the following steps:
step 1, in 3D modeling software, full-size digital model modeling is carried out according to the actual size of a bridge, different types of vehicle models are arranged on a bridge deck, a vehicle behavior model is built, a three-dimensional vehicle condition road condition scene is generated through rendering, the motion state of a vehicle on the bridge deck is obtained through an arranged virtual camera, and analog image data are generated;
step 2, performing generation type countermeasure learning by using the simulated image data and the real image data, generating synthesized simulated image data, and labeling the synthesized simulated image data and the real image data to complete the construction of a synthesized data set;
step 3, training a target detection network by adopting a synthetic data set as a training set and a random sampling data enhancement method based on a YOLO V4 algorithm, synthesizing a training diagram containing various target characteristics, and identifying target vehicle related information;
step 4, in the full-size digital model in the step 1, fitting a lane curve according to a training diagram based on a curve-fitting lane line detection principle, and splicing a plurality of coherent camera views into a complete road lane through view stitching;
step 5, the relevant information of the target vehicle tracked in each camera is mapped through a stitching view coordinate system, so that the vehicle cross-view identification and tracking are realized;
the step 3 specifically includes:
step 3.1: dividing the synthetic data set into n batches for batch processing, taking out one batch, randomly selecting 4 pictures, cutting and splicing the random positions into new pictures, and repeating the batch_size for times to obtain a new fusion data set;
step 3.2: after the image is subjected to the random sampling data enhancement processing in the step 3.1, the image is subjected to the pretreatment of the cmBN and SAT self-countermeasure training;
step 3.3: the method comprises the steps of optimizing a cross-stage gradient information repetition problem by using a cross-stage local network CSPNet, adopting a ResNet residual error structure optimization model integral frame, dividing a feature map of a base layer into two parts, and merging the two parts through a cross-stage hierarchical structure;
step 3.4: sending the feature images with different sizes obtained in the step 3.3 into an SPP network, and carrying out pooling operation with fixed sizes on the feature images to obtain feature images with fixed dimensions;
step 3.5: performing parameter aggregation on the feature graphs with different sizes in the step 3.4 by using an FPN+PAN structure to obtain a fusion feature graph;
step 3.6: and (3) inputting the fusion feature map obtained in the step (3.5) into a Head output layer to obtain a prediction frame output result, and obtaining relevant information for identifying the vehicle.
2. The real-time tracking method for bridge deck vehicles based on the field enhanced synthetic data set according to claim 1, wherein the step 1 specifically comprises:
step 1.1: constructing an actual bridge full-size digital model in 3D modeling software Blender by adopting a solid modeling method, manufacturing three-dimensional vehicle model libraries of various types of vehicles, and inputting the three-dimensional vehicle model libraries into the Blender model libraries;
step 1.2: establishing a multi-type vehicle behavior model conforming to actual traffic rules;
step 1.3: different types of vehicle models are imported according to actual engineering requirements, and specific vehicle behavior models are given to the vehicle models, so that the vehicle models run in a full-size digital model according to set paths and behavior modes, and real traffic conditions are simulated;
step 1.4: adjusting parameters of a simulation system, and simulating the shielding condition of the camera part under different weather conditions and dynamic traffic flow situations to obtain the running states of the vehicle under different situations;
step 1.5: setting a virtual camera, collecting the motion state of a vehicle in a bridge deck lane in the running process of a model, deriving the motion state into a video format, and converting the motion state into a picture format frame by frame to generate analog image data.
3. The real-time tracking method for bridge deck vehicles based on the field enhanced synthetic data set according to claim 1, wherein the step 2 specifically comprises:
step 2.1: firstly, identifying and classifying vehicles to be tracked, importing real images of different types of vehicles into a model as a real image data set to serve as a primary data source, and combining simulation image data obtained by vehicle traffic simulation in the model to serve as a secondary data source;
step 2.2: generating a random noise set { z } and a sample set { x } by performing a generative countermeasure learning based on the analog image data, i.e., the secondary data source, and the real image data, i.e., the primary data source;
step 2.3: randomly sampling m samples from a gaussian noise distribution, i.e., a random noise set { z }; randomly sampling m samples from a real data distribution, namely a random sample set { x };
step 2.4: updating the discriminant S by means of a random gradient descent SGD method w Parameters of (2);
step 2.5: randomly sampling another m samples from Gaussian noise distribution, namely a random noise set { z };
step 2.6: updating the arbiter G by a random gradient descent SGD method θ Parameters of (2);
step 2.7: obtaining generated simulated image data, and using the generated simulated image data as a data set for training a neural network;
step 2.8: and labeling the obtained simulated image data set and the real image data set to complete the construction of the synthetic data set.
4. The real-time tracking method for bridge deck vehicles based on the field enhanced composite data set according to claim 1, wherein the real image data in step 1 is a vehicle picture extracted from other bridge monitoring videos acquired from a network, and the vehicle picture is provided with corresponding identification data, including the model number, the size and the pixel coordinates of the vehicle in the picture.
5. The real-time tracking method for bridge deck vehicles based on the field enhanced synthetic data set according to claim 1, wherein the labeling in the step 2 is to use a labeling tool to mark the vehicle position on the obtained image data in the form of a rectangular frame and mark the tag information in the form of a file name; the relevant information of the target vehicle identified in the step 3 comprises the vehicle position, the center coordinates and the pixel size of the target vehicle.
6. The real-time tracking method for bridge floor vehicles based on the field enhanced synthetic data set according to claim 1, wherein the specific method for obtaining the bridge floor camera layout in the step 4 is as follows:
a) Selecting a camera product with corresponding parameters according to the existing highway parameters;
b) Obtaining the interval and the number of the longitudinal cameras according to the visible distance of the cameras and the lengths of the road surface and the bridge deck;
c) Obtaining the number of the transverse cameras according to the view angles of the cameras, pixels and road parameters;
d) Obtaining the basic layout of the cameras on the highway according to the mountable camera positions on the highway and the data of the step b) and the step c), and checking the basic layout of the cameras through the virtual cameras arranged in the digital model established in the step 1;
e) The camera basic layout is combined, the actual camera layout is corrected according to the actual mountable positions of the cameras on the road surface and the bridge deck as recursive parameters, and the corrected actual camera layout is checked through the virtual cameras arranged in the digital model established in the step 1;
f) Determining the effective recognition distance of the vehicle in the actual camera layout according to the frequency of the target vehicle in the camera view area after the related information of the target vehicle is recognized in the step 3;
g) Correcting the actual camera layout in the step e) by taking the effective recognition distance as a parameter, namely, increasing or decreasing the camera interval to ensure that the effective recognition distances of the adjacent cameras can connect the covered pavement, so as to obtain the optimal coverage rate;
h) And further correcting the layout of the actual cameras according to the coverage rate to obtain the number and the installation positions of the required cameras.
7. The real-time tracking method for bridge deck vehicles based on the field enhanced synthetic data set according to claim 6, wherein the lane curve fitting in the step 4 specifically comprises:
step 4.1: selecting seed points, namely searching a center point of a lane line, performing lane line fitting on a set of lane line center points, obtaining a gray level image through binarization based on lane line images in a training image, distinguishing the outline of a road surface and a road mark by using gray level saturation, extracting a lane mark position by using outline features, and finally obtaining the lane center point and classifying by using consistency features of the marks;
step 4.2: after the seed points, namely the center points of the lanes, are classified, under the condition of better road conditions, the near-sight lane lines are clearer, and the seed points with more sufficient number of lane lines are obtained by scanning and classifying from bottom to top;
step 4.3: after classifying the seed points of the lane lines and when the number of the seed points is sufficient, selecting an optimal subset by using an LMedsquare curve fitting technology to remove redundant noise, and then performing linear fitting in the optimal subset by using a least square method to obtain a lane curve.
8. The real-time tracking method for bridge deck vehicles based on the field enhanced synthetic data set according to claim 7, wherein the specific method for identifying and tracking vehicles across the viewing area in the step 5 is as follows:
a) The target vehicle identified by the camera identifies the lane where the target vehicle is located in the camera view according to the lane curve data fitted in the step 4;
b) According to the camera layout in the step 4, the target vehicle can appear in two adjacent camera vision fields at the same time;
c) And obtaining the lane position of the target vehicle through the fitted lane curve data, and judging whether the camera is the same vehicle or not by combining the real-time data of the target vehicle obtained by the camera and the position of the target vehicle in the camera identification area, so as to realize the tracking of the vehicle across the vision.
9. The real-time tracking method for bridge deck vehicles based on the field enhanced synthetic data set according to claim 8, wherein the specific method for mapping the coordinate system after the cameras are spliced by the vision field in the step 5 comprises the following steps:
step 5.1: acquiring a camera coordinate system, which is a coordinate system of measuring an object by a camera standing on the angle of the camera;
step 5.2: a road image coordinate system is acquired, which is a coordinate system established based on a two-dimensional photograph taken by a camera and used for specifying the position of a vehicle in the photograph, wherein (x, y) is a continuous image coordinate or a space image coordinate, and (u, v) is a discrete image coordinate system or a pixel image coordinate system,
step 5.3: the road image coordinate system is converted into a pixel coordinate system, the coordinates can be directly applied to a camera picture as mark coordinates for identifying vehicles, and can be mapped onto a road simulation system as parameters for data visualization,
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210680866.6A CN115035251B (en) | 2022-06-16 | 2022-06-16 | Bridge deck vehicle real-time tracking method based on field enhanced synthetic data set |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210680866.6A CN115035251B (en) | 2022-06-16 | 2022-06-16 | Bridge deck vehicle real-time tracking method based on field enhanced synthetic data set |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115035251A CN115035251A (en) | 2022-09-09 |
| CN115035251B true CN115035251B (en) | 2024-04-09 |
Family
ID=83124502
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210680866.6A Active CN115035251B (en) | 2022-06-16 | 2022-06-16 | Bridge deck vehicle real-time tracking method based on field enhanced synthetic data set |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115035251B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020061489A1 (en) * | 2018-09-20 | 2020-03-26 | Nvidia Corporation | Training neural networks for vehicle re-identification |
| CN110941996A (en) * | 2019-11-04 | 2020-03-31 | 深圳市唯特视科技有限公司 | Target and track augmented reality method and system based on generation of countermeasure network |
| CN112257683A (en) * | 2020-12-07 | 2021-01-22 | 之江实验室 | Cross-mirror tracking method for vehicle running track monitoring |
| CN112927267A (en) * | 2021-03-15 | 2021-06-08 | 河海大学 | Target tracking method under multi-camera scene |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019168869A1 (en) * | 2018-02-27 | 2019-09-06 | Nvidia Corporation | Real-time detection of lanes and boundaries by autonomous vehicles |
-
2022
- 2022-06-16 CN CN202210680866.6A patent/CN115035251B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020061489A1 (en) * | 2018-09-20 | 2020-03-26 | Nvidia Corporation | Training neural networks for vehicle re-identification |
| CN110941996A (en) * | 2019-11-04 | 2020-03-31 | 深圳市唯特视科技有限公司 | Target and track augmented reality method and system based on generation of countermeasure network |
| CN112257683A (en) * | 2020-12-07 | 2021-01-22 | 之江实验室 | Cross-mirror tracking method for vehicle running track monitoring |
| CN112927267A (en) * | 2021-03-15 | 2021-06-08 | 河海大学 | Target tracking method under multi-camera scene |
Non-Patent Citations (1)
| Title |
|---|
| 深度学习的目标跟踪算法综述;李玺;查宇飞;张天柱;崔振;左旺孟;侯志强;卢湖川;王菡子;;中国图象图形学报;20191216(12);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115035251A (en) | 2022-09-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Dhiman et al. | Pothole detection using computer vision and learning | |
| CN111563442B (en) | Slam method and system for fusing point cloud and camera image data based on laser radar | |
| CN103377476B (en) | Use the image registration of the multimodal data of three-dimensional geographical arc | |
| TWI386625B (en) | Photographic position analysis method | |
| CN110285793A (en) | A kind of Vehicular intelligent survey track approach based on Binocular Stereo Vision System | |
| CN110322702A (en) | A kind of Vehicular intelligent speed-measuring method based on Binocular Stereo Vision System | |
| CN105930819A (en) | System for real-time identifying urban traffic lights based on single eye vision and GPS integrated navigation system | |
| CN110069972A (en) | Automatic detection real world objects | |
| CN110175576A (en) | A kind of driving vehicle visible detection method of combination laser point cloud data | |
| CN111179300A (en) | Method, apparatus, system, device and storage medium for obstacle detection | |
| CN111340855A (en) | Road moving target detection method based on track prediction | |
| CN108107462A (en) | The traffic sign bar gesture monitoring device and method that RTK is combined with high speed camera | |
| US20240017747A1 (en) | Method and system for augmenting lidar data | |
| CN105608417A (en) | Traffic signal lamp detection method and device | |
| CN112070756B (en) | Three-dimensional road surface disease measuring method based on unmanned aerial vehicle oblique photography | |
| CN106558051A (en) | A kind of improved method for detecting road from single image | |
| CN106446785A (en) | Passable road detection method based on binocular vision | |
| CN112581402B (en) | Road and bridge fault automatic detection method based on machine vision technology | |
| CN115388902A (en) | Indoor positioning method and system, AR indoor positioning navigation method and system | |
| Sun et al. | Geographic, geometrical and semantic reconstruction of urban scene from high resolution oblique aerial images. | |
| CN114663473A (en) | Personnel target positioning and tracking method and system based on multi-view information fusion | |
| CN110349209A (en) | Vibrating spear localization method based on binocular vision | |
| CN115035251B (en) | Bridge deck vehicle real-time tracking method based on field enhanced synthetic data set | |
| CN116630267A (en) | Roadbed settlement monitoring method based on unmanned aerial vehicle and laser radar data fusion | |
| Bellusci et al. | Semantic interpretation of raw survey vehicle sensory data for lane-level HD map generation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |