CN116778292B - Method, device, equipment and storage medium for fusing space-time trajectories of multi-mode vehicles - Google Patents

Method, device, equipment and storage medium for fusing space-time trajectories of multi-mode vehicles Download PDF

Info

Publication number
CN116778292B
CN116778292B CN202311041161.0A CN202311041161A CN116778292B CN 116778292 B CN116778292 B CN 116778292B CN 202311041161 A CN202311041161 A CN 202311041161A CN 116778292 B CN116778292 B CN 116778292B
Authority
CN
China
Prior art keywords
track
time
space
data
mode
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311041161.0A
Other languages
Chinese (zh)
Other versions
CN116778292A (en
Inventor
王东锋
余亦阳
姚相松
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Qianhai Zhongdian Huian Technology Co ltd
Original Assignee
Shenzhen Qianhai Zhongdian Huian Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Qianhai Zhongdian Huian Technology Co ltd filed Critical Shenzhen Qianhai Zhongdian Huian Technology Co ltd
Priority to CN202311041161.0A priority Critical patent/CN116778292B/en
Publication of CN116778292A publication Critical patent/CN116778292A/en
Application granted granted Critical
Publication of CN116778292B publication Critical patent/CN116778292B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0464Convolutional networks [CNN, ConvNet]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761Proximity, similarity or dissimilarity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02TCLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00Road transport of goods or passengers
    • Y02T10/10Internal combustion engine [ICE] based vehicles
    • Y02T10/40Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Traffic Control Systems (AREA)

Abstract

The embodiment of the invention discloses a method, a device, equipment and a storage medium for fusing space-time trajectories of a multi-mode vehicle, and relates to the technical field of intelligent security. The method comprises the following steps: acquiring various vehicle space-time track data of different modes; generating a single-mode track diagram of each mode according to the vehicle space-time track data; inputting each single-mode track graph to a trained graph convolution neural network model to generate single-mode track expression vectors of each mode; and calculating the similarity between different modal trajectories according to the single-modal trajectory representation vector, and fusing the trajectories meeting the similarity threshold. The technical scheme provided by the embodiment of the invention realizes the efficient and accurate fusion of the space-time track of the vehicle under the condition of big data, and has good adaptability on the multi-mode sparse track.

Description

Method, device, equipment and storage medium for fusing space-time trajectories of multi-mode vehicles
Technical Field
The embodiment of the invention relates to the technical field of intelligent security, in particular to a method, a device, equipment and a storage medium for fusing space-time trajectories of a multi-mode vehicle.
Background
With the acceleration of the urban process, the urban safety problem is more and more emphasized. The smart safe city needs to comprehensively use various technical means including information fusion technology, internet of things technology, artificial intelligence and the like, so that information sharing and cooperative work among all subsystems of the city are realized, and the safety and intelligence level of the city are improved. The track fusion technology is a technology for fusing target tracks obtained by different sensors or the same sensor in different time periods, and can realize accurate tracking and identification of the target by fusing different tracks of the target. As an information fusion technology, the method can make important contributions to smart safe cities, such as improving the safety and efficiency of public transportation, improving the management efficiency of urban security and protection, improving the efficiency of urban emergency response and the like.
However, in practical application, the application scenario of the track fusion technology has the following characteristics: (1) The real-time performance requirement is extremely high, and in a smart safety urban system, real-time monitoring and data analysis are very important, and particularly when an emergency occurs, the system analysis and response speed can directly influence the operation of the urban safety system; (2) In a smart safe urban system, track data are often from different sensors, and the types, the precision, the sampling rate and the like of the different sensors may be completely different, for example, the positioning precision of a monitoring camera and ETC equipment is high, but the acquisition range is small, the monitoring camera has probability of identifying errors, the acquisition range of a detection code equipment is large, but the positioning precision is low, and the characteristics of the different mode data need to be considered in the track fusion process; (3) There are sparse tracks which are difficult to process, due to the limitation of the number and the precision of the sensors, there are often a considerable proportion of sparse tracks which are difficult to process, the sparse tracks can have a large influence on track fusion, moreover, the anti-reconnaissance awareness of partial targets is strong, and some important track data can be hidden in the sparse tracks.
The existing track fusion method mainly comprises a distance measurement-based method, a time sequence-based method and a filtering-based method. The method based on distance measurement firstly defines the distance between certain tracks, the closer the distance between tracks is, the higher the correlation degree is, then the most relevant tracks are fused based on distance retrieval, the effect of the method is completely dependent on whether the distance definition mode is matched with the solved problem and data, and the data with different modes and different characteristics have different most suitable track distance definitions, so that the effect is not good when the method based on distance performs track fusion between the data with different modes and larger characteristic difference. The method based on the time sequence firstly extracts certain characteristics based on the time sequence, then judges whether the tracks are to be fused according to the characteristics through a model, the method needs to manually design the characteristics, the dependence on the time sequence characteristics is large, the problem of sparse time sequence characteristics caused by different sampling rates is difficult to process, in addition, the manually designed characteristics are highly dependent on the experience of designers, and the effect is poor. The filtering-based method is to fuse observation data with priori knowledge to estimate the state of the track, and mainly comprises Kalman filtering and particle filtering, wherein the Kalman filtering is suitable for a linear system, is difficult to apply in a nonlinear system, and the behavior rules of personnel and vehicles in cities are usually not simple linear rules, and the particle filtering can be suitable for the nonlinear system and a system with serious noise, but the problem of overlarge calculation load is easy to generate in a large-scale system.
Disclosure of Invention
The embodiment of the invention provides a method, a device, equipment and a storage medium for fusing space-time tracks of a multi-modal vehicle, which are used for realizing the efficient and accurate fusion of the space-time tracks of the vehicle under the condition of big data and have good adaptability on the multi-modal sparse tracks.
In a first aspect, an embodiment of the present invention provides a method for fusing space-time trajectories of a multi-modal vehicle, where the method includes:
acquiring various vehicle space-time track data of different modes;
generating a single-mode track diagram of each mode according to the vehicle space-time track data;
inputting each single-mode track graph to a trained graph convolution neural network model to generate single-mode track expression vectors of each mode;
and calculating the similarity between different modal trajectories according to the single-modal trajectory representation vector, and fusing the trajectories meeting the similarity threshold.
Optionally, the generating a single-mode track map of each mode according to the vehicle space-time track data includes:
constructing a global space topological graph according to the communication relation of the acquisition equipment of the space-time track data of the vehicle on the road network;
generating a track space topological graph on the global space topological graph according to the vehicle space-time track data; nodes in the track space topological graph represent the acquisition equipment, node weights represent the acquisition times of the acquisition equipment, connecting edges represent the communication relation among the acquisition equipment, and connecting edge weights represent the track passing times on the corresponding connecting edges;
generating a track time topological graph according to the time law of the vehicle space-time track data; nodes in the track time topological graph represent time intervals, node weights represent acquisition times in corresponding time intervals, and continuous edges represent sequential relation of acquisition in different time intervals;
generating a track space-time topological graph as the single-mode track graph according to the track space topological graph and the track time topological graph; the track space-time topological graph is a double-layer topological graph, the inter-layer continuous edge represents the relation between the acquisition equipment and the time interval, and the inter-layer continuous edge weight represents the acquisition times of the acquisition equipment in the corresponding time interval.
Optionally, before the inputting each of the single-mode trajectory graphs into the trained graph convolution neural network model to generate a single-mode trajectory representation vector of each mode, the method further includes:
acquiring a single-mode track diagram sample of multiple modes;
establishing the graph convolution neural network model;
and training the graph convolution neural network model according to the single-mode track graph sample and the corresponding preset track fusion relation annotation data.
Optionally, before the generating the single-mode track map of each mode according to the vehicle space-time track data, the method further includes:
and preprocessing the space-time track data of the vehicle, wherein the preprocessing comprises deleting the missing data of the key field.
Optionally, before the generating the single-mode track map of each mode according to the vehicle space-time track data, the method further includes:
and storing the space-time track data of the vehicle into a database, and constructing an index based on the fields.
Optionally, the vehicle space-time trajectory data includes IMSI data, license plate data, and ETC data.
Optionally, the acquiring the plurality of vehicle space-time trajectory data of different modalities includes:
acquiring the IMSI data through a code detection device;
obtaining license plate image data through a license plate camera, and inputting the license plate image data into an optical character recognition system to obtain the license plate data;
and acquiring the ETC data through ETC equipment.
In a second aspect, an embodiment of the present invention further provides a device for fusing space-time trajectories of a multi-modal vehicle, where the device includes:
the vehicle track data acquisition module is used for acquiring various vehicle space-time track data of different modes;
the single-mode track diagram generation module is used for respectively generating single-mode track diagrams of all modes according to the vehicle space-time track data;
the track representation vector generation module is used for respectively inputting each single-mode track graph into the trained graph convolution neural network model so as to generate single-mode track representation vectors of each mode;
and the vehicle space-time track fusion module is used for calculating the similarity between different modal tracks according to the single-modal track expression vector and fusing the tracks meeting the similarity threshold.
In a third aspect, an embodiment of the present invention further provides a computer apparatus, including:
one or more processors;
a memory for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method for fusion of space-time trajectories of a multi-modal vehicle provided by any embodiment of the present invention.
In a fourth aspect, an embodiment of the present invention further provides a computer readable storage medium, where a computer program is stored, where the program when executed by a processor implements the method for fusion of space-time trajectories of a multi-modal vehicle provided by any embodiment of the present invention.
The embodiment of the invention provides a fusion method of space-time trajectories of a multi-mode vehicle, which comprises the steps of firstly obtaining various vehicle space-time trajectory data of different modes, then respectively generating single-mode trajectory graphs of all modes according to the obtained vehicle space-time trajectory data, then respectively inputting all the single-mode trajectory graphs into a trained graph convolution neural network model to generate single-mode trajectory expression vectors of all the modes, finally calculating the similarity between the trajectories of the different modes according to the single-mode trajectory expression vectors, and fusing the trajectories meeting a similarity threshold. According to the multi-mode vehicle space-time track fusion method provided by the embodiment of the invention, the space-time tracks are mapped into the graphs, so that the data of different modes can be represented in the same mode, the gap between data with different characteristics is broken, the algorithm and the training model are not required to be designed for the data of each mode, the research and development cost and the applicable cost and the efficiency are greatly saved, the graph convolution neural network can be successfully applied to the track fusion field, the expansion among the multi-mode data with different characteristics can be conveniently realized, meanwhile, the sparse data is converted in the graph structure mode, more global space-time correlation information is obtained, the data representation mode is not sparse, the model interference caused by the sparse data is avoided, and the vehicle space-time track fusion under the condition of big data is realized.
Drawings
FIG. 1 is a flowchart of a method for fusing space-time trajectories of a multi-modal vehicle according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a device for fusing space-time trajectories of a multi-modal vehicle according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a computer device according to a third embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart depicts steps as a sequential process, many of the steps may be implemented in parallel, concurrently, or with other steps. Furthermore, the order of the steps may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, and the like.
Example 1
Fig. 1 is a flowchart of a method for fusing space-time trajectories of a multi-modal vehicle according to an embodiment of the present invention. The embodiment of the invention is applicable to the situation that the space-time track of the vehicle needs to be fused in the applications of a smart city system, a city intelligent security system, a graphic code alliance detection system, a vehicle code association system and the like, the method can be executed by the multi-mode vehicle space-time track fusion device provided by the embodiment of the invention, and the device can be realized by hardware and/or software and can be generally integrated in computer equipment. As shown in fig. 1, the method specifically comprises the following steps:
s11, acquiring various vehicle space-time track data of different modes.
S12, respectively generating a single-mode track map of each mode according to the vehicle space-time track data.
S13, respectively inputting the single-mode track diagrams into the trained diagram convolution neural network model to generate single-mode track expression vectors of all modes.
S14, calculating the similarity between different modal tracks according to the single-modal track expression vector, and fusing the tracks meeting the similarity threshold.
The vehicle spatiotemporal track data may include any data that may characterize the spatiotemporal information of the vehicle's presence, including, optionally, IMSI data, license plate data, and ETC data. Further optionally, the acquiring the plurality of vehicle space-time trajectory data of different modalities includes: acquiring the IMSI data through a code detection device; obtaining license plate image data through a license plate camera, and inputting the license plate image data into an optical character recognition system to obtain the license plate data; and acquiring the ETC data through ETC equipment. Specifically, first, acquisition equipment such as a code detection device, a license plate camera, an ETC device and the like can be debugged to obtain a stable data source, and then when a vehicle appears near the acquisition equipment, corresponding data can be acquired in real time. The IMSI data can be acquired by the code detection device, the code detection device can acquire IMSI signals in a certain surrounding range, record information such as longitude and latitude, time, IMSI numbers, number attribution and the like, the acquisition range is generally 200-1500 m unequal according to the type and parameter setting of the code detection device, the IMSI (International Mobile Subscriber Identity ) is used for distinguishing different users of a mobile network, the IMSI code can be a mobile phone, each mobile phone SIM card has a unique IMSI code, and the vehicle track can be represented by the track of the IMSI data of personnel in the vehicle. The license plate camera can be used for shooting to obtain license plate image data, then the license plate image data can be input into an optical character recognition (Optical Character Recognition, OCR) system for image recognition, so that the license plate number is extracted, and the required license plate data can be obtained by combining information such as the geographic position (such as longitude and latitude) and shooting time of the license plate camera. ETC data can also be obtained through ETC (Electronic Toll Collection ) equipment, and the ETC data can comprise vehicle-mounted electronic tags, acquisition time, longitude and latitude and the like.
Optionally, before the generating the single-mode track map of each mode according to the vehicle space-time track data, the method further includes: and preprocessing the space-time track data of the vehicle, wherein the preprocessing comprises deleting the missing data of the key field. Specifically, the acquisition equipment has probability of error in acquisition and transmission to cause loss of key fields, the data missing the key information can not provide space-time information and can not participate in track fusion calculation, and the data missing in the key fields such as longitude, latitude, time and the like can be deleted to save storage space and facilitate subsequent processing.
Optionally, before the generating the single-mode track map of each mode according to the vehicle space-time track data, the method further includes: and storing the space-time track data of the vehicle into a database, and constructing an index based on the fields. Specifically, an index can be constructed based on fields such as time, longitude and latitude, ID and the like, so that subsequent inquiry and calling of vehicle space-time track data are facilitated.
After the required vehicle space-time track data are obtained, a single-mode track graph of each mode can be respectively constructed aiming at the vehicle space-time track data of each mode so as to convert the track into a topological graph, and therefore the follow-up model can judge the similarity of the track through the similarity of the graph.
Optionally, for each mode of vehicle space-time trajectory data, the generating a single-mode trajectory graph of each mode according to the vehicle space-time trajectory data includes: constructing a global space topological graph according to the communication relation of acquisition equipment (such as a code detection device, a license plate camera or ETC device and the like) of the space-time track data of the vehicle on a road network; generating a track space topological graph on the global space topological graph according to the vehicle space-time track data; nodes in the track space topological graph represent the acquisition equipment, node weights represent the acquisition times of the acquisition equipment, connecting edges represent the communication relation among the acquisition equipment, and connecting edge weights represent the track passing times on the corresponding connecting edges; generating a track time topological graph according to the time law of the vehicle space-time track data; nodes in the track time topological graph represent time intervals, node weights represent acquisition times in corresponding time intervals, and continuous edges represent sequential relation of acquisition in different time intervals; generating a track space-time topological graph as the single-mode track graph according to the track space topological graph and the track time topological graph; the track space-time topological graph is a double-layer topological graph (a double-layer is a track space topological graph and a track time topological graph respectively), the inter-layer continuous edge represents the relation between the acquisition equipment and the time interval (namely whether acquisition exists in the corresponding time interval), and the inter-layer continuous edge weight represents the acquisition times of the acquisition equipment in the corresponding time interval. The track space topological graph is a directed graph, nodes and connecting edges of the directed graph are from the global space topological graph, and the track space topological graph can be generated by determining the movement condition of the track in space according to the space-time track data of the vehicle.
After the single-mode track diagrams of all modes are obtained, the single-mode track diagrams can be respectively input into a trained graph convolution neural network model to conduct single-mode track prediction so as to predict and obtain single-mode track expression vectors of all modes. The graph neural network (Graph Neural Network, GNN) is a neural network based on graph structure data, and greatly expands the data form which can be applied to the neural network. The graph convolution neural network (Graph Convolutional Neural Network, GCN) expands convolution into a graph structure, and local feature learning and abstraction can be better performed through graph convolution calculation. The graph convolution neural network is commonly used in the fields of texts, images, knowledge maps and the like, but due to some characteristics of vehicle space-time trajectory data, the graph convolution neural network cannot be directly applied to the trajectory fusion problem, and the method maps the space-time trajectory into a space-time topological graph by constructing a single-mode trajectory graph, so that the graph convolution neural network can be successfully applied to the trajectory fusion field.
Optionally, before the inputting each of the single-mode trajectory graphs into the trained graph convolution neural network model to generate a single-mode trajectory representation vector of each mode, the method further includes: acquiring a single-mode track diagram sample of multiple modes; establishing the graph convolution neural network model; and training the graph convolution neural network model according to the single-mode track graph sample and the corresponding preset track fusion relation annotation data. Specifically, a required single-mode track map sample can be generated by referring to the generation process of the single-mode track map, then the single-mode track map sample is used as a model input to be provided for a built map convolutional neural network model, meanwhile, corresponding preset track fusion relation labeling data can be provided for the single-mode track map sample, a fusion relation of a small number of tracks can be labeled in the labeling data, and the model can automatically learn according to the labeled fusion relation. By training the model, the model can automatically mine rules from mass data, and how to perform track fusion is learned. After training is completed, the trained graph roll-up neural network model can be deployed for application in an actual scene. In the training process, the single-mode track diagram samples of different modes can be coupled together for training, and after the model is deployed, the single-mode track diagrams of different modes can be respectively inferred, so that the calculation complexity is reduced.
After the single-mode track expression vector of each mode is obtained, the similarity between different mode tracks can be calculated according to each single-mode track expression vector, then the calculated similarity can be compared with a preset similarity threshold, if certain similarity is higher than the similarity threshold, the tracks corresponding to the similarity can be fused, so that a track fusion result is obtained, and the similarity threshold can be selected according to experimental and test results.
According to the technical scheme provided by the embodiment of the invention, various vehicle space-time track data of different modes are firstly obtained, then single-mode track diagrams of all modes are respectively generated according to the obtained vehicle space-time track data, then all the single-mode track diagrams are respectively input into a trained graph convolution neural network model to generate single-mode track expression vectors of all the modes, finally the similarity among different mode tracks is calculated according to the single-mode track expression vectors, and tracks meeting a similarity threshold are fused. By mapping the space-time tracks into the graphs, the data of different modes can be represented in the same mode, gaps among the data with different characteristics are broken, an algorithm and a training model are not needed to be designed for the data of each mode, development and application costs and efficiency are greatly saved, so that the graph convolution neural network can be successfully applied to the track fusion field, expansion among the multi-mode data with different characteristics can be very convenient, meanwhile, sparse data are converted in a graph structure mode, more global space-time correlation information is obtained, the representation mode of the data is not sparse, interference of the sparse data on the model is avoided, and efficient and accurate fusion of the space-time tracks of the vehicle is realized under the condition of big data.
Example two
Fig. 2 is a schematic structural diagram of a device for fusing space-time trajectories of a multi-modal vehicle according to a second embodiment of the present invention, where the device may be implemented in hardware and/or software, and may be generally integrated in a computer device, for executing the method for fusing space-time trajectories of a multi-modal vehicle according to any embodiment of the present invention. As shown in fig. 2, the apparatus includes:
a vehicle track data acquisition module 21, configured to acquire a plurality of vehicle space-time track data of different modes;
a unimodal trajectory graph generating module 22, configured to generate a unimodal trajectory graph of each mode according to the vehicle space-time trajectory data;
a track representation vector generation module 23, configured to input each of the single-mode track graphs to a trained graph convolution neural network model, so as to generate a single-mode track representation vector of each mode;
and the vehicle space-time track fusion module 24 is used for calculating the similarity between different modal tracks according to the single-modal track expression vector and fusing the tracks meeting the similarity threshold.
According to the technical scheme provided by the embodiment of the invention, various vehicle space-time track data of different modes are firstly obtained, then single-mode track diagrams of all modes are respectively generated according to the obtained vehicle space-time track data, then all the single-mode track diagrams are respectively input into a trained graph convolution neural network model to generate single-mode track expression vectors of all the modes, finally the similarity among different mode tracks is calculated according to the single-mode track expression vectors, and tracks meeting a similarity threshold are fused. By mapping the space-time tracks into the graphs, the data of different modes can be represented in the same mode, gaps among the data with different characteristics are broken, an algorithm and a training model are not needed to be designed for the data of each mode, development and application costs and efficiency are greatly saved, so that the graph convolution neural network can be successfully applied to the track fusion field, expansion among the multi-mode data with different characteristics can be very convenient, meanwhile, sparse data are converted in a graph structure mode, more global space-time correlation information is obtained, the representation mode of the data is not sparse, interference of the sparse data on the model is avoided, and efficient and accurate fusion of the space-time tracks of the vehicle is realized under the condition of big data.
Based on the above technical solution, optionally, the single-mode trajectory graph generating module 22 includes:
the global space topological graph construction unit is used for constructing a global space topological graph according to the communication relation of the acquisition equipment of the space-time track data of the vehicle on the road network;
a track space topological graph generating unit, which is used for generating a track space topological graph on the global space topological graph according to the vehicle space-time track data; nodes in the track space topological graph represent the acquisition equipment, node weights represent the acquisition times of the acquisition equipment, connecting edges represent the communication relation among the acquisition equipment, and connecting edge weights represent the track passing times on the corresponding connecting edges;
the track time topological graph generating unit is used for generating a track time topological graph according to the time law of the vehicle space-time track data; nodes in the track time topological graph represent time intervals, node weights represent acquisition times in corresponding time intervals, and continuous edges represent sequential relation of acquisition in different time intervals;
the track space-time topological graph generating unit is used for generating a track space-time topological graph serving as the single-mode track graph according to the track space topological graph and the track time topological graph; the track space-time topological graph is a double-layer topological graph, the inter-layer continuous edge represents the relation between the acquisition equipment and the time interval, and the inter-layer continuous edge weight represents the acquisition times of the acquisition equipment in the corresponding time interval.
On the basis of the above technical solution, optionally, the apparatus for fusing space-time trajectories of a multi-modal vehicle further includes:
the training sample acquisition module is used for acquiring a plurality of modal single-mode track map samples before the single-mode track maps are respectively input into the trained graph convolution neural network model to generate single-mode track representation vectors of various modes;
the model building module is used for building the graph convolution neural network model;
and the model training module is used for training the graph roll neural network model according to the single-mode track graph sample and the corresponding preset track fusion relation annotation data.
On the basis of the above technical solution, optionally, the apparatus for fusing space-time trajectories of a multi-modal vehicle further includes:
the data preprocessing module is used for preprocessing the vehicle space-time track data before the single-mode track diagrams of all modes are respectively generated according to the vehicle space-time track data, and the preprocessing comprises deleting of the missing data of the key field.
On the basis of the above technical solution, optionally, the apparatus for fusing space-time trajectories of a multi-modal vehicle further includes:
and the data storage module is used for storing the vehicle space-time track data into a database and constructing an index based on the fields before the single-mode track map of each mode is respectively generated according to the vehicle space-time track data.
On the basis of the technical scheme, optionally, the vehicle space-time track data comprises IMSI data, license plate data and ETC data.
On the basis of the above technical solution, optionally, the vehicle track data acquisition module 21 includes:
the IMSI data acquisition unit is used for acquiring the IMSI data through the code detection equipment;
the license plate data acquisition unit is used for acquiring license plate image data through a license plate camera and inputting the license plate image data into the optical character recognition system so as to obtain the license plate data;
and the ETC data acquisition unit is used for acquiring the ETC data through ETC equipment.
The multi-mode vehicle space-time track fusion device provided by the embodiment of the invention can execute the multi-mode vehicle space-time track fusion method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
It should be noted that, in the embodiment of the multi-mode vehicle space-time trajectory fusion device, each unit and module included are only divided according to the functional logic, but are not limited to the above-mentioned division, so long as the corresponding functions can be implemented; in addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the present invention.
Example III
Fig. 3 is a schematic structural diagram of a computer device provided in a third embodiment of the present invention, and shows a block diagram of an exemplary computer device suitable for implementing an embodiment of the present invention. The computer device shown in fig. 3 is only an example and should not be construed as limiting the functionality and scope of use of embodiments of the invention. As shown in fig. 3, the computer apparatus includes a processor 31, a memory 32, an input device 33, and an output device 34; the number of processors 31 in the computer device may be one or more, in fig. 3, one processor 31 is taken as an example, and the processors 31, the memory 32, the input device 33, and the output device 34 in the computer device may be connected by a bus or other means, in fig. 3, by a bus connection is taken as an example.
The memory 32 is a computer readable storage medium, and may be used to store software programs, computer executable programs, and modules, such as program instructions/modules corresponding to the method for merging multi-modal vehicle spatiotemporal trajectories in the embodiment of the invention (e.g., the vehicle trajectory data acquisition module 21, the single-modal trajectory graph generation module 22, the trajectory representation vector generation module 23, and the vehicle spatiotemporal trajectory merging module 24 in the multi-modal vehicle spatiotemporal trajectory merging device). The processor 31 executes various functional applications of the computer device and data processing by running software programs, instructions and modules stored in the memory 32, i.e. implements the above-described method of fusion of the space-time trajectories of a multi-modal vehicle.
The memory 32 may mainly include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created according to the use of the computer device, etc. In addition, memory 32 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, memory 32 may further include memory located remotely from processor 31, which may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 33 may be used to acquire a variety of vehicle spatiotemporal trajectory data of different modalities, and to generate key signal inputs related to user settings and function controls of the computer device, etc. The output device 34 may include a display that may be used to present fusion results to a user, etc.
Example IV
A fourth embodiment of the present invention also provides a storage medium containing computer-executable instructions, which when executed by a computer processor, are for performing a method of fusion of space-time trajectories of a multi-modal vehicle, the method comprising:
acquiring various vehicle space-time track data of different modes;
generating a single-mode track diagram of each mode according to the vehicle space-time track data;
inputting each single-mode track graph to a trained graph convolution neural network model to generate single-mode track expression vectors of each mode;
and calculating the similarity between different modal trajectories according to the single-modal trajectory representation vector, and fusing the trajectories meeting the similarity threshold.
The storage medium may be any of various types of memory devices or storage devices. The term "storage medium" is intended to include: mounting media such as CD-ROM, floppy disk or tape devices; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, lanbus (Rambus) RAM, etc.; nonvolatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. In addition, the storage medium may be located in a computer system in which the program is executed, or may be located in a different second computer system connected to the computer system through a network (such as the internet). The second computer system may provide program instructions to the computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (e.g., in different computer systems connected by a network). The storage medium may store program instructions (e.g., embodied as a computer program) executable by one or more processors.
Of course, the storage medium containing the computer executable instructions provided in the embodiments of the present invention is not limited to the above-described method operations, and may also perform the related operations in the method for fusing space-time trajectories of a multi-modal vehicle provided in any embodiment of the present invention.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
From the above description of embodiments, it will be clear to a person skilled in the art that the present invention may be implemented by means of software and necessary general purpose hardware, but of course also by means of hardware, although in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, etc., and include several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments of the present invention.
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.

Claims (9)

1. A method for fusing space-time trajectories of a multi-modal vehicle, comprising:
acquiring various vehicle space-time track data of different modes;
generating a single-mode track diagram of each mode according to the vehicle space-time track data;
inputting each single-mode track graph to a trained graph convolution neural network model to generate single-mode track expression vectors of each mode;
calculating the similarity between different modal trajectories according to the single modal trajectory representation vector, and fusing the trajectories meeting the similarity threshold;
the generating a single-mode track map of each mode according to the vehicle space-time track data comprises the following steps:
constructing a global space topological graph according to the communication relation of the acquisition equipment of the space-time track data of the vehicle on the road network;
generating a track space topological graph on the global space topological graph according to the vehicle space-time track data; nodes in the track space topological graph represent the acquisition equipment, node weights represent the acquisition times of the acquisition equipment, connecting edges represent the communication relation among the acquisition equipment, and connecting edge weights represent the track passing times on the corresponding connecting edges;
generating a track time topological graph according to the time law of the vehicle space-time track data; nodes in the track time topological graph represent time intervals, node weights represent acquisition times in corresponding time intervals, and continuous edges represent sequential relation of acquisition in different time intervals;
generating a track space-time topological graph as the single-mode track graph according to the track space topological graph and the track time topological graph; the track space-time topological graph is a double-layer topological graph, the inter-layer continuous edge represents the relation between the acquisition equipment and the time interval, and the inter-layer continuous edge weight represents the acquisition times of the acquisition equipment in the corresponding time interval.
2. The method of claim 1, further comprising, prior to said separately inputting each of said single-modality trajectory maps into the trained graph-rolling neural network model to generate single-modality trajectory representation vectors for each modality:
acquiring a single-mode track diagram sample of multiple modes;
establishing the graph convolution neural network model;
and training the graph convolution neural network model according to the single-mode track graph sample and the corresponding preset track fusion relation annotation data.
3. The method of claim 1, further comprising, prior to generating the single-mode trajectory graph for each mode from the vehicle spatiotemporal trajectory data, respectively:
and preprocessing the space-time track data of the vehicle, wherein the preprocessing comprises deleting the missing data of the key field.
4. The method of claim 1, further comprising, prior to generating the single-mode trajectory graph for each mode from the vehicle spatiotemporal trajectory data, respectively:
and storing the space-time track data of the vehicle into a database, and constructing an index based on the fields.
5. The method of claim 1, wherein the vehicle space-time trajectory data comprises IMSI data, license plate data, and ETC data.
6. The method for merging space-time trajectories of multi-modal vehicles according to claim 5, wherein the acquiring the plurality of vehicle space-time trajectory data of different modalities includes:
acquiring the IMSI data through a code detection device;
obtaining license plate image data through a license plate camera, and inputting the license plate image data into an optical character recognition system to obtain the license plate data;
and acquiring the ETC data through ETC equipment.
7. A multi-modal vehicle space-time trajectory fusion device, comprising:
the vehicle track data acquisition module is used for acquiring various vehicle space-time track data of different modes;
the single-mode track diagram generation module is used for respectively generating single-mode track diagrams of all modes according to the vehicle space-time track data;
the track representation vector generation module is used for respectively inputting each single-mode track graph into the trained graph convolution neural network model so as to generate single-mode track representation vectors of each mode;
the vehicle space-time track fusion module is used for calculating the similarity between different modal tracks according to the single-modal track expression vector and fusing the tracks meeting the similarity threshold;
the single-mode track diagram generating module comprises:
the global space topological graph construction unit is used for constructing a global space topological graph according to the communication relation of the acquisition equipment of the space-time track data of the vehicle on the road network;
a track space topological graph generating unit, which is used for generating a track space topological graph on the global space topological graph according to the vehicle space-time track data; nodes in the track space topological graph represent the acquisition equipment, node weights represent the acquisition times of the acquisition equipment, connecting edges represent the communication relation among the acquisition equipment, and connecting edge weights represent the track passing times on the corresponding connecting edges;
the track time topological graph generating unit is used for generating a track time topological graph according to the time law of the vehicle space-time track data; nodes in the track time topological graph represent time intervals, node weights represent acquisition times in corresponding time intervals, and continuous edges represent sequential relation of acquisition in different time intervals;
the track space-time topological graph generating unit is used for generating a track space-time topological graph serving as the single-mode track graph according to the track space topological graph and the track time topological graph; the track space-time topological graph is a double-layer topological graph, the inter-layer continuous edge represents the relation between the acquisition equipment and the time interval, and the inter-layer continuous edge weight represents the acquisition times of the acquisition equipment in the corresponding time interval.
8. A computer device, comprising:
one or more processors;
a memory for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of fusion of multi-modal vehicle spatiotemporal trajectories of any of claims 1-6.
9. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements a method of fusion of space-time trajectories of a multi-modal vehicle as claimed in any one of claims 1-6.
CN202311041161.0A 2023-08-18 2023-08-18 Method, device, equipment and storage medium for fusing space-time trajectories of multi-mode vehicles Active CN116778292B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311041161.0A CN116778292B (en) 2023-08-18 2023-08-18 Method, device, equipment and storage medium for fusing space-time trajectories of multi-mode vehicles

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311041161.0A CN116778292B (en) 2023-08-18 2023-08-18 Method, device, equipment and storage medium for fusing space-time trajectories of multi-mode vehicles

Publications (2)

Publication Number Publication Date
CN116778292A CN116778292A (en) 2023-09-19
CN116778292B true CN116778292B (en) 2023-11-28

Family

ID=87986159

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311041161.0A Active CN116778292B (en) 2023-08-18 2023-08-18 Method, device, equipment and storage medium for fusing space-time trajectories of multi-mode vehicles

Country Status (1)

Country Link
CN (1) CN116778292B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117132002A (en) * 2023-10-26 2023-11-28 深圳前海中电慧安科技有限公司 Multi-mode space-time track prediction method, device, equipment and medium
CN117473337B (en) * 2023-12-25 2024-03-22 中国科学院自动化研究所 Track association method, track association device, track association equipment and storage medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109190056A (en) * 2018-08-22 2019-01-11 深圳先进技术研究院 A kind of track of vehicle method for reconstructing, system and electronic equipment
CN110276947A (en) * 2019-06-05 2019-09-24 中国科学院深圳先进技术研究院 A kind of traffic convergence analysis prediction technique, system and electronic equipment
CN111243277A (en) * 2020-03-09 2020-06-05 山东大学 Commuting vehicle space-time trajectory reconstruction method and system based on license plate recognition data
CN112683286A (en) * 2021-03-16 2021-04-20 季华实验室 Method and system for establishing topological road network map, storage medium and electronic equipment
CN113261010A (en) * 2019-01-10 2021-08-13 华为技术有限公司 Object trajectory-based multi-modal sensor fusion method for cross-domain correspondence
CN115100449A (en) * 2022-08-02 2022-09-23 北京卫星信息工程研究所 Remote sensing data multi-target relevance matching and track generation method and equipment
CN115169588A (en) * 2022-09-06 2022-10-11 深圳前海中电慧安科技有限公司 Electrographic computation space-time trajectory vehicle code correlation method, device, equipment and storage medium
CN115203354A (en) * 2022-09-16 2022-10-18 深圳前海中电慧安科技有限公司 Vehicle code track pre-association method and device, computer equipment and storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11494937B2 (en) * 2018-11-16 2022-11-08 Uatc, Llc Multi-task multi-sensor fusion for three-dimensional object detection
CN114913386A (en) * 2021-01-29 2022-08-16 北京图森智途科技有限公司 Training method of multi-target tracking model and multi-target tracking method

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109190056A (en) * 2018-08-22 2019-01-11 深圳先进技术研究院 A kind of track of vehicle method for reconstructing, system and electronic equipment
CN113261010A (en) * 2019-01-10 2021-08-13 华为技术有限公司 Object trajectory-based multi-modal sensor fusion method for cross-domain correspondence
CN110276947A (en) * 2019-06-05 2019-09-24 中国科学院深圳先进技术研究院 A kind of traffic convergence analysis prediction technique, system and electronic equipment
CN111243277A (en) * 2020-03-09 2020-06-05 山东大学 Commuting vehicle space-time trajectory reconstruction method and system based on license plate recognition data
CN112683286A (en) * 2021-03-16 2021-04-20 季华实验室 Method and system for establishing topological road network map, storage medium and electronic equipment
CN115100449A (en) * 2022-08-02 2022-09-23 北京卫星信息工程研究所 Remote sensing data multi-target relevance matching and track generation method and equipment
CN115169588A (en) * 2022-09-06 2022-10-11 深圳前海中电慧安科技有限公司 Electrographic computation space-time trajectory vehicle code correlation method, device, equipment and storage medium
CN115203354A (en) * 2022-09-16 2022-10-18 深圳前海中电慧安科技有限公司 Vehicle code track pre-association method and device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN116778292A (en) 2023-09-19

Similar Documents

Publication Publication Date Title
CN108921200B (en) Method, apparatus, device and medium for classifying driving scene data
CN116778292B (en) Method, device, equipment and storage medium for fusing space-time trajectories of multi-mode vehicles
Ke et al. A smart, efficient, and reliable parking surveillance system with edge artificial intelligence on IoT devices
Ye et al. Lane detection method based on lane structural analysis and CNNs
Villanueva et al. Crowdsensing smart city parking monitoring
CN112163446B (en) Obstacle detection method and device, electronic equipment and storage medium
Naranjo et al. Floating car data augmentation based on infrastructure sensors and neural networks
CN112770265B (en) Pedestrian identity information acquisition method, system, server and storage medium
Wang et al. Realtime wide-area vehicle trajectory tracking using millimeter-wave radar sensors and the open TJRD TS dataset
CN114637884B (en) Method, device and equipment for matching cable-stayed cable-computed space-time trajectory with road network
Mukheja et al. Smartphone‐based crowdsourcing for position estimation of public transport vehicles
CN115690153A (en) Intelligent agent track prediction method and system
Masmoudi et al. Trajectory analysis for parking lot vacancy detection system
CN113807457A (en) Method, device and equipment for determining road network characterization information and storage medium
CN115203354B (en) Vehicle code track pre-association method and device, computer equipment and storage medium
CN115169588A (en) Electrographic computation space-time trajectory vehicle code correlation method, device, equipment and storage medium
Sharma et al. Deep Learning-Based Object Detection and Classification for Autonomous Vehicles in Different Weather Scenarios of Quebec, Canada
CN115576990A (en) Method, device, equipment and medium for evaluating visual truth value data and perception data
CN115062713A (en) GCN-GRU-based open-pit mine truck stay area activity identification method
Yang et al. Numerical calibration method for vehicle velocity data from electronic registration identification of motor vehicles based on mobile edge computing and particle swarm optimization neural network
CN113762043A (en) Abnormal track identification method and device
CN117726883B (en) Regional population analysis method, device, equipment and storage medium
CN112052338A (en) Method, system and storage medium for constructing knowledge graph
Gómez et al. WiFi-based Localization for Fail-Aware Autonomous Driving in Urban Scenarios
Wu et al. Privacy-preserving trajectory classification of driving trip data based on pattern discovery techniques

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant