CN113742536B - Track stream data-oriented continuous range query method and system - Google Patents

Track stream data-oriented continuous range query method and system Download PDF

Info

Publication number
CN113742536B
CN113742536B CN202111091041.2A CN202111091041A CN113742536B CN 113742536 B CN113742536 B CN 113742536B CN 202111091041 A CN202111091041 A CN 202111091041A CN 113742536 B CN113742536 B CN 113742536B
Authority
CN
China
Prior art keywords
query
grid
track
track point
new
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111091041.2A
Other languages
Chinese (zh)
Other versions
CN113742536A (en
Inventor
周思晗
李瑞远
姚斌
张东霞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiaotong University
Original Assignee
Shanghai Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jiaotong University filed Critical Shanghai Jiaotong University
Priority to CN202111091041.2A priority Critical patent/CN113742536B/en
Publication of CN113742536A publication Critical patent/CN113742536A/en
Application granted granted Critical
Publication of CN113742536B publication Critical patent/CN113742536B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/901Indexing; Data structures therefor; Storage structures
    • G06F16/9024Graphs; Linked lists
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/903Querying
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/907Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/909Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Library & Information Science (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a track stream data-oriented continuous range query method and a track stream data-oriented continuous range query system. The invention improves the index module, can ensure that the track points are quickly inserted according to the space position and time sequence, and supports the ID-time range query and the space range query of the track. Then, the query module is improved, a query mechanism combining internal memory and external memory is adopted, and the number of space intersection judgment and the number of data to be retrieved are reduced to the greatest extent. Furthermore, the index module and the query module are supplemented, so that the accuracy of the query can be ensured under the condition of query change. And finally, the method also comprises a memory cleaning module, when the number of inquiry times and the number of concurrent inquiry are increased, the result set is reduced, and the memory burden is lightened. The invention realizes time range query and space range query simultaneously under the condition of ensuring real-time data update, and improves the efficiency of continuous range query.

Description

Track stream data-oriented continuous range query method and system
Technical Field
The invention relates to a continuous range query technology, in particular to a track stream data-oriented continuous range query technology
Background
With the development of global positioning systems and the popularization of mobile devices, mobile objects are continuously generating a large amount of stream data at a high speed. The processing of streaming data is also becoming an emerging technical hotspot, attracting great attention.
Currently, there have been some studies on query stream data.
Real-time queries against microblog data and moving objects are one of them. The index structure of the query is a grid or pyramid structure. Because the data to be processed is of a point type, the two structures involve merging and splitting of nodes when the data is inserted and deleted, and the whole structure can be possibly adjusted, so that the updating is difficult, the efficiency of point-by-point processing is too low, and the efficiency of data insertion needs to be ensured by using a batch updating mode. But batch update indexing does not guarantee that the latest data appears in the query result set in a timely manner. Therefore, the reliability of the query result is insufficient. In addition, the data of the point type is data including only one point, such as microblog data with geographic position and current position data of a mobile object. The data of the point type is free of history data with respect to the data of the line type, such as a track with a sequence of points. Thus, the query cannot support the scenario of a joint query of real-time and historical data.
The joint query of real-time and historical data is realized by partitioning and calculating the data based on a distributed computing framework. The query partitions the received real-time data according to the attributes, and combines the partitioned real-time data with the historical data in the partitions to realize the joint query of the received real-time data and the historical data. Of this type of query, there are two methods that are of interest. One is that WATERWHEEL provides a data partitioning mode, divides a two-dimensional space formed by key-time into a plurality of rectangles, establishes a template B+ tree index in a memory according to a key value range in a range represented by each rectangle, realizes quick update of data, and effectively supports inquiry of the key range and the time range. For example, when the key is min and the time is 10 minutes, data with a min ID that arrives within 10 minutes can be queried. And the other is that Dragoon is based on a data sharing mechanism among different RDDs proposed by a distributed processing framework Spark, so that unnecessary data copying of the Spark native mechanism in data updating is avoided, and real-time indexing and query of data are realized on the basis. RDD (Resilient Distributed Dataset), a resilient distributed dataset, is a data abstraction in spark that represents a collection of immutable, partitionable, parallel-computable elements inside. RDD classifies operations into two categories: conversion and action. Each conversion of RDDs generates a new RDD, so that a front-back dependency relationship similar to a pipeline is formed between RDDs. No matter how many conversion operations are performed, RDD does not actually perform the operation, which is triggered only when an action operation is performed. This is the lazy call mechanism of Spark, which is the basis for efficient computation of Spark. Because of lazy execution, spark can only run in memory more effectively, so that an efficient shared memory mechanism avoids a large number of intermediate results, thereby avoiding the performance consumption caused by disk writing and writing out.
The two methods for combined query of real-time and historical data can be well applied in a certain scene, but have respective limitations. WATERWHEEL does not efficiently support spatial range queries, but can only handle queries based on ID and time attributes. The Dragoon solves the problems of real-time data updating and common single query, and the real-time data updating is still carried out by adopting a batch index updating mode, so that the latest data cannot be timely displayed in a query result set, and in addition, the efficiency is still not improved during continuous query.
Disclosure of Invention
In order to realize time range query and space range query simultaneously and improve the efficiency of continuous range query under the condition of ensuring real-time data update, the invention provides a novel continuous range query technology which comprises a position information sending module, an index module and a query module. The invention improves the index module, can ensure that the track points are quickly inserted according to the space position and time sequence, and supports the ID-time range query and the space range query of the track. Then, the invention improves the query module, adopts a query mechanism combining internal memory and external memory, reduces the number of times of space intersection judgment and the number of times of data to be searched to the greatest extent, namely fully utilizes limited memory resources, improves the efficiency of continuous-range query, and ensures the accuracy of results. On the basis, the invention supplements the index module and the query module, so that the accuracy of query can be ensured under the condition of query change. Further, the technology of the invention also comprises a memory cleaning module. When the number of queries and the number of concurrent queries increases, the huge result set increases the memory burden. Therefore, the reasonable memory cleaning mechanism provided by the memory cleaning module can efficiently support a large number of queries, and meanwhile, the accuracy of the query results is ensured.
In order to more clearly illustrate the technical solution of the present invention, the following relevant definitions are given:
Define 1 track (Trajectory). The trajectory Tr is a time-ordered series of trajectory points produced by the moving object O j (1. Ltoreq.j. Ltoreq.m) in the geospatial, expressed as Wherein, a track point/>Is the ith GNSS record generated by the moving object O j, expressed as a binary/>For the time when this GNSS record was generated,/>To generate this GNSS record, the coordinates of the geographic location where the mobile object is located are denoted (lat, lng). In one track, any subset with the track point number larger than 2 is called a sub-track, and is expressed as/>For convenience, the sub-tracks are also referred to as tracks hereinafter.
GNSS, a satellite navigation system (Global Navigation SATELLITE SYSTEM), is a satellite system for spatial positioning that covers the world, allows a small electronic receiver to determine its location (longitude, latitude, and altitude), and the time signal transmitted along the line of sight via satellite broadcast is accurate to within 10 meters. The precise time and position calculated by the receiver can be used as the reference of scientific experiments. The world's major satellite navigation systems are the global positioning system GPS in the united states, the GLONASS in russia, the beidou satellite navigation positioning system in china, and the GALILEO (GALILEO) positioning system in the european union.
Define 2 track streams (Trajectory Streams). The track stream TS is generated by a moving object O j (1. Ltoreq.j.ltoreq.m) in the geographic space, and contains a series of GNSS records, denoted as tracksWherein, a track point/>Is the ith GNSS record generated by the moving object O j. Unlike tracks, track stream data has no boundaries and is infinite.
Defining 3 space intersection judgment. Given two spatial geometry objects a and B, it is calculated whether there is an intersection of the spatial geometries of a and B. If it isA and B spatially intersect.
A continuous range query of 4 track streams is defined (Continuous Range Query on Trajectory Streams). Given a track stream TS, and a spatial range R, the continuous range query of the track stream is repeatedly executed at fixed time t, and the track tr i continuously meets the following two conditions:
tr i contains the latest one of the trace points r latest generated by its associated object;
all track points in tr i are in track stream TS, and the spatial positions of the track points are in spatial range R.
Namely:
Define 5 Query (Query). Query Q consists of three attributes, a unique identifier Q id, a geometric space range Q range of the query, and a time Q t at which the query was initiated, respectively. Represented as a triplet (q id,qrange,qt).
A query buffer, a set of query buffers, is defined 6 (Query Buffers Set). Given the buffer distance ε, a buffer analysis is performed on all the geometries in the query set to obtain a set of buffered results, as shown in FIG. 1.
Define 7 ID-time range Query (ID-Temporal Query). Given the track set TSet, the parameters of the ID-time range query are the unique identification ID of the moving object and the time range tr= [ t min,tmax ], all tracks TR i e TSet satisfying the following two conditions in the TSet are returned:
tri.id=id;
All track points in TR i are generated within the time range TR.
Namely:
in a first aspect, an embodiment of the present invention provides a method for querying a continuous range of track stream data, where the method includes:
The mobile object obtains the current position information through the satellite navigation system GNSS, and continuously transmits track point data to the inquiry system, wherein the track point is a GNSS record generated by the mobile object, and the GNSS record comprises time generated by the GNSS record, namely, the time attribute and the coordinate of the geographic position of the mobile object, namely, the space attribute.
And indexing the track point, and initiating a query Q.
The attributes of the query Q include: unique identification, query space scope, and time of initiation of the query.
The query Q includes a spatial range query based on spatial attributes of the mesh and a temporal range query based on temporal attributes of the doubly linked list. The index includes an index structure and an index update.
The index structure adopts a structure of combining a grid and a doubly linked list, the spatial attribute of the track point is managed through the grid, and the time attribute of the track point is managed through the doubly linked list.
The grid is obtained by dividing the query space range, the grid preset point coordinates are used as the unique identification of the grid, namely the grid coordinates, all grids in the query space range are used as grid sets, and the preset grid coordinates are used as grid set points.
Each grid has a grid pointer pointing to the latest track point within the grid, all track points within the grid being ordered in reverse order of arrival order.
The track is stored by using the doubly-linked list, each node of the doubly-linked list is a track point, a first node pointer of the node points to a previous track point of the track point in the track, and a second node pointer and a third node pointer of the node point to two track points adjacent to the track point in the grid where the track point is located in the arrival sequence of the track point.
When the space attribute of the new track point falls in the query space range of the query Q, the grid where the new track point is located is calculated according to the relative position of the new track point and the grid set origin, and whether the new track point is the latest track point which arrives in sequence is judged according to the time attribute of the new track point.
When the new track point is the latest track point which arrives in sequence, the new track point is added to the tail end of a double-linked list of a grid where the new track point is located, a first node pointer, a second node pointer and a third node pointer of a node corresponding to the relevant track point of the double-linked list are updated, and a grid pointer of the grid where the new track point is located is updated.
When the new track point is not the latest track point which arrives in sequence, the corresponding insertion position is found in the doubly linked list according to the time attribute of the new track point, the first node pointer, the second node pointer and the third node pointer of the node corresponding to the new track point of the doubly linked list are updated, and the second node pointer and the third node pointer of the node corresponding to the track point adjacent to the time attribute of the new track point in the grid where the new track point is located are updated.
In some embodiments, the method for querying the continuous range of the track-oriented stream data is characterized by comprising the following steps:
The query spatial range is a query buffer, and the query buffer is a spatial range in which the boundary of the query spatial range extends epsilon meters outwards.
In some embodiments, the method for continuous range querying of track-oriented stream data further comprises:
The hash map stores the ID of the query object, which is a moving object whose result of spatial intersection judgment of the trajectory point and the query space range of the query Q is not null.
The hash map stores a first object pointer of the query object that points to an oldest trace point of the query object in the query space.
The hash map stores a second object pointer of the query object that points to a most recent trace point of the query object within the query space.
In some embodiments, the method for continuous range querying of track-oriented stream data further comprises:
the hash map stores the entry timestamp of the query object, i.e., the time attribute of the trace point at which the query object entered the query space range.
The second execution of query Q begins, and the entry timestamp is retrieved from the latest track point to the historical data direction along the doubly linked list, namely the query result is returned.
In some embodiments, the method for continuous range querying of track-oriented stream data further comprises:
When the query spatial range becomes a new query spatial range, a query spatial range variation difference set is calculated.
When the new query space range falls within the query space range, calculating grids in the change difference set of the query space range, namely, reducing grids, deleting the reducing grids from the grid set, deleting the doubly linked list of the latest track point in the reducing grids, updating a first node pointer, a second node pointer and a third node pointer of related nodes of the doubly linked list of the grids in the new query space range, updating an entry time stamp of a new query object, namely, a moving object in the new query space range.
When the new query space range is not in the query space range, calculating the grids in the query space range variation difference set, namely a new grid, to obtain a new grid set and a new grid set origin, and merging the new grid set with the grid set according to the position relationship between the new grid set origin and the grid set origin to obtain a grid set of the new query space range; initiating a new query Q1 by taking the ID of the query object and the spatial range of the newly added grid set as attributes, and indexing track points in a query result; initiating a new query Q2, querying a mobile object of the current position in the newly added grid set to obtain a new query object, initiating a new query Q3 by using the ID of the new query object and the new query space range as attributes, and indexing the track points in the query result.
In some embodiments, the method for querying the continuous range of the track-oriented stream data is characterized by comprising the following steps:
The query system works in a mode of combining internal and external memories, and the working mode of combining internal and external memories comprises the following steps:
when the query Q is executed for the first time, track points of the moving object are read from the external memory, and space intersection judgment is carried out on the track points and the query space range.
The query space range of the query Q is stored in memory.
Only track points of the query object of the query Q in the query space range are indexed, and the index is stored in the memory.
When the query object leaves the query space, the index of the track point of the query object is deleted.
In some embodiments, the method for continuous range query of track-oriented stream data further includes a memory clean, where the memory clean includes:
And clearing indexes of track points of the static object in the query space range, wherein the static object is a query object of which the query system does not receive a new track point in a static time, and the static time can be preset in the query system.
The index of invalid track points, which are track points that do not appear in the query result of query Q, is cleared.
In some embodiments, the memory cleaning in the track-stream data-oriented continuous-range query method further includes:
when the query Q has more than two indexes of invalid query track points, the invalid query track points are track points only appearing in the invalid query, the invalid query is the query Q which does not initiate execution in the invalid query time, and the invalid query time can be preset in the query system.
And clearing indexes of low-efficiency high-sampling track points, wherein the low-efficiency high-sampling track points are track points of low-efficiency objects except the latest track point, the low-efficiency objects are moving objects with the sampling frequency larger than the preset sampling frequency only in the query Q with the lowest query frequency, and after the low-efficiency high-sampling track points are cleared, the track points of the low-efficiency objects are read through a memory when the query Q with the lowest query frequency initiates a query.
In a second aspect, an embodiment of the present invention proposes a system for continuous range query for track stream data, where the system includes:
the system comprises a position information sending module, an indexing module and a query module.
The position information sending module is used for obtaining current position information through a satellite navigation system GNSS on a mobile object, and continuously sending track point data to a query system, wherein the track point is a GNSS record generated by the mobile object, and the GNSS record comprises time, namely a time attribute and coordinates, namely a space attribute, of a geographic position where the mobile object is located.
The query module is configured to initiate a query Q, where attributes of the query Q include: unique identification q id, query space range q range, and time q t at which the query was initiated.
The query module comprises a space range query module and a time range query module.
The spatial range query module is used for performing spatial range query according to the spatial attribute of the grid when the query Q is initiated.
And the time range query module is used for performing time range query according to the time attribute of the doubly linked list when the query Q is initiated.
The index module is used for indexing the track points and comprises an index structure module and an index updating module.
The index structure module is used for constructing a structure combining a grid and a double-linked list, and comprises a grid module and a linked list module, wherein the grid module is used for managing the spatial attribute of the track point, and the linked list module is used for managing the time attribute of the track point.
The grid module comprises a grid dividing module and a grid pointer module.
The grid dividing module is used for dividing the query space range into grids to obtain the grids, taking grid preset point coordinates as unique identifiers of the grids, namely grid coordinates, taking all grids in the query space range as grid sets, and taking the preset grid coordinates as grid set points.
The grid pointer module is used for managing grid pointers of the grids, each grid is provided with a grid pointer, the grid pointer points to the latest track point in the grid range, and all track points in the grid range are ordered according to the reverse order of the arrival order.
The linked list module is used for storing tracks by using the double linked list, each node of the double linked list is a track point, a first node pointer of the node points to a previous track point of the track point in the track, and a second node pointer and a third node pointer of the node point to two track points adjacent to the track point in the grid where the track point is located in the arrival sequence.
The index updating module is used for updating the index when receiving the new track point and comprises a calculating and judging module, a sequential updating module and an out-of-order updating module.
And the calculating and judging module is used for calculating the grid where the new track point is positioned according to the relative position of the new track point and the grid set origin when the spatial attribute of the new track point falls in the query spatial range of the query Q, and judging whether the new track point is the latest track point which sequentially arrives according to the time attribute of the new track point.
And the sequence updating module is used for adding the new track point to the end of the doubly linked list of the grid where the new track point is located when the new track point is the latest track point which arrives in sequence, updating the first node pointer, the second node pointer and the third node pointer of the nodes corresponding to the relevant track point of the doubly linked list, and updating the grid pointer of the grid where the new track point is located.
And the out-of-order updating module is used for finding a corresponding insertion position in the doubly-linked list according to the time attribute of the new track point when the new track point is not the latest track point which arrives in sequence, updating a first node pointer, a second node pointer and a third node pointer of a node corresponding to the new track point of the doubly-linked list, and updating a second node pointer and a third node pointer of a node corresponding to the track point adjacent to the time attribute of the new track point in a grid where the new track point is located.
In some embodiments, the system for continuous range query of track-oriented stream data is characterized in that:
The query spatial range is a query buffer, and the query buffer is a spatial range in which the boundary of the query spatial range extends epsilon meters outwards.
In some embodiments, the system for continuous range querying of track-oriented stream data further comprises a hash-map storage module comprising:
and querying an object ID storage module, a first object pointer storage module and a second object pointer storage module.
The query object ID storage module is used for hash mapping and storing the ID of a query object, wherein the query object is a mobile object with a non-empty result of space intersection judgment between the track point and the query space range of the query Q.
The first object pointer storage module is used for storing a first object pointer of the query object in a hash map mode, and the first object pointer points to the oldest track point of the query object in the query space range.
The second object pointer storage module is used for hash mapping and storing a second object pointer of the query object, wherein the second object pointer points to the latest track point of the query object in the query space range.
In some embodiments, the system for continuous range querying of track-oriented stream data further comprises a historical data retrieval module comprising:
The time stamp storage module and the result return module.
The time stamp storage module is used for storing the entry time stamp of the query object, namely the time attribute of the track point when the query object enters the query space range.
The result returning module is used for searching the entry time stamp from the latest track point to the historical data direction along the doubly linked list from the second execution of the query Q, namely returning the query result.
In some embodiments, the system for continuous-range querying of track-oriented stream data further comprises a spatial-range variation module comprising:
The difference set calculation module, the shrinkage change module and the expansion change module.
The difference set calculation module is used for calculating a change difference set of the query space range when the query space range is changed into a new query space range.
The narrowing change module is configured to calculate a grid in which the change difference of the query space range is concentrated, that is, a decreasing grid, delete the decreasing grid from the grid set, delete a doubly linked list of the latest track point in the decreasing grid, update a first node pointer, a second node pointer, a third node pointer of a related node of the doubly linked list of the grid in the new query space range, update the entry timestamp of the new query object, that is, a moving object in the new query space range, when the new query space range falls in the query space range.
The expansion change module is used for calculating grids in the change difference set of the query space range, namely a new grid to obtain a new grid set and a new grid set origin, and merging the new grid set with the grid set according to the position relationship between the new grid set origin and the grid set origin to obtain the grid set of the new query space range when the new query space range is not in the query space range; initiating a new query Q1 by taking the ID of the query object and the spatial range of the newly added grid set as attributes, and indexing track points in a query result; initiating a new query Q2, querying a mobile object of the current position in the newly added grid set to obtain a new query object, initiating a new query Q3 by using the ID of the new query object and the new query space range as attributes, and indexing the track points in the query result.
In some embodiments, the system for continuous range query of track-oriented stream data is characterized in that:
The query system works in a manner that combines internal and external memory. The working mode of the internal and external memory combination comprises the following steps:
When the query Q is executed for the first time, the track point of the moving object is read from the external memory, and space intersection judgment is carried out with the query space range.
The query space range of the query Q is stored in memory.
Only track points of the query object of the query Q in the query space range are indexed, and the index is stored in the memory.
When the query object leaves the query space, the index of the track point of the query object is deleted.
In some embodiments, the system for continuous range query of track-oriented stream data further includes a memory cleaning module, where the memory cleaning module includes: and the static object clearing module and the invalid track point clearing module are used for clearing.
The static object clearing module is used for clearing indexes of track points of static objects in the query space range, wherein the static objects are query objects of which the query system does not receive new track points in static time, and the static time can be preset in the query system.
The invalid track point clearing module is used for clearing indexes of invalid track points, wherein the invalid track points are track points which are not present in the query result of the query Q.
In some embodiments, the memory cleaning module of the system facing the continuous range query of the track stream data further includes:
And the invalid query track point module and the low-efficiency high-sampling track point module are cleared.
The invalid query track point clearing module is used for clearing indexes of invalid query track points when more than two queries Q exist, wherein the invalid query track points are track points only appearing in invalid queries, the invalid queries are queries Q which do not initiate execution within invalid query time, and the invalid query time can be preset in a query system.
And the low-efficiency and high-sampling track point clearing module is used for clearing indexes of low-efficiency and high-sampling track points when more than two queries Q exist, wherein the low-efficiency and high-sampling track points are track points of low-efficiency objects except the latest track point, the low-efficiency objects are moving objects with sampling frequency larger than a preset sampling frequency only appearing in the query Q with the lowest query frequency, and after the low-efficiency and high-sampling track points are cleared, the track points of the low-efficiency objects are read through a memory when the query Q with the lowest query frequency initiates a query.
When the index method combining the grid and the doubly linked list provided by the invention is comprehensively used, and the query mechanism and the memory cleaning mechanism combining the internal memory and the external memory are used, and the number of space intersection judgment and the number of data to be retrieved are reduced to the greatest extent, the functions of the query system provided by some embodiments at the server side are as shown in fig. 2, and the query system can be divided into three parts: data insertion, query execution, and memory clean-up. White open arrows in fig. 2 indicate the flow of data during data insertion and query execution, and black solid arrows indicate the flow of query results.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a diagram of a query buffer of a method for continuous-range query of track-oriented stream data according to an embodiment of the present invention
Fig. 2 is a functional framework diagram of a server end according to a specific embodiment of a track-stream data-oriented continuous range query method of the present invention
Fig. 3 is a schematic diagram of an index structure of an embodiment of a method for continuous range query of track-oriented stream data according to the present invention
FIG. 4 is a schematic diagram illustrating insertion of new track points into a grid according to an embodiment of the method for continuous-range query of track-stream data of the present invention
FIG. 5 is an exemplary track-to-query range buffer relationship for a continuous-range query method for track-stream data according to one embodiment of the present invention
Fig. 6 is a flowchart illustrating an example of grid updating according to an embodiment of the method for continuous-range query of track-oriented stream data of the present invention
FIG. 7 is an experimental dataset of comparative experiments for one embodiment of a method of continuous-range query for track-stream data according to the present invention
FIG. 8 is a graph showing the initial query efficiency for different query space ranges of the comparison experiment for a continuous-range query for track-stream data according to one embodiment of the present invention
FIG. 9 is a graph showing the comparison of the subsequent query efficiency at different query space ranges for the comparison experiment of one embodiment of the method for continuous-range query of track-oriented stream data of the present invention
FIG. 10 is a graph showing index times for different object numbers for a comparison experiment of a method for continuous-range query of track-oriented stream data according to an embodiment of the present invention
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It will be appreciated by those of ordinary skill in the art that the embodiments described are some, but not all, of the embodiments of the application. Based on the embodiments in the present application, any suitable modification or variation may be made by those skilled in the art to obtain all other embodiments.
The technical scheme of the invention will be described below by taking monitoring and tracking of special vehicles in cities as an example.
In a first aspect, an embodiment of the present invention provides a method for querying a continuous range of track stream data, where the method includes the following steps:
The mobile object obtains the current position information through the satellite navigation system GNSS, and continuously transmits track point data to the inquiry system, wherein the track point is a GNSS record generated by the mobile object, and the GNSS record comprises time generated by the GNSS record, namely, the time attribute and the coordinate of the geographic position of the mobile object, namely, the space attribute.
And indexing the track point, and initiating a query Q.
The attributes of the query Q include: unique identification q id, query space range q range, and time q t at which the query was initiated.
The query Q includes a spatial range query based on spatial attributes of the mesh and a temporal range query based on temporal attributes of the doubly linked list. The index includes an index structure and an index update.
The index structure adopts a structure of combining a grid and a doubly linked list, the spatial attribute of the track point is managed through the grid, and the time attribute of the track point is managed through the doubly linked list.
The grid is obtained by dividing the query space range, the grid preset point coordinates are used as the unique identification of the grid, namely the grid coordinates, all grids in the query space range are used as grid sets, and the preset grid coordinates are used as grid set points.
Each grid has a grid pointer pointing to the latest track point within the grid, all track points within the grid being ordered in reverse order of arrival order.
The track is stored by using the doubly-linked list, each node of the doubly-linked list is a track point, a first node pointer of the node points to a previous track point of the track point in the track, and a second node pointer and a third node pointer of the node point to two track points adjacent to the track point in the grid where the track point is located in the arrival sequence of the track point.
When the space attribute of the new track point falls in the query space range of the query Q, the grid where the new track point is located is calculated according to the relative position of the new track point and the grid set origin, and whether the new track point is the latest track point which arrives in sequence is judged according to the time attribute of the new track point.
When the new track point is the latest track point which arrives in sequence, the new track point is added to the tail end of a double-linked list of a grid where the new track point is located, a first node pointer, a second node pointer and a third node pointer of a node corresponding to the relevant track point of the double-linked list are updated, and a grid pointer of the grid where the new track point is located is updated.
When the new track point is not the latest track point which arrives in sequence, the corresponding insertion position is found in the doubly linked list according to the time attribute of the new track point, the first node pointer, the second node pointer and the third node pointer of the node corresponding to the new track point of the doubly linked list are updated, and the second node pointer and the third node pointer of the node corresponding to the track point adjacent to the time attribute of the new track point in the grid where the new track point is located are updated.
In this embodiment, the moving object is a special vehicle in the city, such as a dangerous chemical vehicle, a muck vehicle, and the like. These special vehicles need to be equipped with a GNSS positioning system, and automatically upload a GNSS record to the inquiry system at regular intervals. The GNSS record, i.e. a track point of the moving object, includes the time, i.e. the time attribute, generated by the GNSS record and the coordinates, i.e. the spatial attribute, of the geographic location where the moving object is located. Currently, the GPS system is more commonly used. If one dangerous chemical vehicle is a moving object a, the query system can initiate a query Q after receiving the track point of the moving object a and indexing the track point. The attributes of the query Q include: unique identification q id, query space range q range, and time q t at which the query was initiated. Unique identification Q id distinguishes between different queries Q. The query space range is determined according to the management requirement of the city, and the key monitoring area of the city can be listed as the query space range. The time of initiating a query represents the time of this query, and one query may initiate multiple queries sequentially at different times. The query Q includes a spatial range query based on spatial attributes of the mesh and a temporal range query based on temporal attributes of the doubly linked list. When the query Q is executed for the first time, an index is established for track points in the range of the query space, and meanwhile, the setting of the query Q is saved in the query system.
The index structure is shown in fig. 3. The index structure adopts a structure of combining a grid and a doubly linked list, the spatial attribute of the track point is managed through the grid, and the time attribute of the track point is managed through the doubly linked list. The grid is obtained by dividing the query space range into grids. g (0, 0) in the top row of fig. 3, g (1, 0) … … represents the individual grids divided. Each grid has a grid pointer pointing to the latest track point within the grid, as in fig. 3 grid g (0, 0) points to track point A6 and grid g (1, 0) points to track point A5. A5 and A6 are the 5 th and 6 th track points of the moving object a, respectively. All the track points in each grid range are ordered according to the reverse order of the arrival order, as shown in fig. 3, the track points received by the grid g (1, 1) in sequence are B5, A3, B4 and B3. The track is stored by using a doubly linked list, each node of the doubly linked list is a track point, a first node pointer of the node points to a previous track point of the track point in the track, and a second node pointer and a third node pointer of the node point to two track points adjacent to the track point in the grid where the track point is located in the arrival sequence. As shown in fig. 3, A1, A2, A3, A4, A5, A6 are track points of the moving object a that arrive successively, and are also nodes of a doubly linked list storing tracks of the moving object a. The first node pointer of each node points to the previous track point in the track where the track point is located, as shown in fig. 1, A6 points to A5, A5 points to A4, A4 points to A3, A3 points to A2, and A2 points to A1. In the grid g (3, 0), the three track points A1, A2, A4 arrive one after the other, adjacent to each other, so that their second node pointer, third node pointer point to each other.
Each grid has a unique identification, the grid coordinates. The grid coordinates may be determined by the coordinates of a grid preset point, which may be a vertex angle of the grid or its center point, in any case, any point where unique coordinates may be determined. And taking all grids in the query space range as grid sets and taking preset point coordinates as grid set points. The preset point may be a definite point of any definite mesh of the mesh set, such as a vertex of a lower left corner of a leftmost mesh of a lowermost row, i.e., a vertex of a lower left corner of a mesh of the lower left corner of the mesh set.
When the space attribute of the new track point falls in the query space range of the query Q, the grid where the new track point is located is calculated according to the relative position of the new track point and the grid set origin, and whether the new track point is the latest track point which arrives in sequence is judged according to the time attribute of the new track point.
When the new track point is the latest track point that arrives sequentially, as shown in fig. 4, the grid g (3, 0) has two track points A1 and A2, after receiving the latest track point A4 that arrives sequentially, the new track point A4 is added to the end of the doubly linked list of the grid where the new track point is located, the first node pointer, the second node pointer and the third node pointer of the node corresponding to the relevant track point of the doubly linked list are updated, and the grid pointer of the grid where the new track point is located is updated.
When the new track point is not the latest track point which arrives in sequence, the corresponding insertion position is found in the doubly linked list according to the time attribute of the new track point, the first node pointer, the second node pointer and the third node pointer of the node corresponding to the new track point of the doubly linked list are updated, and the second node pointer and the third node pointer of the node corresponding to the track point adjacent to the time attribute of the new track point in the grid where the new track point is located are updated. The track definition mentioned above refers to a track consisting of time-ordered GNSS recordings. In reality, however, GNSS records do not necessarily arrive in the order in which they were generated due to network delays and failure of the GNSS positioning apparatus. The prior art assumes that the received GNSS records are ordered data, and does not give an accurate and efficient solution to the problem of data misordering. For example, two track points are respectively generated at 10 points and 30 points, and a track point A generated at 10 points is received later than a track point B generated at 10 points and 30 points due to network delay and the like. The prior art disregards the actual generation time of these two trace points, which may cause errors in the final query results. According to the technical scheme of the embodiment, the storage position of the query system is found in the index after the additional processing is carried out on the query system A, so that the query system can return the correct query result when the query result is returned next time.
In monitoring and tracking of special vehicles in a city, the query Q uses a key monitoring area in the city as a query space range. The query system initiates execution of the query Q according to a certain frequency, and the query system analyzes the received position information of the track point and judges whether the position information is in the query space range or not when the query Q is executed each time. The query system establishes an index for the track points falling in the query space range, and gives real-time position information and historical position information in the query space range according to the time attribute and the space attribute of the track points. According to the real-time position information, the special vehicle can be conveniently intercepted. According to the historical position information, the driving route of the vehicle entering the key monitoring area can be known, so that the safety check of the passing area of the vehicle is realized. The query method is practical application of the query method in the scene of combined query of real-time and historical data.
In some embodiments, the method for querying the continuous range of the track-oriented stream data is characterized by comprising the following steps:
The query spatial range is a query buffer, and the query buffer is a spatial range in which the boundary of the query spatial range extends epsilon meters outwards.
The present embodiment is proposed based on the following assumption: objects currently in the buffer range ε meters of query Q are far more likely to enter the query range than objects outside the buffer range. Taking fig. 5 as an example (solid point is the current position point of the object, and open point is the track point at the historical moment): objects a and B are currently located in the query buffer of query Q, which requires less time to enter the query space range than object C, and objects a and B are more likely to enter the query range at the next time than object C. Therefore, the track points of the moving objects A and B in the query buffer area are indexed, so that the query efficiency can be effectively improved.
In the monitoring and tracking of special vehicles in a city, a key monitoring area in the city is taken as a query space range, the range can be expanded outwards, for example, 100 meters is expanded, the special vehicles within 100 meters from the key monitoring area are brought into the monitoring range, and when the track point of the vehicle is detected to fall into the key monitoring area, the quick response can be realized at the first time.
In some embodiments, the method for continuous range querying of track-oriented stream data further comprises:
The hash map stores the ID of the query object, which is a moving object whose result of spatial intersection judgment of the trajectory point and the query space range of the query Q is not null.
The hash map stores a first object pointer of the query object that points to an oldest trace point of the query object in the query space.
The hash map stores a second object pointer of the query object that points to a most recent trace point of the query object within the query space.
In this embodiment, the hash map stores the ID of the query object, as shown in the bottom row of fig. 3, A, B, C being the ID of the query object. Each query object has a first object pointer pointing to the oldest track point and a second object pointer pointing to the newest track point. As shown in fig. 3, the first object pointer of query object a points to track point A1 and the second object pointer points to track point A6. The technical scheme of the embodiment supports parallel ID-time range inquiry, namely, inquiry of a plurality of ID-time ranges can be performed in parallel.
In the monitoring and tracking of special vehicles in cities, the IDs of the special vehicles entering the key monitoring area, the latest track points and the oldest track points in the key monitoring area are stored through the hash map, so that the position information in a specified time range can be inquired for a plurality of arbitrarily selected vehicles in the special vehicles entering the key monitoring area.
In some embodiments, the method for continuous range querying of track-oriented stream data further comprises:
the hash map stores the entry timestamp of the query object, i.e., the time attribute of the trace point at which the query object entered the query space range.
The second execution of query Q begins, and the entry timestamp is retrieved from the latest track point to the historical data direction along the doubly linked list, namely the query result is returned.
The proposal in this embodiment is based on the following insight: the results of successive queries have relevance, and the historical track that has been returned is always a subset of the results of subsequent queries until the object leaves the query space of the query. According to this feature, after query Q is first executed, its query space range is saved in the query system. Subsequent execution of the query is converted to an ID-time range query without requiring time-consuming spatial intersection decisions. In this embodiment, after query Q is first executed, the IDs of all objects it hits are stored using a hash map, along with the timestamp of when the objects entered the query. The subsequent execution need only retrieve from the latest trace point along the linked list to the specified timestamp in the direction of the historical data. In a specific operation, the search in the direction of the historical data can be set until the first data which is not in the returned result is accessed, and the boundary of the returned data is crossed, namely the query result is returned. Each query only needs to carry out space intersection judgment when being executed for the first time, the subsequent execution is time-digital comparison with constant complexity, the data volume accessed by the subsequent execution is at most 1 more than the returned result number, the query efficiency is improved to the greatest extent, and the requirements of the stream data query and the tracking scene on low-delay query are met.
In the monitoring and tracking of special vehicles in cities, the time stamp of the special vehicles entering the key monitoring area is stored through the hash map, the ID-space range query of the special vehicles entering the key monitoring area is converted into the ID-time range query, a large number of space intersection judgment is avoided, and the real-time and historical position data of a plurality of arbitrarily selected vehicles are simultaneously and rapidly queried.
In some embodiments, the method for continuous range querying of track-oriented stream data further comprises:
When the query spatial range becomes a new query spatial range, a query spatial range variation difference set is calculated.
If the new query space range falls within the query space range, calculating grids in the change difference set of the query space range, namely, reducing grids, deleting the reducing grids from the grid set, deleting the doubly linked list of the latest track point in the reducing grids, updating a first node pointer, a second node pointer and a third node pointer of related nodes of the doubly linked list of the grids in the new query space range, and updating an entry time stamp of a new query object, namely, a moving object in the new query space range.
If the new query space range is not in the query space range, calculating the grids in the change difference set of the query space range, namely a new grid, to obtain a new grid set and a new grid set origin, and merging the new grid set with the grid set according to the position relationship between the new grid set origin and the grid set origin to obtain a grid set of the new query space range; initiating a new query Q1 by taking the ID of the query object and the spatial range of the newly added grid set as attributes, and indexing track points in a query result; initiating a new query Q2, querying a mobile object of the current position in the newly added grid set to obtain a new query object, initiating a new query Q3 by using the ID of the new query object and the new query space range as attributes, and indexing the track points in the query result.
If the new query space range is reduced by a part and is newly increased by a part compared with the query space range, the reduced part is a reduced grid, the newly increased part is a newly increased grid, and the reduced grid and the newly increased grid are processed according to the parts of the reduced grid and the newly increased grid respectively and then are combined.
In this embodiment, the query space range is changed, and the new query space range may or may not fall within the query space range. In the monitoring and tracking of special vehicles in cities, the reduction or increase of important monitoring areas is represented. For example, because a certain area is being removed, the whole area is closed, and vehicles cannot enter the whole area, the area can be removed from the important monitoring area, and the range of the query space is reduced. Or the original urban and rural junction of a certain area is developed for many years, and then the urban and rural junction becomes an emerging urban business area, and an important monitoring area needs to be brought in, namely the expansion of the query space range.
If the new query space range falls within the query space range, deleting the reduced grid from the grid set after the reduced grid is obtained through calculation, and deleting the doubly linked list of the latest track point in the reduced grid; updating a first node pointer, a second node pointer and a third node pointer of related nodes of a two-way linked list of the grid in the new query space range, so that the pointers no longer point to track points in the reduced grid; the entry time stamp of the new query object, i.e., the moving object within the new query space, is updated.
If the new query space range is not in the query space range, calculating the grids in the query space range variation difference set, namely a new grid, to obtain a new grid set and a new grid set origin, and combining the new grid set and the grid set according to the position relationship between the new grid set origin and the grid set origin to obtain a grid set of the new query space range, as shown in fig. 6. And initiating a new query Q1 by taking the ID of the query object and the spatial range of the new added grid set as attributes, and indexing the track points in the query result to obtain the track points of the query object in the new added grid set. And initiating a new query Q2, and querying the mobile object of the current position in the newly added grid set to obtain a new query object. The current position of the new query object is outside the query space range, and the track points of the new query object are not indexed in the query system. And initiating a new query Q3 by using the ID of the new query object and the new query space range as attributes, and indexing the track points in the query result to obtain the track points of the mobile object with the current position in the newly added grid set. By inquiring Q1, Q2 and Q3, the track points in the new inquiring space range are complemented, and the indexing work of all track points is completed. And, the redundant track points are not additionally processed, so that the workload of the query system is at the minimum level.
The technical scheme of the embodiment solves the problem of deleting or supplementing the track points when the query space range is changed. In the monitoring and tracking of special vehicles in cities, the requirement of adjusting key monitoring areas in city management is met, so that the application of the query system in various scenes is more adaptive.
In some embodiments, the method for querying the continuous range of the track-oriented stream data is characterized by comprising the following steps:
The query system works in a mode of combining internal and external memories, and the working mode of combining internal and external memories comprises the following steps:
when the query Q is executed for the first time, track points of the moving object are read from the external memory, and space intersection judgment is carried out on the track points and the query space range.
The query space range of the query Q is stored in memory.
Only track points of the query object of the query Q in the query space range are indexed, and the index is stored in the memory.
When the query object leaves the query space, the index of the track point of the query object is deleted.
In this embodiment, the query system stores the query space range, track point, etc. that are needed to be used by the query in the memory when the query Q is executed for the first time, and all the executions after the query Q are all performed in the memory, thereby avoiding the performance consumption caused by writing and writing out of the disk. The working efficiency of the memory is far higher than that of the external memory, so that the working efficiency of the query system is greatly improved.
In some embodiments, the method for continuous range query of track-oriented stream data further includes a memory clean, where the memory clean includes:
And clearing indexes of track points of the static object in the query space range, wherein the static object is a query object of which the query system does not receive a new track point in a static time, and the static time can be preset in the query system.
The index of invalid track points, which are track points that do not appear in the query result of query Q, is cleared.
In this embodiment, when the moving object does not upload new track point data within the preset rest time, the query system considers that the moving object is in a rest state, and no longer queries the track points of the moving object. The query system indexes all the track points in the query space range, but if the indexed track points are not found in the query result of the query Q, i.e. no query hits the track point, the track point can be considered as an invalid track point, and then the track point is cleared. The technical scheme of the embodiment solves the problem of memory burden caused by excessive stored track point data when the query times are increased.
In the monitoring and tracking of special vehicles in cities, if the special vehicles entering a key monitoring area are stationary for more than a certain time, such as two hours, the query system can estimate that the special vehicles are intercepted and taken over by related department personnel, so that track points of the special vehicles are removed from the query system, and the burden of the memory of the query system is relieved. In another case, although a specific vehicle is driven into the important monitoring area, the specific vehicle is recorded or requested by the relevant management institution, so that each inquiry does not belong to the inquiry scope. For such special vehicles, the query system will also delete its track points to ease the burden of querying the system memory.
In some embodiments, the memory cleaning in the track-stream data-oriented continuous-range query method further includes:
when the query Q has more than two indexes of invalid query track points, the invalid query track points are track points only appearing in the invalid query, the invalid query is the query Q which does not initiate execution in the invalid query time, and the invalid query time can be preset in the query system.
And clearing indexes of low-efficiency high-sampling track points, wherein the low-efficiency high-sampling track points are track points of low-efficiency objects except the latest track point, the low-efficiency objects are moving objects with the sampling frequency larger than the preset sampling frequency only in the query Q with the lowest query frequency, and after the low-efficiency high-sampling track points are cleared, the track points of the low-efficiency objects are read through a memory when the query Q with the lowest query frequency initiates a query.
In this embodiment, when the query Q does not initiate execution within the preset invalid query time, the query Q is determined as an invalid query, and only the trace points appearing in the query result of the invalid query are the invalid query trace points. The removal of the index of invalid query trace points does not affect the results of the query. Clearing the trace point index of the high sample rate trace hit by the low frequency query will have an impact on the query results, and therefore, it needs to be compensated by reading the trace point from the external memory. The technical scheme of the embodiment solves the problem of memory burden caused by excessive stored track point data when the number of concurrent queries is increased. Although reading data through external memory has some influence on the working efficiency of the query system, the improvement of the working efficiency caused by the reduction of the memory burden is more important as a whole.
In the monitoring and tracking of special vehicles in cities, some special vehicles are prohibited from entering an important monitoring area in a designated period and can enter in a non-designated period. Thus, some queries will not initiate execution during unspecified periods of time, becoming invalid queries. Only the special vehicles appearing in the query results of the invalid query do not need to be continuously monitored, and the indexes of the track points can be deleted. There is also a case where queries for some special vehicles are less frequent than others, but these special vehicles may still set a comparatively high track point transmission frequency for some reason. Such as a sprinkler. The sprinkler in the city is particularly slow to travel and the query for it is also a low frequency query. However, if the frequency of the track points sent by the sprinkler is too high, the use efficiency of the track point data is very low, and more memory resources are occupied. At this time, the query system can self-make up for the defect by adopting a mode of clearing indexes of low-efficiency high-sampling track points, and the memory burden is reduced and the working efficiency is improved under the condition of ensuring the query accuracy.
The trace points of the memory clear in the above two embodiments may also be classified into three levels according to the accessed frequency, as shown in fig. 2:
class i is the trace points that do not appear in the result set. Such as those that have not been updated for more than 5 minutes (the maximum sampling frequency set by the querying system), and those that have been hit only by an invalid query (a query that did not send an execution request within the invalid query time).
Class ii refers to trace points that lie in the query space, but are not hit by any query.
Class iii refers to high sample rate trace points that are hit only by low frequency queries.
The track points of the two levels I and II are directly cleared without affecting the query. And selecting the track points of the III level, namely selecting the query Q with the lowest current unclean request frequency, deleting the history track points of the track which has high sampling rate and is not hit by other high-frequency queries, and only keeping the latest track points. After the III-level track is cleaned, only the latest track point is saved in the follow-up process, and the associated query result is returned through the external storage query.
In a second aspect, an embodiment of the present invention proposes a system for continuous range query for track stream data, where the system includes:
the system comprises a position information sending module, an indexing module and a query module.
The position information sending module is used for obtaining current position information through a satellite navigation system GNSS on a mobile object, and continuously sending track point data to a query system, wherein the track point is a GNSS record generated by the mobile object, and the GNSS record comprises time, namely a time attribute and coordinates, namely a space attribute, of a geographic position where the mobile object is located.
The query module is configured to initiate a query Q, where attributes of the query Q include: unique identification q id, query space range q range, and time q t at which the query was initiated.
The query module comprises a space range query module and a time range query module.
The spatial range query module is used for performing spatial range query according to the spatial attribute of the grid when the query Q is initiated.
And the time range query module is used for performing time range query according to the time attribute of the doubly linked list when the query Q is initiated.
The index module is used for indexing the track points and comprises an index structure module and an index updating module.
The index structure module is used for constructing a structure combining a grid and a double-linked list, and comprises a grid module and a linked list module, wherein the grid module is used for managing the spatial attribute of the track point, and the linked list module is used for managing the time attribute of the track point.
The grid module comprises a grid dividing module and a grid pointer module.
The grid dividing module is used for dividing the query space range into grids to obtain the grids, taking grid preset point coordinates as unique identifiers of the grids, namely grid coordinates, taking all grids in the query space range as grid sets, and taking the preset grid coordinates as grid set points.
The grid pointer module is used for managing grid pointers of the grids, each grid is provided with a grid pointer, the grid pointer points to the latest track point in the grid range, and all track points in the grid range are ordered according to the reverse order of the arrival order.
The linked list module is used for storing tracks by using the double linked list, each node of the double linked list is a track point, a first node pointer of the node points to a previous track point of the track point in the track, and a second node pointer and a third node pointer of the node point to two track points adjacent to the track point in the grid where the track point is located in the arrival sequence.
The index updating module is used for updating the index when receiving the new track point and comprises a calculating and judging module, a sequential updating module and an out-of-order updating module.
And the calculating and judging module is used for calculating the grid where the new track point is positioned according to the relative position of the new track point and the grid set origin when the spatial attribute of the new track point falls in the query spatial range of the query Q, and judging whether the new track point is the latest track point which sequentially arrives according to the time attribute of the new track point.
And the sequence updating module is used for adding the new track point to the end of the doubly linked list of the grid where the new track point is located when the new track point is the latest track point which arrives in sequence, updating the first node pointer, the second node pointer and the third node pointer of the nodes corresponding to the relevant track point of the doubly linked list, and updating the grid pointer of the grid where the new track point is located.
And the out-of-order updating module is used for finding a corresponding insertion position in the doubly-linked list according to the time attribute of the new track point when the new track point is not the latest track point which arrives in sequence, updating a first node pointer, a second node pointer and a third node pointer of a node corresponding to the new track point of the doubly-linked list, and updating a second node pointer and a third node pointer of a node corresponding to the track point adjacent to the time attribute of the new track point in a grid where the new track point is located.
In this embodiment, the moving object is a special vehicle in the city, such as a dangerous chemical vehicle, a muck vehicle, and the like. These special vehicles need to be equipped with a GNSS positioning system, and automatically upload a GNSS record to the inquiry system at regular intervals. The GNSS record, i.e. a track point of the moving object, includes the time, i.e. the time attribute, generated by the GNSS record and the coordinates, i.e. the spatial attribute, of the geographic location where the moving object is located. Currently, the GPS system is more commonly used. If one dangerous chemical vehicle is a moving object a, the query system can initiate a query Q after receiving the track point of the moving object a and indexing the track point. The attributes of the query Q include: unique identification q id, query space range q range, and time q t at which the query was initiated. Unique identification Q id distinguishes between different queries Q. The query space range is determined according to the management requirement of the city, and the key monitoring area of the city can be listed as the query space range. The time of initiating a query represents the time of this query, and one query may initiate multiple queries sequentially at different times. The query Q includes a spatial range query based on spatial attributes of the mesh and a temporal range query based on temporal attributes of the doubly linked list. When the query Q is executed for the first time, an index is established for track points in the range of the query space, and meanwhile, the setting of the query Q is saved in the query system.
The index structure is shown in fig. 3. The index structure adopts a structure of combining a grid and a doubly linked list, the spatial attribute of the track point is managed through the grid, and the time attribute of the track point is managed through the doubly linked list. The grid is obtained by dividing the query space range into grids. g (0, 0) in the top row of fig. 3, g (1, 0) … … represents the individual grids divided. Each grid has a grid pointer pointing to the latest track point within the grid, as in fig. 3 grid g (0, 0) points to track point A6 and grid g (1, 0) points to track point A5. A5 and A6 are the 5 th and 6 th track points of the moving object a, respectively. All the track points in each grid range are ordered according to the reverse order of the arrival order, as shown in fig. 3, the track points received by the grid g (1, 1) in sequence are B5, A3, B4 and B3. The track is stored by using a doubly linked list, each node of the doubly linked list is a track point, a first node pointer of the node points to a previous track point of the track point in the track, and a second node pointer and a third node pointer of the node point to two track points adjacent to the track point in the grid where the track point is located in the arrival sequence. As shown in fig. 3, A1, A2, A3, A4, A5, A6 are track points of the moving object a that arrive successively, and are also nodes of a doubly linked list storing tracks of the moving object a. The first node pointer of each node points to the previous track point in the track where the track point is located, as shown in fig. 1, A6 points to A5, A5 points to A4, A4 points to A3, A3 points to A2, and A2 points to A1. In the grid g (3, 0), the three track points A1, A2, A4 arrive one after the other, adjacent to each other, so that their second node pointer, third node pointer point to each other.
Each grid has a unique identification, the grid coordinates. The grid coordinates may be determined by the coordinates of a grid preset point, which may be a vertex angle of the grid or its center point, in any case, any point where unique coordinates may be determined. And taking all grids in the query space range as grid sets and taking preset point coordinates as grid set points. The preset point may be a definite point of any definite mesh of the mesh set, such as a vertex of a lower left corner of a leftmost mesh of a lowermost row, i.e., a vertex of a lower left corner of a mesh of the lower left corner of the mesh set.
When the space attribute of the new track point falls in the query space range of the query Q, the grid where the new track point is located is calculated according to the relative position of the new track point and the grid set origin, and whether the new track point is the latest track point which arrives in sequence is judged according to the time attribute of the new track point.
When the new track point is the latest track point that arrives sequentially, as shown in fig. 4, the grid g (3, 0) has two track points A1 and A2, after receiving the latest track point A4 that arrives sequentially, the new track point A4 is added to the end of the doubly linked list of the grid where the new track point is located, the first node pointer, the second node pointer and the third node pointer of the node corresponding to the relevant track point of the doubly linked list are updated, and the grid pointer of the grid where the new track point is located is updated.
When the new track point is not the latest track point which arrives in sequence, the corresponding insertion position is found in the doubly linked list according to the time attribute of the new track point, the first node pointer, the second node pointer and the third node pointer of the node corresponding to the new track point of the doubly linked list are updated, and the second node pointer and the third node pointer of the node corresponding to the track point adjacent to the time attribute of the new track point in the grid where the new track point is located are updated. The track definition mentioned above refers to a track consisting of time-ordered GNSS recordings. In reality, however, GNSS records do not necessarily arrive in the order in which they were generated due to network delays and failure of the GNSS positioning apparatus. The prior art assumes that the received GNSS records are ordered data, and does not give an accurate and efficient solution to the problem of data misordering. For example, two track points are respectively generated at 10 points and 30 points, and a track point A generated at 10 points is received later than a track point B generated at 10 points and 30 points due to network delay and the like. The prior art disregards the actual generation time of these two trace points, which may cause errors in the final query results. According to the technical scheme of the embodiment, the storage position of the query system is found in the index after the additional processing is carried out on the query system A, so that the query system can return the correct query result when the query result is returned next time.
In monitoring and tracking of special vehicles in a city, the query Q uses a key monitoring area in the city as a query space range. The query system initiates execution of the query Q according to a certain frequency, and the query system analyzes the received position information of the track point and judges whether the position information is in the query space range or not when the query Q is executed each time. The query system establishes an index for the track points falling in the query space range, and gives real-time position information and historical position information in the query space range according to the time attribute and the space attribute of the track points. According to the real-time position information, the special vehicle can be conveniently intercepted. According to the historical position information, the driving route of the vehicle entering the key monitoring area can be known, so that the safety check of the passing area of the vehicle is realized. The query method is practical application of the query method in the scene of combined query of real-time and historical data.
In some embodiments, the system for continuous range query of track-oriented stream data is characterized in that:
The query spatial range is a query buffer, and the query buffer is a spatial range in which the boundary of the query spatial range extends epsilon meters outwards.
The present embodiment is proposed based on the following assumption: objects currently in the buffer range ε meters of query Q are far more likely to enter the query range than objects outside the buffer range. Taking fig. 5 as an example (solid point is the current position point of the object, and open point is the track point at the historical moment): objects a and B are currently located in the query buffer of query Q, which requires less time to enter the query space range than object C, and objects a and B are more likely to enter the query range at the next time than object C. Therefore, the track points of the moving objects A and B in the query buffer area are indexed, so that the query efficiency can be effectively improved.
In the monitoring and tracking of special vehicles in a city, a key monitoring area in the city is taken as a query space range, the range can be expanded outwards, for example, 100 meters is expanded, the special vehicles within 100 meters from the key monitoring area are brought into the monitoring range, and when the track point of the vehicle is detected to fall into the key monitoring area, the quick response can be realized at the first time.
In some embodiments, the system for continuous range querying of track-oriented stream data further comprises a hash-map storage module comprising:
and querying an object ID storage module, a first object pointer storage module and a second object pointer storage module.
The query object ID storage module is used for hash mapping and storing the ID of a query object, wherein the query object is a mobile object with a non-empty result of space intersection judgment between the track point and the query space range of the query Q.
The first object pointer storage module is used for storing a first object pointer of the query object in a hash map mode, and the first object pointer points to the oldest track point of the query object in the query space range.
The second object pointer storage module is used for hash mapping and storing a second object pointer of the query object, wherein the second object pointer points to the latest track point of the query object in the query space range.
In this embodiment, the hash map stores the ID of the query object, as shown in the bottom row of fig. 3, A, B, C being the ID of the query object. Each query object has a first object pointer pointing to the oldest track point and a second object pointer pointing to the newest track point. As shown in fig. 3, the first object pointer of query object a points to track point A1 and the second object pointer points to track point A6. The technical scheme of the embodiment supports parallel ID-time range inquiry, namely, inquiry of a plurality of ID-time ranges can be performed in parallel.
In the monitoring and tracking of special vehicles in cities, the IDs of the special vehicles entering the key monitoring area, the latest track points and the oldest track points in the key monitoring area are stored through the hash map, so that the position information in a specified time range can be inquired for a plurality of arbitrarily selected vehicles in the special vehicles entering the key monitoring area.
In some embodiments, the system for continuous range querying of track-oriented stream data further comprises a historical data retrieval module comprising:
The time stamp storage module and the result return module.
The time stamp storage module is used for storing the entry time stamp of the query object, namely the time attribute of the track point when the query object enters the query space range.
The result returning module is used for searching the entry time stamp from the latest track point to the historical data direction along the doubly linked list from the second execution of the query Q, namely returning the query result.
The proposal in this embodiment is based on the following insight: the results of successive queries have relevance, and the historical track that has been returned is always a subset of the results of subsequent queries until the object leaves the query space of the query. According to this feature, after query Q is first executed, its query space range is saved in the query system. Subsequent execution of the query is converted to an ID-time range query without requiring time-consuming spatial intersection decisions. In this embodiment, after query Q is first executed, the IDs of all objects it hits are stored using a hash map, along with the timestamp of when the objects entered the query. The subsequent execution need only retrieve from the latest trace point along the linked list to the specified timestamp in the direction of the historical data. In a specific operation, the search in the direction of the historical data can be set until the first data which is not in the returned result is accessed, and the boundary of the returned data is crossed, namely the query result is returned. Each query only needs to carry out space intersection judgment when being executed for the first time, the subsequent execution is time-digital comparison with constant complexity, the data volume accessed by the subsequent execution is at most 1 more than the returned result number, the query efficiency is improved to the greatest extent, and the requirements of the stream data query and the tracking scene on low-delay query are met.
In the monitoring and tracking of special vehicles in cities, the time stamp of the special vehicles entering the key monitoring area is stored through the hash map, the ID-space range query of the special vehicles entering the key monitoring area is converted into the ID-time range query, a large number of space intersection judgment is avoided, and the real-time and historical position data of a plurality of arbitrarily selected vehicles are simultaneously and rapidly queried.
In some embodiments, the system for continuous-range querying of track-oriented stream data further comprises a spatial-range variation module comprising:
The difference set calculation module, the shrinkage change module and the expansion change module.
The difference set calculation module is used for calculating a change difference set of the query space range when the query space range is changed into a new query space range.
The narrowing change module is configured to calculate a grid in which the change difference of the query space range is concentrated, that is, a decreasing grid, delete the decreasing grid from the grid set, delete a doubly linked list of the latest track point in the decreasing grid, update a first node pointer, a second node pointer, a third node pointer of a related node of the doubly linked list of the grid in the new query space range, update the entry timestamp of the new query object, that is, a moving object in the new query space range, when the new query space range falls in the query space range.
The expansion change module is used for calculating grids in the change difference set of the query space range, namely a new grid to obtain a new grid set and a new grid set origin, and merging the new grid set with the grid set according to the position relationship between the new grid set origin and the grid set origin to obtain the grid set of the new query space range when the new query space range is not in the query space range; initiating a new query Q1 by taking the ID of the query object and the spatial range of the newly added grid set as attributes, and indexing track points in a query result; initiating a new query Q2, querying a mobile object of the current position in the newly added grid set to obtain a new query object, initiating a new query Q3 by using the ID of the new query object and the new query space range as attributes, and indexing the track points in the query result.
If the new query space range is reduced by a part and is newly increased by a part compared with the query space range, the reduced part is a reduced grid, the newly increased part is a newly increased grid, and the reduced grid and the newly increased grid are processed according to the parts of the reduced grid and the newly increased grid respectively and then are combined.
In this embodiment, the query space range is changed, and the new query space range may or may not fall within the query space range. In the monitoring and tracking of special vehicles in cities, the reduction or increase of important monitoring areas is represented. For example, because a certain area is being removed, the whole area is closed, and vehicles cannot enter the whole area, the area can be removed from the important monitoring area, and the range of the query space is reduced. Or the original urban and rural junction of a certain area is developed for many years, and then the urban and rural junction becomes an emerging urban business area, and an important monitoring area needs to be brought in, namely the expansion of the query space range.
If the new query space range falls within the query space range, deleting the reduced grid from the grid set after the reduced grid is obtained through calculation, and deleting the doubly linked list of the latest track point in the reduced grid; updating a first node pointer, a second node pointer and a third node pointer of related nodes of a two-way linked list of the grid in the new query space range, so that the pointers no longer point to track points in the reduced grid; the entry time stamp of the new query object, i.e., the moving object within the new query space, is updated.
If the new query space range is not in the query space range, calculating the grids in the query space range variation difference set, namely a new grid, to obtain a new grid set and a new grid set origin, and combining the new grid set and the grid set according to the position relationship between the new grid set origin and the grid set origin to obtain a grid set of the new query space range, as shown in fig. 6. And initiating a new query Q1 by taking the ID of the query object and the spatial range of the new added grid set as attributes, and indexing the track points in the query result to obtain the track points of the query object in the new added grid set. And initiating a new query Q2, and querying the mobile object of the current position in the newly added grid set to obtain a new query object. The current position of the new query object is outside the query space range, and the track points of the new query object are not indexed in the query system. And initiating a new query Q3 by using the ID of the new query object and the new query space range as attributes, and indexing the track points in the query result to obtain the track points of the mobile object with the current position in the newly added grid set. By inquiring Q1, Q2 and Q3, the track points in the new inquiring space range are complemented, and the indexing work of all track points is completed. And, the redundant track points are not additionally processed, so that the workload of the query system is at the minimum level.
The technical scheme of the embodiment solves the problem of deleting or supplementing the track points when the query space range is changed. In the monitoring and tracking of special vehicles in cities, the requirement of adjusting key monitoring areas in city management is met, so that the application of the query system in various scenes is more adaptive.
In some embodiments, the system for continuous range query of track-oriented stream data is characterized in that:
The query system works in a manner that combines internal and external memory. The working mode of the internal and external memory combination comprises the following steps:
When the query Q is executed for the first time, the track point of the moving object is read from the external memory, and space intersection judgment is carried out with the query space range.
The query space range of the query Q is stored in memory.
Only track points of the query object of the query Q in the query space range are indexed, and the index is stored in the memory.
When the query object leaves the query space, the index of the track point of the query object is deleted.
In this embodiment, the query system stores the query space range, track point, etc. that are needed to be used by the query in the memory when the query Q is executed for the first time, and all the executions after the query Q are all performed in the memory, thereby avoiding the performance consumption caused by writing and writing out of the disk. The working efficiency of the memory is far higher than that of the external memory, so that the working efficiency of the query system is greatly improved.
In some embodiments, the system for continuous range query of track-oriented stream data further includes a memory cleaning module, where the memory cleaning module includes: and the static object clearing module and the invalid track point clearing module are used for clearing.
The static object clearing module is used for clearing indexes of track points of static objects in the query space range, wherein the static objects are query objects of which the query system does not receive new track points in static time, and the static time can be preset in the query system.
The invalid track point clearing module is used for clearing indexes of invalid track points, wherein the invalid track points are track points which are not present in the query result of the query Q.
In this embodiment, when the moving object does not upload new track point data within the preset rest time, the query system considers that the moving object is in a rest state, and no longer queries the track points of the moving object. The query system indexes all the track points in the query space range, but if the indexed track points are not found in the query result of the query Q, i.e. no query hits the track point, the track point can be considered as an invalid track point, and then the track point is cleared. The technical scheme of the embodiment solves the problem of memory burden caused by excessive stored track point data when the query times are increased.
In the monitoring and tracking of special vehicles in cities, if the special vehicles entering a key monitoring area are stationary for more than a certain time, such as two hours, the query system can estimate that the special vehicles are intercepted and taken over by related department personnel, so that track points of the special vehicles are removed from the query system, and the burden of the memory of the query system is relieved. In another case, although a specific vehicle is driven into the important monitoring area, the specific vehicle is recorded or requested by the relevant management institution, so that each inquiry does not belong to the inquiry scope. For such special vehicles, the query system will also delete its track points to ease the burden of querying the system memory.
In some embodiments, the memory cleaning module of the system facing the continuous range query of the track stream data further includes:
And the invalid query track point module and the low-efficiency high-sampling track point module are cleared.
The invalid query track point clearing module is used for clearing indexes of invalid query track points when more than two queries Q exist, wherein the invalid query track points are track points only appearing in invalid queries, the invalid queries are queries Q which do not initiate execution within invalid query time, and the invalid query time can be preset in a query system.
And the low-efficiency and high-sampling track point clearing module is used for clearing indexes of low-efficiency and high-sampling track points when more than two queries Q exist, wherein the low-efficiency and high-sampling track points are track points of low-efficiency objects except the latest track point, the low-efficiency objects are moving objects with sampling frequency larger than a preset sampling frequency only appearing in the query Q with the lowest query frequency, and after the low-efficiency and high-sampling track points are cleared, the track points of the low-efficiency objects are read through a memory when the query Q with the lowest query frequency initiates a query.
In this embodiment, when the query Q does not initiate execution within the preset invalid query time, the query Q is determined as an invalid query, and only the trace points appearing in the query result of the invalid query are the invalid query trace points. The removal of the index of invalid query trace points does not affect the results of the query. Clearing the trace point index of the high sample rate trace hit by the low frequency query will have an impact on the query results, and therefore, it needs to be compensated by reading the trace point from the external memory. The technical scheme of the embodiment solves the problem of memory burden caused by excessive stored track point data when the number of concurrent queries is increased. Although reading data through external memory has some influence on the working efficiency of the query system, the improvement of the working efficiency caused by the reduction of the memory burden is more important as a whole.
In the monitoring and tracking of special vehicles in cities, some special vehicles are prohibited from entering an important monitoring area in a designated period and can enter in a non-designated period. Thus, some queries will not initiate execution during unspecified periods of time, becoming invalid queries. Only the special vehicles appearing in the query results of the invalid query do not need to be continuously monitored, and the indexes of the track points can be deleted. There is also a case where queries for some special vehicles are less frequent than others, but these special vehicles may still set a comparatively high track point transmission frequency for some reason. Such as a sprinkler. The sprinkler in the city is particularly slow to travel and the query for it is also a low frequency query. However, if the frequency of the track points sent by the sprinkler is too high, the use efficiency of the track point data is very low, and more memory resources are occupied. At this time, the query system can self-make up for the defect by adopting a mode of clearing indexes of low-efficiency high-sampling track points, and the memory burden is reduced and the working efficiency is improved under the condition of ensuring the query accuracy.
The trace points cleaned by the memory cleaning module in the above two embodiments may also be classified into three levels according to the accessed frequency, as shown in fig. 2:
class i is the trace points that do not appear in the result set. Such as those that have not been updated for more than 5 minutes (the maximum sampling frequency set by the querying system), and those that have been hit only by an invalid query (a query that did not send an execution request within the invalid query time).
Class ii refers to trace points that lie in the query space, but are not hit by any query.
Class iii refers to high sample rate trace points that are hit only by low frequency queries.
The track points of the two levels I and II are directly cleared without affecting the query. And selecting the track points of the III level, namely selecting the query Q with the lowest current unclean request frequency, deleting the history track points of the track which has high sampling rate and is not hit by other high-frequency queries, and only keeping the latest track points. After the III-level track is cleaned, only the latest track point is saved in the follow-up process, and the associated query result is returned through the external storage query.
In order to fully explain the practical effect of the technical scheme provided by the invention, comparative experiments are carried out according to some specific embodiments provided by the invention, and the following comparative conditions and experimental data are provided:
using the dataset Porto as the experimental dataset, fig. 7 is statistical information for the dataset. The acquisition time of the data set is 2013, 07, 01, 2014, 06, 30, and the mobile object is 442 taxis in Bohr chart city, and the sampling frequency is about 15 seconds.
The server of the experimental environment was configured as a centOS 7.4, 16-core CPU,32GB memory and 500GB hard disk.
The comparison content is query efficiency and index updating efficiency.
The method is A, the index method combining the grid and the double linked list, the query mechanism combining the internal memory and the external memory and the memory cleaning mechanism are comprehensively used, and the frequency of space intersection judgment and the frequency of data to be searched are reduced to the greatest extent.
The comparison method is B, and is realized based on SOLE. The method uses a grid as an index structure, only the current position of the object is saved, and the rest of historical track points are inquired from an external memory.
Fig. 8 and 9 show the impact of query size on query efficiency. The space ranges of the inquiry are respectively set to be 1, 3, 5, 7 and 10km 2, and the average time required by each inquiry is recorded. In both methods, the initial query (first execution of the query) needs to execute a cache query, so the efficiency is basically the same; in the subsequent inquiry, as long as the inquiry range is not changed, the method A only accesses the memory, only needs to carry out space intersection judgment on a very small amount of data and carries out retrieval on the very small amount of data, so that the inquiry efficiency is far higher than that of the method B.
FIG. 10 shows the effect of object number on index time. The number of objects was set to 0.8k, 1.6k, 3k, 7k, 14k, 28k and 55k, respectively, and the time required to save the full dataset was compared. As can be seen from the graph, as the number of objects increases, the arrival rate of data increases and the time required for data insertion increases. The method B only needs to store the latest track point to the corresponding grid, and the method A also needs to maintain the linked list structure of each track, so the data insertion efficiency of the method B is higher than that of the method A. However, the index structure maintained by method a makes it much more efficient to query than method B, as is available from the previous set of experimental results.

Claims (16)

1. A method for continuous range querying of track stream data, the method comprising:
The method comprises the steps that a mobile object obtains current position information through a satellite navigation system GNSS, track point data are continuously sent to a query system, the track points are GNSS records generated by the mobile object, and the GNSS records comprise time generated by the GNSS records, namely, time attributes and coordinates, namely, space attributes, of a geographic position where the mobile object is located;
indexing the track points, and initiating a query Q;
The attributes of the query Q include: unique identity q id, query space range q range and time q t at which the query was initiated;
the query Q comprises a space range query according to the space attribute of the grid and a time range query according to the time attribute of a doubly linked list;
The index comprises an index structure and an index update;
the index structure adopts a structure of combining a grid and a doubly linked list, the spatial attribute of the track point is managed through the grid, and the time attribute of the track point is managed through the doubly linked list;
the grid is obtained by dividing the query space range, grid preset point coordinates are used as unique identifiers of the grid, namely grid coordinates, all the grids in the query space range are used as grid sets, and the preset grid coordinates are used as grid set points;
Each grid is provided with a grid pointer which points to the latest track point in the grid range, and all track points in the grid range are ordered according to the reverse order of the arrival order;
The track is stored by using the doubly-linked list, each node of the doubly-linked list is one track point, a first node pointer of the node points to the track point before the track point in the track, and a second node pointer and a third node pointer of the node point to two track points adjacent to the track point in the grid where the track point is located in the arrival sequence;
the index updating is performed when a new track point is received, when the spatial attribute of the new track point falls in the query spatial range of the query Q, the grid where the new track point is located is calculated according to the relative position of the new track point and the grid origin, and whether the new track point is the latest track point which arrives in sequence is judged according to the time attribute of the new track point;
When the new track point is the latest track point which arrives in sequence, adding the new track point to the tail end of the doubly linked list of the grid where the new track point is located, updating the first node pointer, the second node pointer and the third node pointer of the node corresponding to the track point related to the doubly linked list, and updating the grid pointer of the grid where the new track point is located;
When the new track point is not the latest track point which arrives in sequence, a corresponding insertion position is found in the doubly linked list according to the time attribute of the new track point, the first node pointer, the second node pointer and the third node pointer of the node corresponding to the new track point of the doubly linked list are updated, and the second node pointer and the third node pointer of the node corresponding to the track point adjacent to the time attribute of the new track point in the grid are updated.
2. The method according to claim 1, characterized in that:
The query space range is a query buffer zone, and the query buffer zone is a space range in which the boundary of the query space range extends epsilon meters outwards.
3. The method according to claim 1, wherein the method further comprises:
The hash map stores the ID of a query object, wherein the query object is the mobile object with a non-empty result of space intersection judgment between the track point and the query space range of the query Q;
the hash map stores a first object pointer of the query object, the first object pointer pointing to an oldest track point of the query object in the query space range;
The hash map stores a second object pointer of the query object that points to a most recent trace point of the query object in the query space.
4. A method according to claim 3, characterized in that the method further comprises:
the hash map stores an entry timestamp of the query object, namely the time attribute of the track point when the query object enters the query space range;
And starting the second execution of the query Q, and retrieving the entry time stamp from the latest track point to the historical data direction along the doubly linked list, namely returning a query result.
5. The method according to claim 4, wherein the method further comprises:
when the query space range is changed into a new query space range, calculating a change difference set of the query space range;
When the new query space range falls within the query space range, calculating the grids in the query space range variation difference set, namely, reducing grids, deleting the reducing grids from the grids, deleting the doubly linked list of the latest track point in the reducing grids, updating the first node pointer, the second node pointer and the third node pointer of relevant nodes of the doubly linked list of the grids in the new query space range, updating the entry time stamp of a new query object, namely, the moving object in the new query space range;
when the new query space range is not in the query space range, calculating the grids in the query space range variation difference set, namely a new grid, to obtain a new grid set and a new grid set origin, and merging the new grid set with the grid set according to the position relation between the new grid set origin and the grid set origin to obtain the grid set of the new query space range; initiating a new query Q1 by taking the ID of the query object and the spatial range of the newly added grid set as attributes, and indexing the track points in the query result; initiating a new query Q2, querying the mobile object with the current position in the new added grid set to obtain a new query object, initiating a new query Q3 by using the ID of the new query object and the new query space range as attributes, and indexing the track points in the query result.
6. The method according to any one of claims 1-4, wherein:
the query system works in a mode of combining internal and external memories, and the working mode of combining internal and external memories comprises the following steps:
When the query Q is executed for the first time, the track point of the moving object is read from an external memory, and space intersection judgment is carried out on the track point and the query space range;
storing the query space range of the query Q in a memory;
only carrying out the index on the track points of the query object of the query Q in the query space range, wherein the index is stored in a memory;
when the query object leaves the query space range, deleting the index of the track point of the query object.
7. The method of claim 6, further comprising a memory clean-up, the memory clean-up comprising:
Clearing the index of the track points of a static object in the query space range, wherein the static object is the query object of which the query system does not receive the new track points in a static time, and the static time can be preset in the query system;
An index of invalid trace points is cleared, the invalid trace points being the trace points that do not appear in the query result of the query Q.
8. The method of claim 7, wherein the memory clean-up further comprises:
When there are more than two of the queries Q,
Clearing indexes of invalid query track points, wherein the invalid query track points are the track points only appearing in invalid queries, the invalid queries are the queries Q which do not initiate execution within invalid query time, and the invalid query time can be preset in the query system;
And clearing indexes of low-efficiency high-sampling track points, wherein the low-efficiency high-sampling track points are track points of low-efficiency objects except the latest track points, the low-efficiency objects are moving objects with sampling frequency larger than preset sampling frequency only in the query Q with the lowest query frequency, and after the low-efficiency high-sampling track points are cleared, when the query Q with the lowest query frequency initiates a query, the track points of the low-efficiency objects are read through the external memory.
9. A system for continuous range querying of track stream data, the system comprising:
The system comprises a position information sending module, an index module and a query module;
The position information sending module is used for obtaining current position information through a satellite navigation system GNSS on a mobile object, and continuously sending track point data to a query system, wherein the track point is a GNSS record generated by the mobile object, and the GNSS record comprises time generated by the GNSS record, namely a time attribute and coordinates of a geographic position where the mobile object is located, namely a space attribute;
the query module is configured to initiate a query Q, where attributes of the query Q include: unique identification q id, query space range q range, and time q t at which the query was initiated;
The query module comprises a space range query module and a time range query module;
The space range query module is used for performing space range query according to the space attribute of the grid when the query Q is initiated;
The time range query module is used for performing time range query according to the time attribute of the doubly linked list when the query Q is initiated;
the index module is used for indexing the track points and comprises an index structure module and an index updating module;
the index structure module is used for constructing a structure combining a grid and a double-linked list, and comprises a grid module and a linked list module, wherein the grid module is used for managing the spatial attribute of the track point, and the linked list module is used for managing the time attribute of the track point;
the grid module comprises a grid dividing module and a grid pointer module;
The grid dividing module is configured to divide the query space range into grids to obtain the grids, take grid preset point coordinates as unique identifiers of the grids, that is, grid coordinates, take all the grids in the query space range as grid sets, and take the preset grid coordinates as grid set origins;
The grid pointer module is used for managing grid pointers of the grids, each grid is provided with one grid pointer, the grid pointers point to the latest track points in the grid range, and all track points in the grid range are ordered according to the reverse order of the arrival order;
The linked list module is used for storing tracks by using the doubly linked list, each node of the doubly linked list is one track point, a first node pointer of the node points to the track point before the track point in the track, and a second node pointer and a third node pointer of the node point to two track points adjacent to the track point in the grid where the track point is located in the arrival sequence of the track point;
The index updating module is used for updating the index when receiving the new track point and comprises a calculating and judging module, a sequential updating module and an out-of-order updating module;
the calculation and judgment module is used for calculating the grid where the new track point is located according to the relative position of the new track point and the grid set origin when the spatial attribute of the new track point falls in the query spatial range of the query Q, and judging whether the new track point is the latest track point which arrives sequentially according to the time attribute of the new track point; a sequential updating module, configured to, when the new track point is the latest track point that arrives sequentially, add the new track point to the end of the doubly linked list of the mesh where the new track point is located, update the first node pointer, the second node pointer, the third node pointer of the node corresponding to the track point related to the doubly linked list, and update the mesh pointer of the mesh where the new track point is located;
and the out-of-order updating module is used for searching a corresponding insertion position in the doubly-linked list according to the time attribute of the new track point when the new track point is not the latest track point which arrives in sequence, updating the first node pointer, the second node pointer and the third node pointer of the node corresponding to the new track point of the doubly-linked list, and updating the second node pointer and the third node pointer of the node corresponding to the track point adjacent to the time attribute of the new track point in the grid where the new track point is located.
10. The system according to claim 9, wherein:
The query space range is a query buffer zone, and the query buffer zone is a space range in which the boundary of the query space range extends epsilon meters outwards.
11. The system of claim 9, further comprising a hash-map storage module, the hash-map storage module comprising:
Inquiring an object ID storage module, a first object pointer storage module and a second object pointer storage module;
The query object ID storage module is used for hash mapping and storing the ID of a query object, wherein the query object is the mobile object with the non-empty result of space intersection judgment between the track point and the query space range of the query Q;
the first object pointer storage module is used for hash mapping and storing a first object pointer of the query object, wherein the first object pointer points to the oldest track point of the query object in the query space range;
The second object pointer storage module is used for hash mapping and storing a second object pointer of the query object, wherein the second object pointer points to the latest track point of the query object in the query space range.
12. The system of claim 11, further comprising a historical data retrieval module, the historical data retrieval module comprising:
the time stamp storage module and the result return module;
The time stamp storage module is used for storing the entry time stamp of the query object in a hash mapping mode, namely the time attribute of the track point when the query object enters the query space range;
And the result returning module is used for searching the entry time stamp from the latest track point to the historical data direction along the doubly linked list from the second execution of the query Q, namely returning the query result.
13. The system of claim 12, further comprising a spatial range variation module, the spatial range variation module comprising:
The difference set calculation module, the shrinkage change module and the expansion change module;
the difference set calculation module is used for calculating a change difference set of the query space range when the query space range is changed into a new query space range;
The reduction change module is configured to calculate the grids in the query space range change difference set, that is, reduce grids, delete the reduced grids from the grids set, delete the doubly linked list of the newest track point in the reduced grids, update the first node pointer, the second node pointer, the third node pointer of the doubly linked list of the grids in the new query space range, update the entry timestamp of a new query object, that is, the moving object in the new query space range, when the new query space range falls in the query space range;
The expansion change module is configured to calculate the grid in the change difference set of the query space range, that is, a new grid, to obtain a new grid set and a new grid set origin, and merge the new grid set and the grid set according to a positional relationship between the new grid set origin and the grid set origin, to obtain the grid set of the new query space range when the new query space range is not within the query space range; initiating a new query Q1 by taking the ID of the query object and the spatial range of the newly added grid set as attributes, and indexing the track points in the query result; initiating a new query Q2, querying the mobile object with the current position in the new added grid set to obtain a new query object, initiating a new query Q3 by using the ID of the new query object and the new query space range as attributes, and indexing the track points in the query result.
14. The system according to any one of claims 9-12, wherein:
the query system works in a mode of combining internal and external memories, and the working mode of combining internal and external memories comprises the following steps:
When the query Q is executed for the first time, the track point of the moving object is read from an external memory, and space intersection judgment is carried out on the track point and the query space range;
storing the query space range of the query Q in a memory;
only carrying out the index on the track points of the query object of the query Q in the query space range, wherein the index is stored in a memory;
when the query object leaves the query space range, deleting the index of the track point of the query object.
15. The system of claim 14, further comprising a memory cleaning module, the memory cleaning module comprising:
A static object clearing module and an invalid track point clearing module;
The static object clearing module is configured to clear the index of the track point of a static object in the query space range, where the static object is the query object for which the query system does not receive the new track point in a static time, and the static time can be preset in the query system;
The invalid track point clearing module is used for clearing indexes of invalid track points, wherein the invalid track points are track points which do not appear in the query result of the query Q.
16. The system of claim 15, wherein the memory cleaning module further comprises:
A trace point removing module for invalid inquiry and a trace point removing module for low efficiency and high sampling;
The invalid query track point clearing module is configured to clear, when the query Q has more than two, an index of an invalid query track point, where the invalid query track point is a track point that only appears in an invalid query, the invalid query is the query Q that has not initiated execution within an invalid query time, and the invalid query time may be preset in the query system;
The low-efficiency and high-sampling track point clearing module is configured to clear indexes of low-efficiency and high-sampling track points when the query Q has more than two, where the low-efficiency and high-sampling track points are track points of a low-efficiency object except the latest track point, the low-efficiency object is the moving object that only appears in the query Q with the lowest query frequency and has a sampling frequency greater than a preset sampling frequency, and after the low-efficiency and high-sampling track points are cleared, when the query Q with the lowest query frequency initiates a query, the track points of the low-efficiency object are read through the external memory.
CN202111091041.2A 2021-09-17 2021-09-17 Track stream data-oriented continuous range query method and system Active CN113742536B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111091041.2A CN113742536B (en) 2021-09-17 2021-09-17 Track stream data-oriented continuous range query method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111091041.2A CN113742536B (en) 2021-09-17 2021-09-17 Track stream data-oriented continuous range query method and system

Publications (2)

Publication Number Publication Date
CN113742536A CN113742536A (en) 2021-12-03
CN113742536B true CN113742536B (en) 2024-04-19

Family

ID=78739587

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111091041.2A Active CN113742536B (en) 2021-09-17 2021-09-17 Track stream data-oriented continuous range query method and system

Country Status (1)

Country Link
CN (1) CN113742536B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116578569B (en) * 2023-07-12 2023-09-12 成都国恒空间技术工程股份有限公司 Satellite space-time track data association analysis method

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20080095533A (en) * 2007-04-25 2008-10-29 제주대학교 산학협력단 Structure and method of indexing for processing of moving object queries in road network databases
CN107291842A (en) * 2017-06-01 2017-10-24 武汉理工大学 The track querying method encoded based on track
WO2018099480A1 (en) * 2016-12-01 2018-06-07 中兴通讯股份有限公司 Vehicle driving trajectory monitoring method and system
CN110119408A (en) * 2019-03-22 2019-08-13 西安电子科技大学 Mobile object continuous-query method under geographical space real-time streaming data

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20080095533A (en) * 2007-04-25 2008-10-29 제주대학교 산학협력단 Structure and method of indexing for processing of moving object queries in road network databases
WO2018099480A1 (en) * 2016-12-01 2018-06-07 中兴通讯股份有限公司 Vehicle driving trajectory monitoring method and system
CN107291842A (en) * 2017-06-01 2017-10-24 武汉理工大学 The track querying method encoded based on track
CN110119408A (en) * 2019-03-22 2019-08-13 西安电子科技大学 Mobile object continuous-query method under geographical space real-time streaming data

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
大规模轨迹数据的Geohash编码组织及高效范围查询;向隆刚;王德浩;龚健雅;武汉大学学报. 信息科学版;第42卷(第1期);全文 *

Also Published As

Publication number Publication date
CN113742536A (en) 2021-12-03

Similar Documents

Publication Publication Date Title
Cudre-Mauroux et al. Trajstore: An adaptive storage system for very large trajectory data sets
CN106649656B (en) Database-oriented space-time trajectory big data storage method
Jeung et al. Trajectory pattern mining
US9275131B2 (en) Data indexing system, data indexing method and data querying method
CN107423422B (en) Spatial data distributed storage and search method and system based on grid
Chakka et al. Indexing large trajectory data sets with SETI.
CN110442444A (en) A kind of parallel data access method and system towards mass remote sensing image
CN105512317B (en) A kind of uncertain Indexing for Moving Object based on group division
Li et al. Robust inferences of travel paths from GPS trajectories
CN113742536B (en) Track stream data-oriented continuous range query method and system
CN113946575A (en) Space-time trajectory data processing method and device, electronic equipment and storage medium
Thomas et al. Design of high performance cluster based map for vehicle tracking of public transport vehicles in smart city
Fang et al. Dragoon: a hybrid and efficient big trajectory management system for offline and online analytics
Sarwat Interactive and scalable exploration of big spatial data--a data management perspective
CN109145225B (en) Data processing method and device
Ali et al. Sentinel-1 data cube exploitation: Tools, products, services and quality control
CN113779105B (en) Distributed track flow accompanying mode mining method
Abbasifard et al. Efficient indexing for past and current position of moving objects on road networks
Bashir et al. An intelligent linear time trajectory data compression framework for smart planning of sustainable metropolitan cities
US10095724B1 (en) Progressive continuous range query for moving objects with a tree-like index
Brakatsoulas et al. Practical data management techniques for vehicle tracking data
Sasikala et al. Uncertain data prediction on dynamic road network
Bakalov et al. Time dependent transportation network models
CN109947884B (en) Efficient organization query method for global ICESat/GLAS point cloud
Rslan et al. Spatial R-tree index based on grid division for query processing

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant