CN113806376B

CN113806376B - Index construction method and device

Info

Publication number: CN113806376B
Application number: CN202111318136.3A
Authority: CN
Inventors: 张艾琳; 谢炯; 王方
Original assignee: Alibaba China Co Ltd; Alibaba Cloud Computing Ltd
Current assignee: Alibaba China Co Ltd; Alibaba Cloud Computing Ltd
Priority date: 2021-11-09
Filing date: 2021-11-09
Publication date: 2022-05-13
Anticipated expiration: 2041-11-09
Also published as: CN113806376A

Abstract

The present specification provides an index construction method and an index construction device, wherein the index construction method includes: dividing received track data into at least two track segment data, and determining a primary index table and a primary index value of each track segment data according to each track segment data and corresponding first attribute information; constructing a primary index of each track segment data based on the primary index table and a primary index value; determining a secondary index value according to the second attribute information of each track segment data, and determining a secondary index table associated with the primary index based on the secondary index value; and constructing a secondary index of each track segment data based on the primary index and a secondary index table associated with the primary index.

Description

Index construction method and device

Technical Field

The specification relates to the technical field of computers, in particular to an index construction method.

Background

With the rapid development of internet technology, the demand for mobile object management is increasing in the fields of automatic driving, shared travel, internet of vehicles, public safety and the like, and the demand specifically includes real-time query of trajectory data which is acquired by a mobile object and has space-time characteristics and the like. However, the track data is large in size and contains complex characteristics such as position information, so that the query efficiency of the track data is low, and the time cost of query operation is greatly increased.

Disclosure of Invention

In view of this, the embodiments of the present specification provide an index building method. The present specification also relates to an index building apparatus, a computing device, a computer readable storage medium, and a computer program to solve the technical drawbacks of the prior art.

According to a first aspect of embodiments of the present specification, there is provided an index building method, including:

dividing received track data into at least two track segment data, and determining a primary index table and a primary index value of each track segment data according to each track segment data and corresponding first attribute information;

constructing a primary index of each track segment data based on the primary index table and a primary index value;

determining a secondary index value according to the second attribute information of each track segment data, and determining a secondary index table associated with the primary index based on the secondary index value;

and constructing a secondary index of each track segment data based on the primary index and a secondary index table associated with the primary index.

According to a second aspect of embodiments herein, there is provided an index building apparatus including:

the first determining module is configured to divide the received track data into at least two track segment data, and determine a primary index table and a primary index value of each track segment data according to each track segment data and corresponding first attribute information;

the first construction module is configured to construct a primary index of each track segment data based on the primary index table and a primary index value;

a second determining module configured to determine a secondary index value according to the second attribute information of each track segment data, and determine a secondary index table associated with the primary index based on the secondary index value;

a second construction module configured to construct a secondary index for each track segment data based on the primary index and a secondary index table associated with the primary index.

According to a third aspect of embodiments herein, there is provided a computing device comprising:

a memory and a processor;

the memory is configured to store computer-executable instructions and the processor is configured to execute the computer-executable instructions, which when executed by the processor implement the steps of the index building method described above.

According to a fourth aspect of embodiments herein, there is provided a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the index building method described above.

According to a fifth aspect of embodiments herein, there is provided a computer program, wherein when the computer program is executed in a computer, the computer program causes the computer to perform the steps of the index building method described above.

The index construction method provided by the specification comprises the following steps: dividing received track data into at least two track segment data, and determining a primary index table and a primary index value of each track segment data according to each track segment data and corresponding first attribute information; constructing a primary index of each track segment data based on the primary index table and a primary index value; determining a secondary index value according to the second attribute information of each track segment data, and determining a secondary index table associated with the primary index based on the secondary index value; and constructing a secondary index of each track segment data based on the primary index and a secondary index table associated with the primary index.

Specifically, the index construction method divides the received track data into at least two track segment data, and constructs the primary index and the secondary index of each track segment data based on each track segment data and the first attribute information and the second attribute information of each track segment data, thereby realizing the follow-up primary index and secondary index based, improving the query efficiency of the track data, and reducing the time cost of query operation on the track data.

Drawings

Fig. 1 is a schematic structural diagram of a track management system on a cloud in an index building method according to an embodiment of the present specification;

FIG. 2 is a flow chart of a method for index construction according to an embodiment of the present disclosure;

FIG. 3 is a flowchart illustrating a process applied to an index building method based on an index query trajectory data scenario according to an embodiment of the present specification;

FIG. 4 is a schematic structural diagram of an index building apparatus according to an embodiment of the present disclosure;

fig. 5 is a block diagram of a computing device according to an embodiment of the present disclosure.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. This description may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, as those skilled in the art will be able to make and use the present disclosure without departing from the spirit and scope of the present disclosure.

The terminology used in the description of the one or more embodiments is for the purpose of describing the particular embodiments only and is not intended to be limiting of the description of the one or more embodiments. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It will be understood that, although the terms first, second, etc. may be used herein in one or more embodiments to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first can also be referred to as a second and, similarly, a second can also be referred to as a first without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

First, the noun terms to which one or more embodiments of the present specification relate are explained.

A multi-mode database: a multi-model database product can provide database services of various scales and models.

Secondary indexing: refers to a native global secondary index solution proposed to solve the problem of multidimensional queries. According to the scheme, the secondary index is established for a certain column of the main table, and the mapping relation between the column value and the row key of the index table is achieved, so that the storage and query costs are reduced.

Space-time indexing: indices defined in consideration of spatio-temporal properties of data, such as R-tree, kd-tree, quadtree, and space-filling curve-coded indices.

Trajectory segmentation line model: the method comprises the steps of segmenting a track into a plurality of track segments based on a certain segmentation strategy; in practical applications, each segment is stored as a record, and the necessary information for splicing back to the entire track is preserved during storage.

GeoMesa: the tool kit is an open-source tool kit for massive spatio-temporal data query and analysis based on a distributed computing system.

HBase: is a distributed, column-oriented open source database.

Spark: is a unified analysis engine for large-scale data processing.

OLAP: refers to an on-line analytical process.

Schema: also known as modes, architectures; refers to the organization and structure of a database.

Track data: the method refers to data information obtained by sampling one or more moving objects in a motion process under a space-time environment, and the data information comprises sampling point positions, sampling time, speed and the like.

A Filter: a filter is provided.

With the rapid development of internet technology, the demand for mobile object management in the fields of automatic driving, shared travel, internet of vehicles, public safety and the like is increasing, and the demand specifically includes storage, real-time query, data mining and the like of data with space-time characteristics acquired by a mobile object. However, the track data volume is large, and complicated characteristics such as position information are included, so that many databases cannot meet the functional requirements.

Based on this, the solution provided in this specification integrates Spark by using HBase in the storage layer by using the open source indexing tool GeoMesa, and provides a uniform SQL (structured query language) on top of it for query, calculation, and analysis functions. And storing each track as a record, wherein key is a space-time index value, and value comprises a track outer frame, a starting point and time, an ending point and time, a list of track points and other attribute columns.

However, this solution has three disadvantages:

(1) and the time-space index is carried out on the whole track, so that the efficiency of the track with a large time-space range is not high.

(2) The query for the combined attribute column is an OLAP system, and the Spark Context is also initiated for a simple query, resulting in inefficiency.

(3) Each index table needs to store the attribute columns repeatedly, and the storage cost is large.

Based on this, the track management system on the cloud has come, and the index construction method provided in this specification is applied to the track management system on the cloud, see fig. 1, where fig. 1 is a structural diagram of the track management system on the cloud in the index construction method provided in an embodiment of this specification; the track management system on the cloud comprises a client and a storage layer based on a multimode database; through the client of the track management system on the cloud, schema for defining a track segmentation line model, building spatio-temporal indexes XZ2T/OTS, data operation (such as insertion, update and deletion), data query and data table operation (such as table building, writing and query) supporting secondary indexes can be realized. Meanwhile, the storage layer of the track management system on the cloud is realized based on the low-cost distributed storage processing, the secondary index and the cold-hot separation function of the multimode database, and the calculation push-down can also be realized based on the multimode database. Therefore, the track management system on the cloud provided by the specification effectively utilizes a multi-mode database secondary index mechanism, establishes a segmentation model (track segmentation line model) for track data, and creates ots and xz2t secondary indexes for tracks, so that the database management system with low storage cost and high query efficiency is realized.

Specifically, in the present specification, there is provided an index construction method, and the present specification simultaneously relates to an index construction apparatus, a computing device, a computer-readable storage medium, and a computer program, which are described in detail one by one in the following embodiments.

Referring to fig. 2, fig. 2 is a flowchart illustrating an index building method according to an embodiment of the present disclosure, which specifically includes the following steps:

step 202: dividing the received track data into at least two track segment data, and determining a primary index table and a primary index value of each track segment data according to each track segment data and corresponding first attribute information.

The track segment data may be track segment data obtained by segmenting a track based on a certain segmentation strategy, and in specific implementation, the track segment data may be a track segment line model. The first attribute information may be set according to an actual application scenario, for example, metadata of track segment data; a primary index table may be understood as a table that stores track segment data, as well as indexes and metadata associated with the track segment data; in practical applications, the primary index table may be a main table. A primary index value may be understood to be an index value, e.g., a rowkey value, stored in a primary index table for facilitating a query of track segment data in the primary index table.

In practical applications, the track segment data can be stored in the form of key-value pairs (key-values); the key may be understood as a rowkey value, and the value may be understood as track segment data, as well as metadata of the track segment data.

Specifically, after receiving track data, a track management system (hereinafter referred to as a system) on the cloud divides the data into at least two track segment data, determines first attribute information of each track segment data, and determines a primary index table and a primary index value of each track segment data based on each track segment data and the first attribute information of each track segment data.

For example, taking an example that the index construction method is applied to establish a secondary index for track segment data, a primary index table and a primary index value for determining each track segment data are further described.

There is an increasing demand in the field of automated driving for managing moving objects (e.g., vehicles), including storing trajectory data collected by vehicles with spatiotemporal characteristics, querying in real time, and the like. Based on this, the index construction method provided in this specification defines a track segmentation line model (TrajectoryFeatureType) in the track management system on the cloud. The trajectory segmentation line model can be understood as a data structure.

After the definition of the track segmentation line model is completed, when the track management system on the cloud receives a whole track data of the moving vehicle, the track data is segmented according to a specific rule to obtain at least two track segments, and a track _ segment object is respectively established for each track segment according to the definition of the track segmentation line model (trackFeatureType). Thereby obtaining at least two track segment data corresponding to the complete track data. The structure of the trajectory segmentation line model can be shown in table 1:

TABLE 1

See table 1, where the trajectory segmentation line model (TrajectoryFeatureType) includes: object _ id: the moving object id. get: the trajectory geometry. In this specification, the term "geo" is used as an attribute name set for a track geometric attribute, and in this embodiment, the term "geo" is taken as an example to describe the attribute name of the track geometric attribute. Most of moving object processing systems (for example, JUST) store track points in a List [ Point ] form in a track line model, and when a spatial range query is performed, judge whether some points fall within a query range, if so, consider that the track and the query range intersect. However, in practical application, there is a case where the collected trace points do not fall within the query range, but the connecting lines of adjacent points intersect with the range. In consideration of the fact that it is still desirable to judge that intersection returns true under such a condition, the index construction method provided by the specification adopts a LineString type stored track point connecting line to support accurate intersection judgment of the type. List [ Point ] can also be obtained from LineString when discrete trace points are needed. traj _ start _ time: the starting time of the whole track data to which the track segment belongs. segment _ start _ time: the start time of the track segment. segment _ index: the track segment is the number of segments in the whole track (namely the track segment number); list [ traj _ point _ time ]: and each trace point time corresponds to the order of the midpoint of the get one by one. List [ traj _ point _ attr1], List [ traj _ point _ attr2] …: and (4) defining attributes by each track point, wherein each attribute is a List, and the attribute value in each List corresponds to the order of the midpoint of the get.

In the process of segmenting the track data according to the specific rule, the specific rule can be set according to the actual application scene, for example, the whole track is segmented according to the number of fixed track points, the whole track is segmented according to a preset time range, and the like.

After at least two track segment data (track _ segment objects) are obtained by segmenting the complete track data, the metadata of each track segment data can be obtained, and based on the metadata and the track _ segment objects, a main table storing each track segment data and a rowkey value stored in the main table are determined.

In specific implementation, the index construction method provided in this specification determines a primary index table and a primary index value, as shown below.

The determining a primary index table and a primary index value of each track segment data according to each track segment data and corresponding first attribute information includes:

constructing a primary index table of each track segment data according to each track segment data and the first attribute information of each track segment data;

and determining a corresponding primary index value for each track segment data in the primary index table.

Following the above example, after the track _ segment objects and the metadata are determined, each track _ segment object and the metadata corresponding to each object are used as columns in the main table, and a main table is constructed based on the track _ segment object columns and the metadata columns.

After the creation of the main table is completed, for each target _ segment object in the main table, a rowkey value corresponding to the object is generated respectively and stored in the main table. Wherein, the structure of the main table can be seen in table 2;

TABLE 2

See table 2 where rowkey is the name of the rowkey value column in the main table and r1, r2 are the main table rowkey values, unique per row. The main table has only one family "d". In the column "m" included in the column family "d", metadata of each track segment is stored. The serialized track segment data (track _ segment object) defined by the track segment line model is stored in the column "a".

In the embodiment of the present specification, a primary index table of each track segment data is constructed according to each track segment data and first attribute information of each track segment data, and a corresponding primary index value is determined for each track segment data in the primary index table. Therefore, a primary index is conveniently constructed subsequently based on the primary index table and the primary index value, and the retrieval efficiency of the track data is further improved.

Step 204: and constructing a primary index of each track segment data based on the primary index table and the primary index value.

According to the above example, after the construction of the main table is completed and the corresponding rowkey value is determined for each row of track segment data in the main table, the construction of the primary index can be completed based on the main table and the rowkey value, and then the corresponding track segment data can be found based on the rowkey value to complete the query of the track segment data.

Step 206: and determining a secondary index value according to the second attribute information of each track segment data, and determining a secondary index table associated with the primary index based on the secondary index value.

The second attribute information may be set according to an actual application scenario, for example, a start time of each track segment data, spatio-temporal feature data such as a track segment outer frame, a moving object ID (i.e., object _ ID) to which each track segment belongs, a start time of complete track data (i.e., traj _ start _ time) to which each track segment data belongs, a track segment number (i.e., segment _ index), and the like.

The secondary index table may be understood as a table associated with the primary index that stores values of the secondary index.

In practical application, determining a secondary index value according to the second attribute information of each track segment data includes:

coding second attribute information of each track segment data to obtain a second attribute code of each track segment data;

and obtaining a secondary index value of each track segment data based on the second attribute codes and the primary index values.

The second attribute code may be understood as a code obtained by performing a coding process on the second attribute information.

According to the above example, after the construction of the primary index is completed, in order to further improve the space-time query efficiency of the track data, the query efficiency can be improved by constructing a space-time secondary index on the basis of the primary index, on the basis, after the construction of the primary index is completed, the space-time feature data of each track segment data is obtained, the track segment outer package frame in the space-time feature data is subjected to coding processing, the coding of the track segment outer package frame is obtained, and the starting time of each track segment data in the space-time feature data and the number of the time interval to which the track segment data belongs are determined; the time interval division interval and the starting point can be customized.

After determining the number of the encoding and time interval of the track segment outsourcing frame, generating an initial spatio-temporal index value based on the number of the encoding and time interval of the track segment outsourcing frame. And the initial index value and the primary index value are added to obtain a spatiotemporal secondary index value of each track segment data, so that a spatiotemporal secondary index is conveniently generated based on the spatiotemporal secondary index value, and the spatiotemporal query efficiency is improved.

In another case, the index construction method provided in this specification can select target attribute information, such as attribute information like object _ id, traj _ start _ time, segment _ index, etc., from the second attribute information of each track segment data; and generating a secondary index value based on the attribute information, thereby avoiding the problem of low index construction efficiency caused by encoding the attribute information, and the specific mode is as follows.

The determining a secondary index value according to the second attribute information of each track segment data further includes:

selecting target attribute information from the second attribute information of each track segment data according to a preset selection rule;

and obtaining a secondary index value of each track segment data based on the target attribute information and the primary index value.

The target attribute information may be understood as attribute information that satisfies a preset selection rule in the second attribute information, where the preset selection rule may be set according to an actual application scenario, and for example, the preset selection rule may be to obtain attribute information such as object _ id, traj _ start _ time, and segment _ index.

The above example is used, wherein the object attribute information may be object _ id, traj _ start _ time, segment _ index, and other attribute information.

After the construction of the primary index is completed, in order to realize the back check of the complete track data based on the track segment data, the back check operation can be conveniently carried out subsequently by constructing an OTS secondary index on the basis of the primary index. Based on this, after the construction of the primary index is completed, the attribute information such as object _ id, traj _ start _ time, segment _ index and the like is acquired from the second attribute information of each track segment data according to the preset selection rule.

And generating an initial OTS secondary index value based on attribute information such as the object _ id, traj _ start _ time, segment _ index and the like, and adding the initial OTS secondary index value and the rowkey value to obtain the OTS secondary index value, so that the OTS secondary index is conveniently generated based on the OTS secondary index value subsequently, and the full track data is rechecked.

In practical application, in the process of constructing the secondary index based on the secondary index value, the secondary index value needs to be added to the primary index table as the secondary index column in the primary index table, and the secondary index table is created for the secondary index column, each row of data of the secondary index table uniquely corresponds to a certain row in the main table, and the specific implementation manner is as follows.

The determining a secondary index table associated with the primary index based on the secondary index value comprises:

adding the secondary index value to the primary index table and generating a column value associated with the primary index value;

constructing a secondary index table according to the secondary index value and the column value;

establishing an incidence relation between the primary index table and the secondary index table based on the secondary index value;

and determining the secondary index table as a secondary index table associated with the primary index based on the association relation.

Where column values may be understood as values in the secondary index table that are introduced for maintaining data consistency with the primary table, each column value has a corresponding rowkey value in the primary table.

Following the above example, after determining the secondary index value, the secondary index value is first added to the primary table as the secondary index column in the primary table, see table 2, where table 2 also shows the secondary index column in the primary table when the secondary index value is the OTS secondary index value and the spatio-temporal secondary index value. Wherein, the OTS column contains OTS secondary index values; column xz2t contains the spatio-temporal secondary index value. In practical application, other indexes may be added to the main table, and redundant description is not repeated again.

After the secondary index value is added to the main table, a column value associated with the rowkey value in the main table needs to be determined, and a secondary index table is constructed for the secondary index value column in the main table based on the column value and the secondary index value. After the construction of the secondary index table is completed, the incidence relation between the main table and the secondary index table is established based on the secondary index value, and the secondary index table is determined to be associated with the primary index based on the incidence relation, so that the secondary index can be conveniently constructed based on the secondary index table and the primary index.

In specific implementation, when the secondary index value rows are different, the secondary index tables constructed for the secondary index value rows also have a certain difference. The spatio-temporal secondary index table constructed for the spatio-temporal secondary index value column can be seen from table 3.

TABLE 3

The XZ2t index1 and XZ2t index2 included in the rowkey column in the spatio-temporal secondary index table are XZ2T index values (spatio-temporal secondary index values) of each track segment, and the spatio-temporal secondary index values are identical to the spatio-temporal secondary index values in the main table. The values 1 and 2 are column values in the spatio-temporal secondary index table, and the column values are in one-to-one correspondence with the rowkey values in the main table.

The OTS secondary index table constructed for the OTS secondary index value column can be seen in table 4.

TABLE 4

The row key column in the OTS secondary index table contains OTS secondary index values, the OTS index1 and the OTS index2 are the OTS index values of each track segment data, and the OTS secondary index values are consistent with the OTS secondary index values in the main table. The values 1 and 2 are column values in the OTS secondary index table, and the column values are in one-to-one correspondence with the rowkey values in the main table.

In the embodiment of the present specification, a secondary index value is added to a primary index table, and a column value associated with the primary index value is generated; thereby realizing the construction of a secondary index table according to the secondary index value and the column value; after the construction of the secondary index table is completed, establishing an incidence relation between the primary index table and the secondary index table based on the secondary index value; and determining the secondary index table as a secondary index table associated with the primary index based on the association relation. Therefore, the construction of the secondary index is completed based on the primary index and the secondary index table subsequently. And the query efficiency of the track data is improved.

Step 208: and constructing a secondary index of each track segment data based on the primary index and a secondary index table associated with the primary index.

According to the above example, after the primary index of each track segment data is constructed and the secondary index table associated with the primary index is determined, the primary index and the secondary index table associated with the primary index are constructed as the secondary index of each track segment data, so that the track data can be conveniently queried based on the secondary index.

In specific implementation, under the condition that the secondary index tables are different, the establishment of the secondary index based on the secondary index tables is also different; the structure of the OTS secondary index constructed based on the OTS secondary index table can be seen in table 5.

TABLE 5

Wherein the OTS secondary index is used to review the complete track data. The remaining fields may correspond to attribute information of track segment data defined based on the track segment model in table 1, except for the shrard number. Long and int in table 5 are a data type, 1byte and id.length are column sizes, respectively, and the shrrd column is used to store the number of the slice of the OTS secondary index.

When creating a secondary index for the track segment OTS attribute information (moving object id (object id)) + whole track start Time (Time) + track segment number (segment index)), an OTS secondary index column needs to be created in the main table, the OTS secondary index column stores the OTS index value, and then a secondary index table is created for the index column. Each row of data of the secondary index table is uniquely corresponding to a certain row in the main table.

The structure of the XZ2T secondary index constructed based on the spatio-temporal secondary index table can be seen from table 6.

TABLE 6

Wherein, the length and short in table 6 are a data type and 1byte is the column size respectively. The shard column is used to store the number of the shards of the spatio-temporal secondary index.

The XZ2T secondary index is used to improve spatiotemporal query efficiency. Wherein, the timebin is the number of the time interval to which the track segment start time belongs, and the time interval division interval and the start point can be defined by users; xz2 code is the xz2 encoding of the track segment bounding box. When creating a secondary index for the spatiotemporal attributes of the track segment data, a corresponding xz2t secondary index column needs to be created in a main table, the column stores the xz2t code, and then a secondary index table is created for the index column. Each row of data of the secondary index table is uniquely corresponding to a certain row in the main table.

According to the index construction method provided by the specification, the received track data is divided into at least two track segment data, and the primary index and the secondary index of each track segment data are constructed based on each track segment data and the first attribute information and the second attribute information of each track segment data, so that the follow-up operation based on the primary index and the secondary index is realized, the track data query efficiency is improved, and the time cost of the track data query operation is reduced.

Meanwhile, the index construction method provided by the specification uses the track segmentation line model, can be compatible with the line model supporting JUST for short track data, and can be used for storing long track data after segmentation, so that the data volume returned to a calculation layer in the query process is effectively reduced. And the storage cost is effectively reduced by using the native secondary index of the multi-mode database. In practical applications, where two secondary index tables are used, the storage cost of the track data is reduced to at least 50% of the full storage.

In a specific implementation process, after the construction of the secondary index is completed, the target track data to be queried can be queried through the secondary index, and a specific implementation manner is as follows.

And after the secondary index of each track segment data is constructed, the method also comprises a step one to a step three.

The method comprises the following steps: receiving a track query condition;

the track query condition may be set according to an actual application scenario, and the track query condition may be a space-time range, for example, the track query condition may be to query track information of all moving objects in a route location a of 10, 1, and 2020.

Specifically, the track management system on the cloud can receive the track query condition, and the track query suite can be sent to the track management system on the cloud through an SQL statement.

Step two: determining at least one target track segment data corresponding to the track query condition according to the primary index and the secondary index;

the target track segment data can be understood as track segment data meeting track query conditions.

Specifically, the determining at least one target track segment data corresponding to the track query condition according to the primary index and the secondary index includes:

scanning the secondary index based on the track query condition to obtain a scanning result corresponding to the track query condition;

and under the condition that the index value scanning result is not empty, scanning the primary index based on the scanning result to obtain at least one target track segment data corresponding to the track query condition.

Specifically, after receiving a track query condition, scanning the secondary index based on the track query condition to obtain a scanning result of the track query condition, and in case that the scanning result is not empty, scanning the primary index again based on the scanning result to obtain at least one target track segment data corresponding to the query condition object.

In practical applications, the trajectory query condition may be a space-time range, but the query condition of the space-time range is different from a format of an index value in the space-time secondary index corresponding to the query condition, and therefore, the query condition needs to be encoded, so that the secondary index is scanned based on the encoding, which is specifically implemented as follows.

The scanning the secondary index based on the track query condition to obtain a scanning result corresponding to the track query condition includes:

coding the track query condition to obtain a track query code;

determining a secondary index table corresponding to the track query code from the secondary index;

and scanning the secondary index values in the secondary index table based on the track query codes to obtain a scanning result corresponding to the track query condition.

Wherein, the scanning result can be understood as a secondary index value meeting the track query condition in the secondary index table.

After receiving the track query condition which is a space-time range, carrying out coding processing on the space-time range to generate a space-time coding range; a primary table corresponding to the space-time coding range is determined, and a space-time index column (xz 2t index value column) for scanning the primary table is specified based on the space-time coding range. For the xz2t index value column with the spatio-temporal secondary index established, the track management system on the cloud automatically scans the xz2t index values contained in the secondary index table of the spatio-temporal secondary index. And generates a scan result for the xz2t index value.

In the embodiment of the present specification, a track query code is obtained by performing coding processing on a track query condition; and scanning the secondary index values in the secondary index table corresponding to the secondary index table to obtain a scanning result corresponding to the track query condition, so that target track data can be rapidly and accurately queried based on the scanning result, and the query efficiency of the track data is improved.

Further, the scanning the primary index based on the scanning result to obtain at least one target track segment data corresponding to the track query condition includes:

determining a target column value corresponding to the scanning result from the secondary index table, and determining a primary index value associated with the target column value from a primary index table of the primary index;

and acquiring at least one target track segment data corresponding to the track query condition from a primary index table based on the primary index value associated with the target column value.

In the case where the scan result is a secondary index value, the target column value may be understood as a column value in a row with the secondary index value.

Following the above example, when there is an xz2t index value satisfying the trajectory query condition for the spatio-temporal range in the secondary index table, a column value in the same row as the xz2t index value is determined as a target column value, a rowkey value associated with the target column value is determined from the primary index table, and at least one target trajectory segment data corresponding to the trajectory query condition is determined from the primary index table based on the rowkey value.

In the embodiment of the present specification, a target column value corresponding to a scanning result is determined from a secondary index table, and a primary index value associated with the target column value is determined from a primary index table of a primary index; and acquiring at least one target track segment data corresponding to the track query condition from the primary index table based on the primary index value associated with the target column value. The target track data corresponding to the track query condition is convenient to determine based on the target track segment data subsequently, and the query efficiency of the track data is improved.

In the process of scanning the secondary index values based on the track query condition, if the second index value (rowkey) of the track query condition satisfying the space-time range in the secondary index table is empty, the full-table scanning is performed, and the specific manner is as follows.

After the scanning of the secondary index based on the track query condition and the obtaining of the scanning result corresponding to the track query condition, the method further includes:

under the condition that the scanning result is empty, performing full-table scanning on a primary index table of the primary index and a secondary index table of the secondary index based on the track query condition to obtain a full-table scanning result;

determining at least one target track segment data corresponding to the track query condition based on the full-table scan result if the full-table scan result is not empty.

According to the above example, when the xz2t index value meeting the track query condition of the space-time range does not exist in the secondary index table, the data in the primary table of the primary index is subjected to full-table scanning based on the track query condition, and a full-table scanning result is obtained, and when the full-table scanning result is not null, that is, when the track segment data meeting the track query condition exists in the primary table, the track segment is determined to be at least one target track segment data corresponding to the track query condition.

In this specification, in the case that the scan result is empty, full-table scanning is performed on the primary index table of the primary index based on the track query condition, and in the case that the full-table scan result is not empty, at least one target track segment data corresponding to the track query condition is obtained. Therefore, the track data query diversity is provided, and the query efficiency of the track data is further improved.

Step three: and determining target track data corresponding to the track query condition based on each target track segment data.

The target trajectory data may be understood as trajectory data required by the trajectory query condition.

In the above example, after determining the track segment data corresponding to the track query condition based on the primary index and the secondary index, the track segment data may be directly used as the target track data, so as to obtain the target track data corresponding to the track query condition.

Further, in another case, when the complete track data needs to be obtained, the index construction method provided in this specification may further perform a review based on attribute information of the target track segment data, so as to obtain one or more track segment data belonging to the same complete track data as the target track segment data, and splice the one or more track segment data with the target track segment data, so as to obtain the complete track data corresponding to the track query condition, where a specific implementation manner is as follows.

The determining target track data corresponding to the track query condition based on each target track segment data includes:

acquiring target attribute information in the second attribute information of each target track segment data;

determining at least one piece of review track segment data corresponding to the target attribute information according to the primary index and the secondary index;

and splicing the data of each target track segment with the data of each back-check track segment to obtain the target track data corresponding to the track query condition.

In the above example, since the complete track is divided into track segments for storage, if the complete track needs to be returned during query, the review needs to be performed based on the track segments. Based on this, the attribute information such as object id, segment _ start _ time, segment _ index and the like in the second attribute information of the target track segment data is obtained, and according to the above-mentioned manner of determining the target track segment data corresponding to the track query condition, according to the primary index and the OTS secondary index, one or more track segment data corresponding to the attribute information is determined, and the splicing is performed based on the one or more track segment data and the segment index attribute information of the target track segment data, so as to obtain complete track data, and the complete track data is used as the target track data corresponding to the track query condition. And realizing efficient splicing and returning of a complete track during query.

In the index construction method provided by the present specification, after receiving a track query condition, a track management system on the cloud; and determining at least one target track segment data corresponding to the track query condition according to the primary index and the secondary index, and determining the target track data corresponding to the track query condition based on each target track segment data. The track query performance is improved, and the data transmission in the simple query process is effectively reduced by calculating the push-down part to the storage layer of the multimode database.

During specific implementation, the track management system on the cloud can be conveniently accessed to a Spark big data analysis framework, so that Spark is used for complex query, simple query is pushed down to a multi-mode database, and the efficiency of track query is further improved.

The following description further explains the index construction method provided in this specification with reference to fig. 3 by taking an example of application of the index construction method to a scene based on index query trajectory data. Fig. 3 shows a processing flow chart of an index construction method applied to a scene of querying track data based on an index according to an embodiment of the present specification, and in practical application, after table construction and data writing are performed according to the structures in tables 2 to 6, a spatiotemporal range query related to a track can be performed by using a secondary index. The method specifically comprises the following steps:

step 302: and (5) analyzing the query statement.

Specifically, the query condition received by the trace management system on the cloud may be a complex condition, including both the spatiotemporal scope and other attributes, and thus the query statement needs to be parsed.

Step 304: a query plan based on a spatiotemporal index is generated.

Specifically, a query plan based on a spatiotemporal index is generated based on the parsed query statement. Spatio-temporal coding ranges are generated from the spatio-temporal ranges of the query and specify a spatio-temporal index column (i.e., a column of xz2t index values) that scans the main table.

Step 306: and judging whether the index scanning range of the query plan is not empty.

Specifically, for the index column (xz 2t index value column) with the established secondary index, the multimodal database automatically scans the secondary index corresponding to the index column, including performing scan filtering on the xz2t index value in the secondary index table corresponding to the index column.

Meanwhile, it is determined whether the index scan range of the query plan is non-empty, that is, whether an index value corresponding to the spatio-temporal range required by the query condition exists in the xz2t index values of the secondary index table, if yes, step 308 is executed; if not, go to step 310.

Step 308: and scanning and filtering the rowkey of the secondary index table.

Specifically, when there is an index value corresponding to the spatio-temporal range required by the query condition in the xz2t index values in the secondary index table, the rowkey column (xz 2t index value column) in the secondary index table is scanned and filtered to determine the xz2t index value satisfying the spatio-temporal range.

A column value corresponding to the xz2t index value is determined from the secondary index table, a rowkey value corresponding to the column value is determined from the primary table based on the column value, and track segment data corresponding to the query condition is determined based on the rowkey value corresponding to the column value.

Step 310: full table scanning.

Specifically, when no index value corresponding to the spatio-temporal range required by the query condition exists in the xz2t index values of the secondary index table, the primary table is scanned in a full table based on the query suite, so that track segment data corresponding to the query condition is obtained.

In practical applications, the full-table scan consumes a lot of computer processing resources, so the operation of performing the full-table scan should be avoided as much as possible.

Step 312: and judging whether a complete track needs to be returned or not.

Specifically, after track segment data corresponding to the query condition is obtained, it is necessary to determine whether the query plan needs to return a complete track; if yes, go to step 314, if no, determine that the query plan does not need to return the complete trajectory data, directly take the trajectory segment data as the target trajectory, and directly go to step 318.

Step 314: the OTS index column is reviewed.

Specifically, under the condition that the query plan needs to return complete track data, the OTS secondary index is constructed for rechecking, so that other track segment data belonging to the same complete track data as the track segment data are obtained.

Step 316: and (6) splicing the complete tracks.

Specifically, after data review, the obtained track segment data and other track segment data belonging to the same complete track data as the track segment data are subjected to track splicing, so that complete track data are obtained, and the complete track data are used as target data.

For example, since the trace is segmented into segment stores, if the whole trace needs to be returned during query, the trace needs to be reviewed. The OTS index is defined as: the moving object id (object id) + the entire track start Time (Time) + the track segment number (segment index). Wherein OT combinations can uniquely identify a track and OTs can uniquely identify a track segment. And designating OT during the back check, returning the segments belonging to the same track, and splicing the segments into a complete track result according to the serial number sequence.

Step 318: and (5) performing preliminary query results.

Specifically, the queried target trajectory data is used as a preliminary query result, and the preliminary query result is returned to the client.

Step 320: and judging whether a client filter exists.

Specifically, scanning the secondary index may return an inaccurate result (e.g., spatio-temporal fence query, spatio-temporal coding can only perform a coarse filtering function), and therefore, after sending the preliminary query result to the client, it needs to be determined whether the client has a filter, if so, step 322 is executed, and if not, step 324 is executed.

Step 322: and carrying out fine filtering on the transmitted data.

Specifically, under the condition that the client has the filter, the query result is subjected to secondary filtering, and the result after the secondary filtering is the final query result.

Step 324: and querying the result.

Specifically, in the case that the client does not have a filter, the client directly takes the received preliminary query result as a final query result.

Corresponding to the above method embodiment, the present specification further provides an index building apparatus embodiment, and fig. 4 shows a schematic structural diagram of an index building apparatus provided in an embodiment of the present specification. As shown in fig. 4, the apparatus includes:

a first determining module 402, configured to divide the received track data into at least two track segment data, and determine a primary index table and a primary index value of each track segment data according to each track segment data and corresponding first attribute information;

a first constructing module 404 configured to construct a primary index of each track segment data based on the primary index table and a primary index value;

a second determining module 406, configured to determine a secondary index value according to the second attribute information of each track segment data, and determine a secondary index table associated with the primary index based on the secondary index value;

a second constructing module 408 configured to construct a secondary index for each track segment data based on the primary index and a secondary index table associated with the primary index.

Optionally, the first determining module 402 is further configured to:

Optionally, the second determining module 406 is further configured to:

Optionally, the index building apparatus further includes a query module configured to:

receiving a track query condition;

determining at least one target track segment data corresponding to the track query condition according to the primary index and the secondary index;

and determining target track data corresponding to the track query condition based on each target track segment data.

Optionally, the query module is further configured to:

and under the condition that the scanning result is not empty, scanning the primary index based on the scanning result to obtain at least one target track segment data corresponding to the track query condition.

Optionally, the query module is further configured to:

coding the track query condition to obtain a track query code;

Optionally, the query module is further configured to:

The above is a schematic scheme of an index building apparatus of the present embodiment. It should be noted that the technical solution of the index building apparatus and the technical solution of the index building method belong to the same concept, and details that are not described in detail in the technical solution of the index building apparatus can be referred to the description of the technical solution of the index building method.

FIG. 5 illustrates a block diagram of a computing device 500 provided in accordance with one embodiment of the present description. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. Processor 520 is coupled to memory 510 via bus 530, and database 550 is used to store data.

Computing device 500 also includes access device 540, access device 540 enabling computing device 500 to communicate via one or more networks 560. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. The access device 540 may include one or more of any type of network interface (e.g., a Network Interface Card (NIC)) whether wired or wireless, such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a global microwave interconnect access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.

In one embodiment of the present description, the above-described components of computing device 500, as well as other components not shown in FIG. 5, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device architecture shown in FIG. 5 is for purposes of example only and is not limiting as to the scope of the present description. Those skilled in the art may add or replace other components as desired.

Computing device 500 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), mobile phone (e.g., smartphone), wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 500 may also be a mobile or stationary server.

Wherein the processor 520 is configured to execute computer-executable instructions that, when executed by the processor 520, implement the steps of any of the index construction methods.

The above is an illustrative scheme of a computing device of the present embodiment. It should be noted that the technical solution of the computing device and the technical solution of the index building method belong to the same concept, and details that are not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the index building method.

An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which when executed by a processor, implement the steps of the index building method described above.

The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the index building method, and details that are not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the index building method.

An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the index building method.

The above is an illustrative scheme of a computer program of the present embodiment. It should be noted that the schematic solution of the computer program and the technical solution of the index building method described above belong to the same concept, and details that are not described in detail in the schematic solution of the computer program can be referred to the description of the technical solution of the index building method described above.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.

It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present disclosure is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present disclosure. Further, those skilled in the art will appreciate that the embodiments described in this specification are presently considered to be preferred embodiments and that acts and modules are not necessarily required to be described in this specification.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

The preferred embodiments of the present specification disclosed above are intended only to aid in the description of the specification. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the specification and its practical application, to thereby enable others skilled in the art to best understand the specification and its practical application. The specification is limited only by the claims and their full scope and equivalents.

Claims

1. An index building method, comprising:

constructing a secondary index of each track segment data based on the primary index and a secondary index table associated with the primary index;

receiving a track query condition;

2. The index building method according to claim 1, wherein the determining a primary index table and a primary index value of each track segment data according to each track segment data and corresponding first attribute information comprises:

3. The index building method of claim 1, wherein determining a secondary index value according to the second attribute information of each track segment data comprises:

4. The index building method according to claim 1, wherein the determining a secondary index value according to the second attribute information of each track segment data further comprises:

5. The index building method of claim 1, the determining a secondary index table associated with the primary index based on the secondary index value, comprising:

6. The index construction method according to claim 1, wherein the determining at least one target track segment data corresponding to the track query condition according to the primary index and the secondary index comprises:

7. The index construction method according to claim 6, wherein the scanning the secondary index based on the track query condition to obtain a scanning result corresponding to the track query condition comprises:

coding the track query condition to obtain a track query code;

8. The index building method according to claim 6, wherein after the secondary index is scanned based on the track query condition and a scanning result corresponding to the track query condition is obtained, the method further comprises:

9. The index building method according to claim 6, wherein the scanning the primary index based on the scanning result to obtain at least one target track segment data corresponding to the track query condition comprises:

10. An index building apparatus comprising:

a second constructing module configured to construct a secondary index for each track segment data based on the primary index and a secondary index table associated with the primary index;

the query module is configured to receive a track query condition, determine at least one target track segment data corresponding to the track query condition according to the primary index and the secondary index, acquire target attribute information in second attribute information of each target track segment data, determine at least one review track segment data corresponding to the target attribute information according to the primary index and the secondary index, and splice each target track segment data with each review track segment data to acquire the target track data corresponding to the track query condition.

11. A computing device, comprising:

a memory and a processor;

the memory is configured to store computer-executable instructions and the processor is configured to execute the computer-executable instructions, which when executed by the processor, perform the steps of the index building method of any one of claims 1 to 9.

12. A computer-readable storage medium storing computer-executable instructions which, when executed by a processor, implement the steps of the index building method of any one of claims 1 to 9.