SQLite-based spatial data local storage method, medium and electronic device
Technical Field
The invention relates to the field of spatial data storage, in particular to a local storage method, medium and electronic equipment for spatial data based on SQLite.
Background
With the large-scale application of a plurality of remote measuring means such as remote sensing, aerial survey and the like, a large amount of spatial data with geographical position information is generated. How to perform efficient storage for massive spatial data becomes a very important part in the utilization process of the spatial data. The traditional spatial data adopts a HDF storage format and a local storage scheme of storage service, and has the following problems:
1) it is inconvenient to perform fast migration of files. A registered storage service is required to be installed, and the normal operation of the storage service is required to be ensured in the migration process;
2) the data is vulnerable to corruption. The file is easily damaged and cannot be repaired when a power failure or program abnormality occurs. In the using and developing process, scenes that data opening fails and storage service needs to be restarted due to abnormal closing of a program often occur, and the problem that data damage cannot be repaired also occurs;
3) local data of the desktop platform cannot be directly used at the mobile terminal and needs to be converted.
How to better and efficiently store spatial data is a problem which needs to be solved urgently.
In view of this, in order to overcome the above technical defects, it is an urgent problem in the art to provide a spatial data local storage method based on SQLite.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a spatial data local storage method, medium and electronic equipment based on SQLite, which are convenient for rapid file migration, ensure the stability of data and realize efficient spatial data storage and accurate spatial query functions.
In order to solve the technical problems, the technical scheme of the invention is as follows: the SQLite-based spatial data local storage method is characterized by comprising the following steps of:
101. defining a data model: designing a spatial data storage table, and defining a data model;
102. data compression: compressing the target data to reduce the size of the physical file;
103. storing in SQLite database: storing the compressed data into an SQLite database;
104. inverse solution of data: reversely solving the coordinate value before data compression;
105. and (6) space query.
According to the technical scheme, the data model comprises a vector data model, a raster data model and a three-dimensional data model.
According to the above technical solution, the step 102 specifically includes the following sub-steps:
1021. acquiring a spatial data coordinate value;
1022. converting the coordinate value of the space data into a binary representation to obtain the effective byte digit M of the space data;
1023. confirming the number of bytes which can be compressed to obtain the number of bytes N which can be compressed finally;
1024. the highest bit of the coordinate value is added with a symbol and a prefix.
According to the above technical solution, the step 1021 specifically includes: acquiring floating point coordinate values x, y and z of spatial data, defining a shift factor p and a scale factor q according to a reference system, shifting and scaling the x, y and z coordinates respectively by using a formula (x-p) q to obtain a shaping value, and expressing the shaping value by using compact bytes.
According to the above technical solution, the step 1023 specifically is: obtaining the number N of bytes which can be compressed finally, wherein if M%8=0, N = M/8 + 1; if M%8 ≠ 0, then N = (M/8 + 1) + 1.
According to the above technical solution, the step 1024 specifically includes: one byte represented by 8 bits, from high to low: the first bit is a sign bit, 0 represents a positive number, and 1 represents a negative number; of the remaining bits, the next few 1's indicate that several bytes need to be read; the remaining number of bits is represented by 0.
According to the above technical scheme, the step 104 specifically comprises: in contrast to the process of data compression in step 102, the binary storage is decompressed to obtain a shaping value, and then coordinate transformation is performed by using the offset factor p and the scaling factor q to solve the coordinate value before compression.
According to the above technical solution, the step 105 specifically includes the following substeps:
1051. carrying out coarse filtering on the outsourcing rectangle and the query box recorded in the R tree index;
1052. and carrying out accurate filtering on the geometric data through a relational operation function.
A computer-readable medium, on which a computer program is stored which, when being executed by a processor, is adapted to carry out the method as set forth in the preceding claims.
An electronic device, comprising:
one or more processors;
memory having one or more programs stored thereon, which when executed by the one or more processors, perform the method as described in the previous claims.
Compared with the prior art, the invention has the following beneficial effects:
1) the method supports the rapid migration of the local file, does not depend on storage service, and is not easy to lose data and cause failure of an additional database;
2) the stability of the data is improved, the data is not easy to damage, and the data can be restored after power failure;
3) the data lossy compression is supported, and the geometric size is reduced by more than half;
4) the data encryption and decryption are supported, the performance loss is not too large, and the data uploading and querying efficiency is improved;
5) meanwhile, rough filtering and secondary accurate filtering are carried out through the geometric relation function of the R tree index, and the accuracy of space query is improved.
Drawings
FIG. 1 is a schematic flow chart of an embodiment of the present invention;
FIG. 2 is a schematic diagram of a data compression process according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating an exemplary shaping compression storage process according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating an example of determining geometric inclusion relationship according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating an example of determining a geometric intersection relationship according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Many aspects of the invention are better understood with reference to the following drawings. The components in the drawings are not necessarily to scale. Instead, emphasis is placed upon clearly illustrating the components of the present invention. Moreover, in the several views of the drawings, like reference numerals designate corresponding parts.
The word "exemplary" or "illustrative" as used herein means serving as an example, instance, or illustration. Any embodiment described herein as "exemplary" or "illustrative" is not necessarily to be construed as preferred or advantageous over other embodiments. All of the embodiments described below are exemplary embodiments provided to enable persons skilled in the art to make and use the examples of the disclosure and are not intended to limit the scope of the disclosure, which is defined by the claims. In other instances, well-known features and methods are described in detail so as not to obscure the invention. For purposes of the description herein, the terms "upper," "lower," "left," "right," "front," "rear," "vertical," "horizontal," and derivatives thereof shall relate to the invention as oriented in fig. 1. Furthermore, there is no intention to be bound by any expressed or implied theory presented in the preceding technical field, background, brief summary or the following detailed description. It is also to be understood that the specific devices and processes illustrated in the attached drawings, and described in the following specification are simply exemplary embodiments of the inventive concepts defined in the appended claims. Hence, specific dimensions and other physical characteristics relating to the embodiments disclosed herein are not to be considered as limiting, unless the claims expressly state otherwise.
Referring to fig. 1 to 5, the present invention discloses a spatial data local storage method based on SQLite, which is different in that the method comprises the following steps:
101. defining a data model: designing a storage table of spatial data in SQLite, and defining a data model;
102. data compression: compressing the target data to reduce the size of the physical file;
103. storing in SQLite database: storing the compressed data into an SQLite database;
104. inverse solution of data: reversely solving the coordinate value before data compression;
105. and (6) space query.
Specifically, the data model includes a vector data model (simple element class, annotation class, object class), a raster data model (raster directory, raster data set, mosaic data set), a three-dimensional data model, and other spatial data models.
Specifically, the step 102 specifically includes the following sub-steps:
1021. acquiring a spatial data coordinate value; acquiring floating point coordinate values x, y and z of spatial data, defining a shift factor p and a scaling factor q according to a reference system, shifting and scaling the x, y and z coordinates respectively by using a formula (x-p) q to obtain a shaping value, and expressing the shaping value by using compact bytes;
in the embodiment of the invention, floating point coordinate values x = -946437.976, y =2569735.554 and z =0 of spatial data are acquired, offset factors px = -5123200, py = -10002100 and pz =0 are defined according to a reference system, scaling factors qx =10000, qy =10000 and qz =1, and the x, y and z coordinates are respectively shifted and scaled by a formula (x-p) × q to become a shaping value. Obtaining: (x-px) × qx =41767620240, (y-py) × qy =125718355540, (z-pz) × qz = 0.
1022. Converting the coordinate value of the space data into a binary representation to obtain the effective byte digit M of the space data;
in the embodiment of the present invention, the coordinate values of the spatial data are converted into binary representation, taking x coordinate as an example, the conversion result of 41767620240 is 100110111001100010110100111010010000, and the effective byte bit number M = 36.
1023. Confirming the number of bytes which can be compressed to obtain the number of bytes N which can be compressed finally, wherein if M%8=0, N = M/8 + 1; if M%8 ≠ 0, then N = (M/8 + 1) + 1;
in the embodiment of the present invention, since 36%8 ≠ 0, N = (36/8 + 1) +1, which results in the number of bytes that can be compressed N = 6.
1024. The highest bit of the coordinate value is added with a sign and a prefix, 8 bits represent one byte, and the highest bit is added with the sign and the prefix, and the highest bit is added with the 8 bits from the high bit to the low bit: the first bit is a sign bit, 0 represents a positive number, and 1 represents a negative number; of the remaining bits, the next few 1's indicate that several bytes need to be read; the remaining number of bits is represented by 0;
in the embodiment of the present invention, the coordinate value 41767620240 is determined to be a positive number, so the first bit is 0; in its binary representation 5 bytes need to be read, so it is immediately 11111. The binary complement bit is 01111100.
In step 103, 011111000000100110111001100010110100111010010000 is stored as compressed data in the SQLite database.
Specifically, the step 104 specifically includes: in contrast to the process of data compression in step 102, the binary storage is decompressed to obtain a shaping value, and then coordinate transformation is performed by using the offset factor p and the scaling factor q to solve the coordinate value before compression.
In the embodiment of the invention, 011111000000100110111001100010110100111010010000 is decompressed and encoded, the integer value 41767620240 is decoded reversely, and the coordinate value before compression is decoded to be 946437.976 reversely according to the offset factor px = -5123200 and the scaling factor qx = 10000.
Specifically, the step 105 specifically includes the following sub-steps:
1051. carrying out coarse filtering on the outsourcing rectangle and the query box recorded in the R tree index;
1052. and carrying out accurate filtering on the geometric data through a relational operation function.
Thereby judging the relationship between the two geometric objects, including the query of the mutual containing and intersecting relationship between the two geometric objects.
1) The geometry comprises: as shown in fig. 4, if the outsourced rectangle recorded in the R-tree index is used to determine the complete containment relationship with the query box, the query result will be less than the geometric data. Therefore, the conditions included by the intersection are firstly used for rough filtering, and then the geometric relation function is used for precise filtering, so that accurate results can be obtained.
2) Geometric intersection: as shown in fig. 5, if the outsourced rectangle recorded in the R-tree index is used to determine the intersection relationship with the query box, the query result will have more geometric data. Therefore, accurate results can be obtained by performing coarse filtering by using the conditions included in the intersection and performing accurate filtering by using the geometric relation function.
In some possible embodiments, the aspects of the invention may also be implemented as a computer-readable medium, on which a computer program is stored, which, when being executed by a processor of an electronic device, is adapted to carry out the steps of the method according to various embodiments of the invention described in the above-mentioned solutions of the present description.
In some other embodiments of the present invention, the electronic device includes a memory storing one or more programs, and one or more processors, which when executing the one or more programs, are also configured to perform the above-described method steps.
It should be noted that: the above-mentioned medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take a variety of forms, including, but not limited to: an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the consumer electronic device, partly on a remote electronic device, or entirely on the remote electronic device or server. In the case of remote electronic devices, the remote electronic devices may be connected to the consumer electronic device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external electronic device (e.g., through the internet using an internet service provider).
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
In some possible embodiments, an electronic device according to embodiments of the present invention may include at least one processor, and at least one memory. Wherein the memory stores a program (computer program) which, when executed by the processor, causes the processor to perform the steps of the method according to various embodiments of the present invention described in the above-mentioned technical solutions of the present specification.
The spatial data stored by the traditional local storage scheme adopting the HDF storage format and the storage service has the problems of inconvenient file migration, easy data damage and the like, and when the data volume is very large, the efficient storage and utilization of the spatial data cannot be realized. The invention provides an SQLite-based method for locally storing spatial data, which realizes efficient spatial data storage and accurate spatial query functions by performing lossy compression on the spatial data and storing the spatial data in an SQLite database, gradually replaces a local HDF storage mode, facilitates the rapid migration of files and ensures the stability of the data.
The foregoing is a more detailed description of the present invention that is presented in conjunction with specific embodiments, and the practice of the invention is not to be considered limited to those descriptions. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.