CN111832059A - Space big data management method and system based on cloud service - Google Patents
Space big data management method and system based on cloud service Download PDFInfo
- Publication number
- CN111832059A CN111832059A CN202010971217.2A CN202010971217A CN111832059A CN 111832059 A CN111832059 A CN 111832059A CN 202010971217 A CN202010971217 A CN 202010971217A CN 111832059 A CN111832059 A CN 111832059A
- Authority
- CN
- China
- Prior art keywords
- database
- spatial
- data
- information
- verification code
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/278—Data partitioning, e.g. horizontal or vertical partitioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6227—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database where protection concerns the structure of data, e.g. records, types, queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Bioethics (AREA)
- Data Mining & Analysis (AREA)
- Remote Sensing (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a space big data management method and a system based on cloud service, wherein the method comprises the following steps: constructing N spatial databases, wherein the N spatial databases comprise a first database, a second database and a database from the Nth database to the Nth database, and the databases respectively comprise different types of spatial big data; generating a first verification code according to the first database, wherein the first verification code corresponds to the first database one to one; generating a second verification code according to the second database and the first verification code; by parity of reasoning, generating an Nth verification code according to the Nth database and the (N-1) th verification code, and respectively copying and storing all databases and verification codes on M devices to obtain a spatial data set; and clustering and dividing the data information in the spatial data set, and storing the clustered data information in the N spatial databases respectively, so that the technical effects of improving the storage safety of spatial data and accurately classifying and storing the spatial data are achieved.
Description
Technical Field
The invention relates to the field of big data management, in particular to a space big data management method and system based on cloud service.
Background
The rapid development of mobile internet brings human beings into the age of big data in location. More and more Location Based Services (LBS) are integrated into daily life of people, provide Services such as point of interest query, social network real-time Location sharing, route planning and navigation, and provide great convenience for people. The safe sharing and releasing requirements of the big data of the position track cannot be kept away from the technical support of the position track privacy protection.
However, in the process of implementing the technical solution of the invention in the embodiments of the present application, the inventors of the present application find that the above-mentioned technology has at least the following technical problems:
the technical problems of unsafe space data storage and inaccurate classification exist in the prior art.
Disclosure of Invention
The embodiment of the application provides a cloud service-based spatial big data management method and system, solves the technical problems of unsafe spatial data storage and inaccurate classification in the prior art, and achieves the technical effects of improving the spatial data storage safety and accurately classifying and storing the spatial data.
In view of the foregoing problems, embodiments of the present application provide a method and a system for managing spatial big data based on cloud services.
In a first aspect, an embodiment of the present application provides a method for managing spatial big data based on a cloud service, where the method includes: constructing N spatial databases, wherein the N spatial databases comprise a first database, a second database and a database from the Nth database to the Nth database, and the databases respectively comprise different types of spatial big data; generating a first verification code according to the first database, wherein the first verification code corresponds to the first database one to one; generating a second verification code according to the second database and the first verification code; by analogy, generating an Nth verification code according to the Nth database and the Nth-1 verification code, wherein N is a natural number greater than 1; respectively copying and storing all databases and verification codes on M devices, wherein M is a natural number greater than 1; obtaining a spatial data set; and clustering and dividing the data information in the spatial data set, and respectively storing the clustered data information in the N spatial databases.
On the other hand, the application also provides a space big data management system based on cloud service, wherein the system comprises: the system comprises a first construction unit, a second construction unit and a third construction unit, wherein the first construction unit is used for constructing N spatial databases, wherein the N spatial databases comprise a first database, a second database and a database from the Nth database, and the databases respectively comprise spatial big data of different categories; a first obtaining unit, configured to generate a first verification code according to the first database, where the first verification code corresponds to the first database one to one; a second obtaining unit, configured to generate a second verification code according to the second database and the first verification code; by analogy, generating an Nth verification code according to the Nth database and the Nth-1 verification code, wherein N is a natural number greater than 1; the first saving unit is used for respectively copying and saving all databases and verification codes on M devices, wherein M is a natural number greater than 1; a third obtaining unit for obtaining a spatial data set; and the second storage unit is used for clustering and dividing the data information in the spatial data set and respectively storing the clustered data information in the N spatial databases.
In a third aspect, the present invention provides a space big data management system based on cloud services, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps of the method according to the first aspect when executing the program.
One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:
the method adopts the mode of encrypting the logic of the N constructed spatial databases based on block chain encryption and storing the spatial data sets in the spatial databases after clustering division, thereby achieving the technical effects of accurately classifying the spatial data and improving the security of the spatial data.
The foregoing description is only an overview of the technical solutions of the present application, and the present application can be implemented according to the content of the description in order to make the technical means of the present application more clearly understood, and the following detailed description of the present application is given in order to make the above and other objects, features, and advantages of the present application more clearly understandable.
Drawings
Fig. 1 is a schematic flowchart of a cloud service-based spatial big data management method according to an embodiment of the present application;
fig. 2 is a schematic flow chart illustrating clustering and partitioning of data information in a spatial data set in a spatial big data management method based on cloud services according to an embodiment of the present application;
fig. 3 is a schematic flow chart illustrating clustering and partitioning of data information in the spatial data set according to a polynomial regression algorithm in the cloud service-based spatial big data management method according to the embodiment of the present application;
fig. 4 is a diagram illustrating a first result judged in the cloud service-based space big data management method according to the embodiment of the present application
A flow chart showing whether all the numerical information of (1) is 0;
fig. 5 is a schematic flowchart of inputting size information x of data of the updated data information and spatial arrangement information Y of the data information into formula (1) and performing iterative computation in a cloud service-based spatial big data management method according to an embodiment of the present application;
fig. 6 is a schematic flowchart illustrating a process of respectively copying and storing all databases and verification codes in M devices in a cloud service-based spatial big data management method according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a space big data management system based on cloud services according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an exemplary electronic device according to an embodiment of the present application.
Description of reference numerals: a first building unit 11, a first obtaining unit 12, a second obtaining unit 13, a first saving unit 14, a third obtaining unit 15, a second saving unit 16, a bus 300, a receiver 301, a processor 302, a transmitter 303, a memory 304, a bus interface 306.
Detailed Description
The embodiment of the application provides a cloud service-based spatial big data management method and system, solves the technical problems of unsafe spatial data storage and inaccurate classification in the prior art, and achieves the technical effects of improving the spatial data storage safety and accurately classifying and storing the spatial data. Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are merely some embodiments of the present application and not all embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein.
Summary of the application
The rapid development of mobile internet brings human beings into the age of big data in location. More and more Location Based Services (LBS) are integrated into daily life of people, provide Services such as point of interest query, social network real-time Location sharing, route planning and navigation, and provide great convenience for people. The safe sharing and releasing requirements of the big data of the position track cannot be kept away from the technical support of the position track privacy protection. However, the technical problems of unsafe spatial data storage and inaccurate classification exist in the prior art.
In view of the above technical problems, the technical solution provided by the present application has the following general idea:
the embodiment of the application provides a space big data management method based on cloud service, and the method comprises the following steps: constructing N spatial databases, wherein the N spatial databases comprise a first database, a second database and a database from the Nth database to the Nth database, and the databases respectively comprise different types of spatial big data; generating a first verification code according to the first database, wherein the first verification code corresponds to the first database one to one; generating a second verification code according to the second database and the first verification code; by analogy, generating an Nth verification code according to the Nth database and the Nth-1 verification code, wherein N is a natural number greater than 1; respectively copying and storing all databases and verification codes on M devices, wherein M is a natural number greater than 1; obtaining a spatial data set; and clustering and dividing the data information in the spatial data set, and respectively storing the clustered data information in the N spatial databases.
Having thus described the general principles of the present application, various non-limiting embodiments thereof will now be described in detail with reference to the accompanying drawings.
Example one
As shown in fig. 1, an embodiment of the present application provides a cloud service-based space big data management method, where the method includes:
step S100: constructing N spatial databases, wherein the N spatial databases comprise a first database, a second database and a database from the Nth database to the Nth database, and the databases respectively comprise different types of spatial big data;
specifically, the spatial database is a database storing spatial data, and N storage databases are constructed, where N is a natural number greater than 1. And each database in the N databases stores different types of spatial big data information.
Step S200: generating a first verification code according to the first database, wherein the first verification code corresponds to the first database one to one;
in particular, the validation code is a unique identification code used to validate the database. And performing hash function calculation according to the first database to generate first verification codes corresponding to the first database one by one. The hash function may convert a data into a token that is closely related to each byte of the source data. The hash function also has a characteristic that it is difficult to find a reverse rule. And performing hash function calculation on the first database to obtain a unique corresponding first verification code, thereby ensuring the security of the database.
Step S300: generating a second verification code according to the second database and the first verification code; by analogy, generating an Nth verification code according to the Nth database and the Nth-1 verification code, wherein N is a natural number greater than 1;
in particular, the blockchain technique, also referred to as a distributed ledger technique, is an emerging technique in which several computing devices participate in "accounting" together, and maintain a complete distributed database together. The blockchain technology has been widely used in many fields due to its characteristics of decentralization, transparency, participation of each computing device in database records, and rapid data synchronization between computing devices. And taking the second database and the first verification code as a whole, performing hash function calculation to obtain a second verification code, and generating an Nth verification code according to the Nth database and the Nth-1 verification code in the same way. And forming a chain-shaped encryption mode to encrypt the verification codes and the databases through the association of the verification codes of each next database and the previous database, so that the verification code information cannot be easily tampered, and the technical effect of ensuring the safety of the databases and the verification codes is further achieved.
Step S400: respectively copying and storing all databases and verification codes on M devices, wherein M is a natural number greater than 1;
specifically, all databases and verification codes are respectively copied and stored on M devices, wherein the first database and the first verification code are stored on one device as a first block, the second database and the second verification code are stored on one device as a second block, the Nth database and the Nth verification code are stored on one device as an Nth block, when the data needs to be called, after each subsequent node receives the data stored by the previous node, the data is verified and stored through a common identification mechanism, and each block is connected in series through a hash function, so that the data is not easy to lose and damage, and the databases and the verification codes are encrypted through logic of a block chain, so that the safety of the databases and the verification codes is ensured.
Step S500: obtaining a spatial data set;
specifically, the spatial data set is a set of spatial data, and specifically includes: data size information X of the data information, and spatial arrangement information Y of the data information.
Step S600: and clustering and dividing the data information in the spatial data set, and respectively storing the clustered data information in the N spatial databases.
Specifically, the clustering division refers to classifying and dividing the data set, each division represents a category, fitting the variation trend of the data information through a polynomial model, clustering and dividing the data information with the similar trend of the data size information and the spatial arrangement information, and respectively storing the clustered and divided data information in N spatial databases. By means of clustering and dividing the data information, the technical effects of accurately classifying the spatial data and calling the spatial data are achieved.
As shown in fig. 2, in the clustering and partitioning the data information in the spatial data set, an embodiment S600 of the present application further includes:
step S610: obtaining data size information x of the data information in the spatial data set;
step S620: obtaining spatial arrangement information Y of the data information in the spatial data set;
step S630: and according to the size information x of the data and the spatial arrangement information Y of the data information, clustering and dividing the data information in the spatial data set according to a polynomial regression algorithm.
Specifically, the data size information refers to information about a storage space occupied by data. The spatial arrangement information refers to information related to a spatial position represented by data, and for example, the data information may include longitude and latitude information and height information in a three-dimensional space. The polynomial regression algorithm is to perform trend fitting on the data through a polynomial regression model, perform predictive classification on the trend of the data, and further obtain more accurate spatial data classification.
Further, the data information in the spatial data set is clustered and partitioned according to a polynomial regression algorithm, where the formula is as follows:
wherein p is a known parameter;
the above-mentioned
Information of different categories;
specifically, the formula (1) is a regression model of x and Y, and is called polynomial regression. The P represents number information of different categories, the
For different kinds of information, for example, when the kind information P =2, the regression polynomial is a parabolic equation, and the polynomial regression becomes a parabolic regression. And clustering and dividing the data information through the polynomial regression algorithm to achieve the technical effect of obtaining more accurate spatial data classification.
As shown in fig. 3, the clustering and partitioning the data information in the spatial data set according to a polynomial regression algorithm, in step S630 of this embodiment of the present application, further includes:
step S631: inputting the size information x of the data and the spatial arrangement information Y of the data information into formula (1) to obtain a first result, wherein the first result is
The numerical value information of (a);
step S632: judging the first result
Whether all the numerical information of (2) is 0;
step S633: if in the first result
The numerical information of (2) is not all 0, and a first regression function is obtained;
step S634: obtaining a first cluster according to the first regression function and the data information;
step S635: and constructing the first database according to the first cluster.
In particular, by examining p coefficients
Whether all the numerical values are 0 is judged to determine whether the clustering is successful. If all the values are 0, the linear regression is not obvious, namely the clustering is not successful; when in the first result
If not all the numerical information of (2) is 0, the linear regression is considered to be significant, namely the clustering is successful, and a first regression function is obtained. And according to the first regression function with obvious linear regression and the data information, carrying out cluster division on the data information with similar data size information and spatial arrangement information trends to obtain a first cluster, and constructing a first database according to the first cluster. Subjecting the data to the polynomial regression algorithmThe information is clustered and divided, so that the technical effect of obtaining more accurate spatial data classification is achieved.
As shown in fig. 4, the determination is made in the first result
Whether or not all the numerical information of (2) are 0, embodiment S632 of the present application further includes:
step S6321: if in the first result
All the numerical information of (2) is 0, and the regression of the formula (1) is determined to be not significant;
step S6322: updating data information in the spatial data set;
step S6323: inputting the size information x of the updated data information and the spatial arrangement information Y of the data information into the formula (1) and carrying out iterative computation until the updated data information and the spatial arrangement information Y of the data information are obtained
Not all of the numerical information of (2) is 0;
step S6324: obtaining a second regression function;
step S6325: obtaining a second cluster according to the second regression function and the data information;
step S6326: and constructing the second database according to the second cluster.
In particular, when in the first result
All the numerical information of (1) is 0, the regression of the formula (1) is determined to be not significant, the data information is discrete at the moment, the data information cannot be divided into the same category, the data is accurately updated in order to accurately classify the data information, the updated data information is subjected to iterative computation, the position of the central point of the data is changed at the moment, the computation is continuously carried out according to the steps, and the function is gradually reducedError value until the objective function value converges, i.e.:
and obtaining a second regression function to obtain a final clustering result, gradually optimizing the clustering result, and continuously redistributing the target data set to each clustering center to obtain an optimal solution, wherein the numerical information of the target data set is not all 0. And then, obtaining a second cluster according to a second regression function and the data information, and constructing the second database according to the second cluster.
As shown in fig. 5, the step S6323 of this embodiment of the present invention further includes inputting the size information x of the updated data of the data information and the spatial arrangement information Y of the data information into equation (1) and performing iterative computation:
step S63231: the data information still can not be obtained after iterative computation
When the numerical value information of (2) is not all 0 results, the data information is used as first data of other classifications;
step S63232: through the method, second data and third data of other classifications are obtained till the Nth data;
step S63233: obtaining a first check code uniquely corresponding to the first data, generating a second check code according to the second data and the first check code, and generating an Nth check code according to the Nth data and the (N-1) th check code by analogy;
step S63234: and storing the data and the check code on the M devices.
Specifically, when the data cannot be clustered and divided, the data is classified into scattered data, the scattered data is encrypted based on block chain logic, when the scattered data needs to be called, after each subsequent node receives data stored in a previous node, the data is verified and stored through a common identification mechanism, and each data is connected in series through a hash function, so that the data and a verification code are not easily lost and damaged, and the scattered data can be accurately and safely stored through a block chain logic encryption processing mode.
As shown in fig. 6, the step S400 further includes, by copying and storing all the databases and the verification codes on M devices, respectively:
step S410: the method comprises the steps of obtaining storage time of a first database, wherein the storage time of the first database is the time required to be recorded by the first database;
step S420: obtaining P pieces of entrusting equipment according to the storage time of the first database, wherein the P pieces of entrusting equipment are equipment with storage authority formed after the M pieces of equipment entrust the storage authority to other equipment;
step S430: obtaining a first device from the P entrusted devices;
step S440: sending the storage right to the first device, wherein the first device executes the storage right of the first database;
step S450: the device that obtains the most interest from the P trusted devices is the first device.
Specifically, in order to ensure the security of data in a database, the storage time of the first database is obtained, the storage time of the first database is the time required for storing the first database, a storage device which cannot store the first database at a specified time is excluded, and the storage of the first database is entrusted to P entrusted devices. The first device which obtains the most rights and interests among the P entrusted devices is distributed according to the amount of rights and interests occupied by the M devices, and the probability of obtaining the storage right is higher when the more rights and interests are occupied. Therefore, the data in the database is accurately, safely and quickly recorded according to the amount of rights and interests of the equipment, and the technical effect of ensuring the safety of the data is achieved.
To sum up, the cloud service-based spatial big data management method and system provided by the embodiment of the application have the following technical effects:
1. the method adopts the mode of encrypting the logic of the N constructed spatial databases based on block chain encryption and storing the spatial data sets in the spatial databases after clustering division, thereby achieving the technical effects of accurately classifying the spatial data and improving the security of the spatial data.
2. Because a clustering division mode based on a polynomial regression model is adopted, the error value of the function is gradually reduced until the target function value is converged, a final clustering result is obtained, the clustering result is gradually optimized, and the target data set is continuously redistributed to each clustering center to obtain an optimal solution. And respectively storing the clustered and divided data information in N spatial databases, so as to achieve the technical effects of accurately classifying the spatial data and calling the accurate spatial data.
3. Because a mode of entrusting the database recording right according to a block chain entrusting mechanism is adopted, the probability of obtaining the storage right is higher for the equipment with more occupied rights and interests, so that the data in the database is accurately, safely and quickly recorded according to the amount of rights and interests distributed by the equipment, and the technical effect of ensuring the safety of the data is achieved.
Example two
Based on the same inventive concept as the cloud service-based space big data management method in the foregoing embodiment, the present invention further provides a cloud service-based space big data management system, as shown in fig. 7, the system includes:
the first construction unit 11 is used for constructing N spatial databases, wherein the N spatial databases comprise a first database, a second database and up to an Nth database, and the databases respectively contain different types of spatial big data;
a first obtaining unit 12, where the first obtaining unit 12 is configured to generate a first verification code according to the first database, where the first verification code is in one-to-one correspondence with the first database;
a second obtaining unit 13, where the second obtaining unit 13 is configured to generate a second verification code according to the second database and the first verification code; by analogy, generating an Nth verification code according to the Nth database and the Nth-1 verification code, wherein N is a natural number greater than 1;
a first storage unit 14, where the first storage unit 14 is configured to copy and store all databases and verification codes on M devices, where M is a natural number greater than 1;
a third obtaining unit 15, the third obtaining unit 15 being configured to obtain a spatial data set;
a second storing unit 16, where the second storing unit 16 is configured to perform cluster division on the data information in the spatial data set, and store the clustered data information in the N spatial databases respectively.
Further, the system further comprises:
a fourth obtaining unit, configured to obtain data size information x of the data information in the spatial data set;
a fifth obtaining unit configured to obtain spatial arrangement information Y of the data information in the spatial data set;
and the first dividing unit is used for clustering and dividing the data information in the spatial data set according to a polynomial regression algorithm according to the size information x of the data and the spatial arrangement information Y of the data information.
Further, the system further comprises:
a sixth obtaining unit configured to obtain equation (1) as follows:
(1) wherein p is a known parameter; the above-mentioned
Information of different categories;
following a normal distribution of N (0,
）。
further, the system further comprises:
a seventh obtaining unit configured to input size information x of the data and spatial arrangement information Y of the data information into equation (1) to obtain a first result, the first result being
The numerical value information of (a);
a first judgment unit for judging the first result
Whether all the numerical information of (2) is 0;
an eighth obtaining unit for if in the first result
The numerical information of (2) is not all 0, and a first regression function is obtained;
a ninth obtaining unit, configured to obtain a first cluster according to the first regression function and the data information;
a second constructing unit, configured to construct the first database according to the first cluster.
Further, the system further comprises:
a first determination unit for determining if one of the first results is included in the first result
All the numerical information of (2) is 0, and the regression of the formula (1) is determined to be not significant;
a tenth obtaining unit, configured to update data information in the spatial data set;
a first input unit forInputting the size information x of the updated data information and the spatial arrangement information Y of the data information into the formula (1) and carrying out iterative calculation until the updated data information is updated to the spatial arrangement information Y
Not all of the numerical information of (2) is 0;
an eleventh obtaining unit configured to obtain a second regression function;
a twelfth obtaining unit, configured to obtain a second cluster according to the second regression function and the data information;
a third constructing unit, configured to construct the second database according to the second cluster.
Further, the system further comprises:
a thirteenth obtaining unit, configured to obtain a storage time of the first database, where the storage time of the first database indicates a time that needs to be recorded by the first database;
a fourteenth obtaining unit, configured to obtain P pieces of entrusting equipment according to the storage time of the first database, where the P pieces of entrusting equipment are equipment with storage authority, where the equipment with storage authority is formed after the M pieces of equipment entrust the storage authority to other equipment;
a fifteenth obtaining unit configured to obtain a first device from the P delegating devices;
a first sending unit, configured to send the storage right to the first device, where the first device executes the storage right of the first database.
Further, the system further comprises:
a sixteenth obtaining unit, configured to obtain, as the first device, a device with the most rights from the P delegating devices.
Various changes and specific examples of the cloud service-based spatial big data management method in the first embodiment of fig. 1 are also applicable to the cloud service-based spatial big data management system in this embodiment, and through the foregoing detailed description of the cloud service-based spatial big data management method, those skilled in the art can clearly know the implementation method of the cloud service-based spatial big data management system in this embodiment, so for the brevity of the description, detailed descriptions are not provided here.
Exemplary electronic device
The electronic apparatus of the embodiment of the present application is described below with reference to fig. 8.
Fig. 8 illustrates a schematic structural diagram of an electronic device according to an embodiment of the present application.
Based on the inventive concept of the cloud service-based space big data management method in the foregoing embodiments, the present invention further provides a cloud service-based space big data management system, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps of any one of the foregoing cloud service-based space big data management methods.
Where in fig. 8 a bus architecture (represented by bus 300), bus 300 may include any number of interconnected buses and bridges, bus 300 linking together various circuits including one or more processors, represented by processor 302, and memory, represented by memory 304. The bus 300 may also link together various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface 306 provides an interface between the bus 300 and the receiver 301 and transmitter 303. The receiver 301 and the transmitter 303 may be the same element, i.e., a transceiver, providing a means for communicating with various other systems over a transmission medium.
The processor 302 is responsible for managing the bus 300 and general processing, and the memory 304 may be used for storing data used by the processor 302 in performing operations.
The embodiment of the invention provides a space big data management method based on cloud service, which comprises the following steps: constructing N spatial databases, wherein the N spatial databases comprise a first database, a second database and a database from the Nth database to the Nth database, and the databases respectively comprise different types of spatial big data; generating a first verification code according to the first database, wherein the first verification code corresponds to the first database one to one; generating a second verification code according to the second database and the first verification code; by analogy, generating an Nth verification code according to the Nth database and the Nth-1 verification code, wherein N is a natural number greater than 1; respectively copying and storing all databases and verification codes on M devices, wherein M is a natural number greater than 1; obtaining a spatial data set; and clustering and dividing the data information in the spatial data set, and respectively storing the clustered data information in the N spatial databases. The technical problems of unsafe space data storage and inaccurate classification in the prior art are solved, and the technical effects of improving the safety of space data storage and accurately classifying and storing the space data are achieved.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a system for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction system which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (9)
1. A cloud service-based space big data management method comprises the following steps:
constructing N spatial databases, wherein the N spatial databases comprise a first database, a second database and a database from the Nth database to the Nth database, and the databases respectively comprise different types of spatial big data;
generating a first verification code according to the first database, wherein the first verification code corresponds to the first database one to one;
generating a second verification code according to the second database and the first verification code; by analogy, generating an Nth verification code according to the Nth database and the Nth-1 verification code, wherein N is a natural number greater than 1;
respectively copying and storing all databases and verification codes on M devices, wherein M is a natural number greater than 1;
obtaining a spatial data set;
and clustering and dividing the data information in the spatial data set, and respectively storing the clustered data information in the N spatial databases.
2. The method of claim 1, wherein the method comprises;
obtaining data size information x of the data information in the spatial data set;
obtaining spatial arrangement information Y of the data information in the spatial data set;
and according to the size information x of the data and the spatial arrangement information Y of the data information, clustering and dividing the data information in the spatial data set according to a polynomial regression algorithm.
3. The method of claim 2, wherein the clustering the data information in the spatial data set is performed according to a polynomial regression algorithm, wherein the formula is as follows:
wherein p is a known parameter;
the above-mentioned
Information of different categories;
4. the method of claim 3, wherein the method comprises:
inputting the size information x of the data and the spatial arrangement information Y of the data information into formula (1) to obtain a first result, wherein the first result is
The numerical value information of (a);
judging the first result
Whether all the numerical information of (2) is 0;
if in the first result
The numerical information of (2) is not all 0, and a first regression function is obtained;
obtaining a first cluster according to the first regression function and the data information;
and constructing the first database according to the first cluster.
5. The method of claim 4, wherein the method comprises:
if in the first result
All the numerical information of (2) is 0, and the regression of the formula (1) is determined to be not significant;
updating data information in the spatial data set;
inputting the size information x of the data of the updated data information and the spatial arrangement information Y of the data information into formula (1) and performing iterative computationUntil to
Not all of the numerical information of (2) is 0;
obtaining a second regression function;
obtaining a second cluster according to the second regression function and the data information;
and constructing the second database according to the second cluster.
6. The method of claim 1, wherein the method comprises:
obtaining the storage time of a first database, wherein the storage time of the first database represents the time required to be recorded by the first database;
obtaining P pieces of entrusting equipment according to the storage time of the first database, wherein the P pieces of entrusting equipment are equipment with storage authority formed after the M pieces of equipment entrust the storage authority to other equipment;
obtaining a first device from the P entrusted devices;
and sending the storage authority to the first equipment, and executing the storage authority of the first database by the first equipment.
7. The method of claim 6, wherein the method comprises:
the device that obtains the most interest from the P trusted devices is the first device.
8. A space big data management system based on cloud service, wherein the system comprises:
the system comprises a first construction unit, a second construction unit and a third construction unit, wherein the first construction unit is used for constructing N spatial databases, wherein the N spatial databases comprise a first database, a second database and a database from the Nth database, and the databases respectively comprise spatial big data of different categories;
a first obtaining unit, configured to generate a first verification code according to the first database, where the first verification code corresponds to the first database one to one;
a second obtaining unit, configured to generate a second verification code according to the second database and the first verification code; by analogy, generating an Nth verification code according to the Nth database and the Nth-1 verification code, wherein N is a natural number greater than 1;
the first saving unit is used for respectively copying and saving all databases and verification codes on M devices, wherein M is a natural number greater than 1;
a third obtaining unit for obtaining a spatial data set;
and the second storage unit is used for clustering and dividing the data information in the spatial data set and respectively storing the clustered data information in the N spatial databases.
9. A cloud services based space big data management system comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method of any one of claims 1 to 7 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010971217.2A CN111832059B (en) | 2020-09-16 | 2020-09-16 | Space big data management method and system based on cloud service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010971217.2A CN111832059B (en) | 2020-09-16 | 2020-09-16 | Space big data management method and system based on cloud service |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111832059A true CN111832059A (en) | 2020-10-27 |
| CN111832059B CN111832059B (en) | 2020-12-29 |
Family
ID=72918710
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010971217.2A Active CN111832059B (en) | 2020-09-16 | 2020-09-16 | Space big data management method and system based on cloud service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111832059B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112069527A (en) * | 2020-11-10 | 2020-12-11 | 大象慧云信息技术有限公司 | Tax control invoice protection method and system based on multiple safety protection measures |
| CN112287397A (en) * | 2020-12-28 | 2021-01-29 | 曜立科技(北京)有限公司 | System and method for improving and guaranteeing safety of patient information transmission |
| CN117454396A (en) * | 2023-10-24 | 2024-01-26 | 深圳市马博士网络科技有限公司 | Forced access control system and method for private cloud system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106330431A (en) * | 2016-08-29 | 2017-01-11 | 北京瑞卓喜投科技发展有限公司 | Data processing method, apparatus and system based on block chain technology |
| CN106777240A (en) * | 2016-12-27 | 2017-05-31 | 广州创显科教股份有限公司 | A kind of method for quickly retrieving and system based on Hash calculation |
| CN106973048A (en) * | 2017-03-21 | 2017-07-21 | 南京云创大数据科技股份有限公司 | A kind of method that intelligent hardware devices data are safely and fast stored on publicly-owned cloud service platform |
| CN110555138A (en) * | 2019-08-05 | 2019-12-10 | 慧镕电子系统工程股份有限公司 | hybrid cloud storage method under cloud computing architecture |
| WO2020010763A1 (en) * | 2018-07-12 | 2020-01-16 | 中国科学院深圳先进技术研究院 | Blockchain spatio-temporal data querying method and system, and electronic apparatus |
| US20200057991A1 (en) * | 2017-09-29 | 2020-02-20 | Orestis Afordakos | Electronic Logistics Control Platform For Parcel Transportation |
| CN111541567A (en) * | 2020-04-22 | 2020-08-14 | 彭小雪 | Artificial intelligence cloud computing big data management system and method |
-
2020
- 2020-09-16 CN CN202010971217.2A patent/CN111832059B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106330431A (en) * | 2016-08-29 | 2017-01-11 | 北京瑞卓喜投科技发展有限公司 | Data processing method, apparatus and system based on block chain technology |
| CN106777240A (en) * | 2016-12-27 | 2017-05-31 | 广州创显科教股份有限公司 | A kind of method for quickly retrieving and system based on Hash calculation |
| CN106973048A (en) * | 2017-03-21 | 2017-07-21 | 南京云创大数据科技股份有限公司 | A kind of method that intelligent hardware devices data are safely and fast stored on publicly-owned cloud service platform |
| US20200057991A1 (en) * | 2017-09-29 | 2020-02-20 | Orestis Afordakos | Electronic Logistics Control Platform For Parcel Transportation |
| WO2020010763A1 (en) * | 2018-07-12 | 2020-01-16 | 中国科学院深圳先进技术研究院 | Blockchain spatio-temporal data querying method and system, and electronic apparatus |
| CN110555138A (en) * | 2019-08-05 | 2019-12-10 | 慧镕电子系统工程股份有限公司 | hybrid cloud storage method under cloud computing architecture |
| CN111541567A (en) * | 2020-04-22 | 2020-08-14 | 彭小雪 | Artificial intelligence cloud computing big data management system and method |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112069527A (en) * | 2020-11-10 | 2020-12-11 | 大象慧云信息技术有限公司 | Tax control invoice protection method and system based on multiple safety protection measures |
| CN112287397A (en) * | 2020-12-28 | 2021-01-29 | 曜立科技(北京)有限公司 | System and method for improving and guaranteeing safety of patient information transmission |
| CN112287397B (en) * | 2020-12-28 | 2021-04-23 | 曜立科技(北京)有限公司 | System and method for improving and guaranteeing safety of patient information transmission |
| CN117454396A (en) * | 2023-10-24 | 2024-01-26 | 深圳市马博士网络科技有限公司 | Forced access control system and method for private cloud system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111832059B (en) | 2020-12-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111832059B (en) | Space big data management method and system based on cloud service | |
| US9229997B1 (en) | Embeddable cloud analytics | |
| CN108520181B (en) | Data model training method and device | |
| WO2020124317A1 (en) | Multi-access edge computing node with distributed ledger | |
| Jiang et al. | A method to determine generalized basic probability assignment in the open world | |
| CN115943394A (en) | Method, device and system for secure longitudinal federal learning | |
| US11228444B2 (en) | Tracking provenance of digital data | |
| CN114072820A (en) | Executing machine learning models | |
| CN111885153B (en) | Block chain-based data acquisition method, device, computer equipment and storage medium | |
| US20220067570A1 (en) | Training machine learning models with training data | |
| CN112084218A (en) | Cloud data management method and system based on block chain | |
| Jing et al. | A Universal neighbourhood rough sets model for knowledge discovering from incomplete heterogeneous data | |
| US20210319353A1 (en) | Verification of stochastic gradient descent | |
| JP6410932B2 (en) | Embedded cloud analytics | |
| US20230252416A1 (en) | Apparatuses and methods for linking action data to an immutable sequential listing identifier of a user | |
| Huang et al. | Semantic integration of sensor knowledge on artificial internet of things | |
| Khan et al. | Tread: privacy preserving incentivized connected vehicle mobility data storage on interplanetary-file-system-enabled blockchain | |
| US11620580B2 (en) | Methods and systems for probabilistic filtering of candidate intervention representations | |
| US11636185B2 (en) | AI governance using tamper proof model metrics | |
| John Joseph et al. | A novel trust-scoring system using trustability co-efficient of variation for identification of secure agent platforms | |
| Altay et al. | Chaos numbers based a new representation scheme for evolutionary computation: Applications in evolutionary association rule mining | |
| CN114611127B (en) | Database data security management system | |
| EP3945478A1 (en) | Method and system for managing a plurality of networks of vehicles | |
| CN116414875A (en) | Data processing apparatus and data processing method | |
| JP2023520634A (en) | Maintaining contextual integrity |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20210629 Address after: 242000 room 2105, building 2, Xuancheng Zhigu Software Industrial Park, 188 Xiangshan Road, Xuanzhou District, Xuancheng City, Anhui Province Patentee after: Anhui Yucheng Data Technology Co.,Ltd. Address before: No. 242-2, 11, Haidian Road, Beijing Patentee before: Beijing Changlong iFLYTEK Technology Co.,Ltd. |
|
| TR01 | Transfer of patent right |