CN110765192A - GIS data management and processing method based on cloud platform - Google Patents
GIS data management and processing method based on cloud platform Download PDFInfo
- Publication number
- CN110765192A CN110765192A CN201910995548.7A CN201910995548A CN110765192A CN 110765192 A CN110765192 A CN 110765192A CN 201910995548 A CN201910995548 A CN 201910995548A CN 110765192 A CN110765192 A CN 110765192A
- Authority
- CN
- China
- Prior art keywords
- data
- gis
- cloud platform
- database
- gis data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/282—Hierarchical databases, e.g. IMS, LDAP data stores or Lotus Notes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/256—Integrating or interfacing systems involving database management systems in federated or virtual databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
- H04L63/083—Network architectures or network communication protocols for network security for authentication of entities using passwords
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2221/00—Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/21—Indexing scheme relating to G06F21/00 and subgroups addressing additional information or applications relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/2141—Access rights, e.g. capability lists, access control lists, access tables, access matrices
Abstract
The invention discloses a GIS data management and processing method based on a cloud platform, which comprises the following specific steps: building a cloud platform database for storing GIS data; the method comprises the steps of constructing a cloud platform database storage framework capable of realizing automatic layering of data blocks, dividing GIS data into three layers of hot data, warm data and cold data according to the number of times of data access, enabling the hot data to be migrated into an uppermost layer SSD of the cloud platform storage framework, and migrating the cold data into a lowermost layer SATA disk of the cloud platform storage framework, so as to realize cold and hot balance of the GIS data; according to different types of GIS data, identity authentication is carried out on users accessing the GIS data, and corresponding GIS data are encrypted, so that data sharing is achieved on the premise that the GIS data in the cloud platform database are protected.
Description
Technical Field
The invention relates to the field of application of GIS data and cloud platform combination, in particular to a GIS data management and processing method based on a cloud platform.
Background
The rapid development of information technologies such as GIS and cloud computing technologies powerfully promotes the informatization process of the urban planning industry, and a good software and hardware architecture provides a basic guarantee for the application of the information technologies.
Because GIS data processing capacity is large, users can only use in the intranet environment. The core data can be downloaded to the client, so that the safety risk of sensitive data leakage exists, and the problems that the software and hardware system of the GIS system is high in cost, dispersed in resources, complex in management and the like exist.
Because the planning and designing unit generally adopts the single machine mode management in the GIS application process at present, a large amount of GIS data can disperse in each design team, bring great management pressure for the design department, be difficult to carry out unified management and maintenance, more importantly because technical staff's level is uneven, lead to the accuracy of data input to be difficult to grasp, go on to investigate after the problem and maintain the degree of difficulty very big, seriously influenced work efficiency.
The GIS plays a key role in the urban planning field as an important technology in the urban planning, can effectively reduce the fault of the urban planning and improve the accuracy of the urban planning, however, in the urban planning practice, due to the lack of scientific and unified data analysis and study and judgment technology, the planning information can be lost and distorted in the transmission process, and the urban planning can have larger fault and error. Therefore, the GIS geographic information technology is actively utilized, the geographic information database system is actively constructed, the collection and the arrangement of a plurality of information in the urban planning field are realized, and the centralization of information management is comprehensively promoted.
Urban and rural planning relates to a dynamic process of urban development aspects, GIS data can be dynamically adjusted and maintained according to economic and social development and planning implementation conditions, so that the work is a long-lasting special work, but due to multi-source data difference processing, different departments are disjointed in specific engineering design, and finally metadata meeting warehousing standards are difficult to form, so that the effectiveness, integrity, normalization and availability of data results are difficult to synchronously update and share the content on the same information platform.
Although virtualization technologies based on cloud computing are becoming mature, cloud desktop systems are also widely used, and users in a cloud platform can complete various work of a GIS system in a low-cost and efficient manner through the cloud desktop systems.
However, in the process of combining the cloud platform and the GIS data system, in the process of uniformly calling virtualized resources by multiple systems or data encryption, besides the problems of complex maintenance, user dispersion, poor cooperativity and the like, the problem of high resource occupancy rate also exists in the encryption process of GIS data in the virtualized environment, which restricts the further development of the requirement combination of the cloud desktop and the GIS system.
The space data storage of the GIS has the characteristics of high capacity, high concurrency and fast growth, the software virtualization has the problems of high software cost, poor data safety and the like, on one hand, the traditional GIS workstation is relatively fixed in hardware resource allocation, is not beneficial to realizing the dynamic configuration and the full utilization of resources, and has low utilization rate of hardware resources; on the other hand, the traditional GIS workstation solution has no independent display card resource allocated independently, so that the situation of extremely unsmooth operation can occur even in the very simple two-dimensional graphic image design, and the performance requirement of a user on the design can not be met.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a GIS data management and processing method based on a cloud platform, which can realize GIS data balanced storage and realize data resource protection and sharing.
In order to achieve the purpose, the invention adopts the following technical scheme:
a GIS data management and processing method based on a cloud platform comprises the following specific steps:
building a cloud platform database for storing GIS data;
the method comprises the steps of constructing a cloud platform database storage framework capable of realizing automatic layering of data blocks, dividing GIS data into three layers of hot data, warm data and cold data according to the number of times of data access, enabling the hot data to be migrated into an uppermost SSD of the cloud platform storage framework, migrating the cold data into a lowermost SATA disk of the cloud platform storage framework, storing the warm data in a fiber channel disk, and realizing cold and hot balance of the GIS data;
according to different types of GIS data, identity authentication is carried out on users accessing the GIS data, and corresponding GIS data are encrypted, so that data sharing is achieved on the premise that the GIS data in the cloud platform database are protected.
Further, the specific steps of building a cloud platform database for storing GIS data are as follows: an Orale large database and an ArcSDE spatial database engine are adopted, and the database is divided into three databases of a basic database, a process database and a result database according to functions;
the basic database supports data sources of various forms, can be accessed from a plurality of heterogeneous data sources, and supports data association access across databases; the ArcSDE spatial database engine can automatically optimize the merging and conversion process, and avoids performance loss caused by repeated data fetching, repeated association and repeated landing;
the process library formulates a data conversion standard and a calling rule according to the basic data collected by the basic library, compiles a data resource catalog, establishes a GIS data center, comprehensively manages and daily updates GIS data, processes the data and formulates a unified shared database, so that the data processing process is more efficient;
the result library is used as a core database of the GIS data center after integrating, classifying and processing the data in each basic library, and supports various planning services; the achievement library also takes the basic library as a surface and the process library as a line, collects business data dispersed in GIS data of the basic library and the process library, and realizes development and operation without delay.
Further, the data processing supports batch creation, data sampling and multi-person collaborative definition.
Further, the method for constructing the cloud platform database storage architecture capable of realizing automatic layering of the data blocks comprises the following specific steps:
s1: dividing GIS data in a cloud platform database into different data blocks according to data types, and simultaneously storing at least one copy in each data block;
s2: counting the access frequency of each data block;
s3: when the accessed frequency of the data block is higher than a hot data threshold value, the data block is judged to be hot data, the data block needing to be migrated is inquired, the position of the data block is found, the data block is migrated and stored in a copy pool in the SSD; if the accessed frequency of the data block is lower than the cold data threshold value, judging that the data block is cold data, and storing the data block in an archiving backup pool in the SATA disk; if the frequency of access of the data block is between the threshold frequency of access of cold data and hot data, the data is stored in the fibre channel disk.
Further, the period for counting the access frequency of each data block is to monitor and migrate the data block every 128 MB.
Further, the identity authentication of the user accessing the GIS data includes the specific steps of: adopting Microsoft active directory AD, GIS data users and third party middleware to realize user identity authentication for accessing GIS data;
a third-party management middleware for user authority distribution, user identity authentication, user information extraction and user authority extraction is arranged on the cloud platform;
the third-party management middleware divides users into a system administrator, a central administrator, a department administrator and a common user, simultaneously sets management groups of the system administrator, the central administrator, the department administrator and the common user, and sends a user name and a password for accessing GIS data to the third-party management middleware;
the third-party management middleware accesses the active directory AD by using a Kerberos protocol and sends user name and password information to a domain controller of the active directory AD to complete identity authentication;
the domain controller returns the verification result to a third-party management middleware, and the third-party management middleware extracts the GIS data operation authority of the user from an authorized database to complete the association among the database, the GIS data and the Windows Server;
GIS data users access various GIS data resources in the cloud platform database through a 128-bit RSA token unique to the VPN and a third-party management middleware.
Further, encrypting the corresponding GIS data is to directly embed a data encryption engine into the virtualization system, and specifically includes:
the data encryption virtual machine protects all virtual machines under one physical machine by deploying a special safe virtual device without installing a data encryption client on each virtual machine;
virtual machine compaction agent: the method comprises the steps that the privilege state information of each virtual machine is accessed by utilizing an API (application programming interface), wherein the privilege state information comprises a memory, a state and network communication flow of the virtual machine, and a comprehensive safety encryption function comprising antivirus, a firewall, IDS/IPS and system integrity monitoring is provided for each virtual machine;
a virtualization management module: the system comprises a data encryption virtual machine, a data encryption engine and a data processing module, wherein the data encryption virtual machine and the virtual machine are used for supporting the data encryption virtual machine and simplifying communication to be carried out on a virtualization management layer, monitoring file activity of the data encryption virtual machine, informing the data encryption engine, scanning by the data encryption engine and returning disposal information; when data encryption is required, the existing encryption and decryption tools are used to specify operations to be performed while affecting encryption and decryption operations in the data encryption virtual machine.
The invention has the beneficial effects that: the cloud platform database built by the invention is divided into three databases, namely a basic database, a process database and a result database, GIS data processing and management are realized, data encryption and data sharing are realized, cross-industry department and cross-unit collaborative design work is guaranteed, GIS data in the cloud platform database are accurately read, the accuracy of city planning is guaranteed, the work effect is greatly improved, all GIS data are stored in the cloud platform database, various pieces of running information and data reading of the wind can be well performed through user identity authentication, and the data security is improved by adopting a data encryption means.
Drawings
FIG. 1 is a schematic diagram of a cloud platform database;
FIG. 2 is a schematic diagram of a GIS data storage strategy;
FIG. 3 is a schematic diagram of identity authentication for accessing GIS data;
FIG. 4 is a schematic diagram of GIS data encryption management logic;
FIG. 5 is a schematic diagram of GIS data encryption of the present invention.
Detailed Description
The invention will be further described with reference to the accompanying drawings and the detailed description below:
example one
A GIS data management and processing method based on a cloud platform comprises the following specific steps:
the method comprises the following steps: building a cloud platform database for storing GIS data;
the cloud platform virtualizes a large amount of hardware resources and database applications into a huge resource pool, provides flexible storage and computing resource allocation for the operation of the cloud platform and uses a database, and the cloud platform database adopts an Orale large database and an ArcSDE spatial database engine and is divided into three databases of a basic database, a process database and a result database according to functions;
as shown in fig. 1, the base database supports various forms of data sources, can be accessed from multiple heterogeneous data sources, and supports data association access across databases; the ArcSDE spatial database engine can automatically optimize the merging and conversion process, and avoids performance loss caused by repeated data fetching, repeated association and repeated landing;
the process library formulates a data conversion standard and a calling rule according to basic data collected by the basic library, compiles a data resource catalog, analyzes, cleans, classifies, converts, extracts and fuses GIS data to form various data warehouses, such as a planning data warehouse, a basic space data warehouse, a remote sensing current situation data warehouse and a provincial and municipal collection data warehouse, formulates metadata and the data resource catalog to form a map data result, namely, a GIS data center is built, GIS data comprehensive management and daily update and data processing are performed, and a unified shared database is formulated, so that the data processing process is more efficient;
the result library is used as a core database of the GIS data center after integrating, classifying and processing the data in each basic library, and supports various planning services; the result library also takes the basic library as a surface, takes the process library as a line, collects service data dispersed in GIS data of the basic library and the process library, realizes development and operation without delay, supports batch creation, data sampling and multi-person collaborative definition by data processing, and forms the result library with a map function, service management, background management and mobile display on the basis of the GIS data.
The cloud platform database is expanded from the original two-stage linkage of 'result library-basic library' to the three-stage linkage of 'result library-process library-basic library', the cloud platform database design is synchronously completed during linkage, one-stop data processing service is provided, unified management and maintenance are realized, and the data accuracy is improved.
The cloud platform database adopts a mode of combining a B/S (browser/server) structure and a C/S (client/server) structure. The C/S mode is used for basic library manufacturing and process library management, operates in a local area network, and meets the working requirements of high database performance and high data safety. The B/S mode is used for a result library, directly faces to users and meets the requirements of high continuity and linkage of services. The C/S mode copies the processed 'basic library' result data to the B/S part through manual work, and supports the business application of the B/S; and the B/S downloads the uploaded data and the service linkage data and then leads the data back to the C/S for data processing. And the data of the B/S and the C/S are mutually supported to form a data flow loop and are updated in an iterative manner.
Step two: the method comprises the steps of building a cloud platform database storage framework capable of realizing automatic layering of data blocks, dividing GIS data into three layers of hot data, warm data and cold data according to the number of times of data access, enabling the hot data to be migrated into an uppermost SSD of the cloud platform storage framework, migrating the cold data into a lowermost SATA disk of the cloud platform storage framework, storing the warm data in an optical fiber channel disk, realizing cold and hot balance of the GIS data, and really reducing the pressure of each GIS data storage load so as to construct a lightweight, high-response and extensible cloud data database storage framework; fig. 2 is a schematic diagram showing a mobile hierarchy of GIS data.
The method for constructing the cloud platform database storage architecture capable of realizing automatic layering of the data blocks comprises the following specific steps:
s1: dividing GIS data in a cloud platform database into fragments according to data types, wherein each fragment comprises a plurality of data blocks, and simultaneously storing a plurality of copies of each data block;
s2: counting the access frequency of each data block by using Adaptive Optimization in 3 PAR; wherein Adaptive Optimization in 3PAR is a granular, policy driven, automated hierarchical storage software solution that can optimize the service levels of enterprises and cloud data centers at as low a cost as possible while increasing flexibility and minimizing risk; adaptive Optimization is interpreted as Adaptive Optimization;
s3: when the access frequency of the data block is higher than a hot data threshold value, the data block is judged to be hot data, the data block needing to be migrated is inquired, the position of the data block is utilized and found, the data block is migrated and stored in a copy pool in an SSD, and the hot data generally needs to be accessed immediately under the environment with high performance, high availability and high requirement; if the accessed frequency of the data block is lower than the cold data threshold value, judging that the data block is cold data, wherein the cold data is usually accessed for a few times and is usually used for filing and backup, and storing the data block in a filing and backup pool in an SATA disk; if the access frequency of the data block is between the access frequency threshold values of the cold data and the hot data, storing the data in a near-line or on-line backup environment, and enabling a user to quickly access the data, wherein the access frequency is less;
the statistical period of the access frequency of each data block by Adaptive Optimization in 3PAR is that the data block is monitored and migrated every 128MB, the monitoring records of the history are comprehensively judged, and relatively stable hot or cold data are migrated.
The GIS data is stored in a cold-hot classification mode, the hot data with high updating frequency is stored in the SSD, the cold data with low updating frequency is stored in the SATA disk, the erasing frequency of each storage particle can be balanced, and the service life of the storage is greatly prolonged.
Step three: according to different types of GIS data, identity authentication is carried out on users accessing the GIS data, and corresponding GIS data are encrypted, so that data sharing is achieved on the premise that the GIS data in the cloud platform database are protected.
As shown in fig. 3, the active directory AD for microsoft, the GIS data user and the third-party middleware are adopted to realize the user identity authentication for accessing the GIS data;
a third-party management middleware for user authority distribution, user identity authentication, user information extraction and user authority extraction is arranged on the cloud platform; the third party management middleware divides users into four color groups: a system administrator: the method is responsible for the distribution of the user permission of the cloud platform database; a central administrator: the system is responsible for the construction and management of all data base libraries and the verification and warehousing of data; department manager: the system is responsible for the assistant management of a basic library and a result library and the maintenance and management of a process library; the common user: the system is responsible for data processing, processing and sharing and is used for data support of various projects and researches.
The third-party management middleware sets a management group of a system administrator, a center administrator, a department administrator and a common user by using an active directory AD domain user, and each GIS data client sends a user name and a password to the third-party management middleware;
the third-party management middleware accesses the active directory AD by using a Kerberos protocol, sends the encrypted user name and password information to a domain controller of the active directory AD to complete identity verification, acquires an LDAP host and a domain name, and can bind an LDAP service after LDAP validity authentication, wherein the LDAP is a lightweight directory access protocol;
the domain controller returns the verification result to the third-party management middleware, the third-party management middleware extracts the GIS data operation authority of the user from the authorized database, the identity authentication successfully realizes qualified login, the association among the database, the GIS data and the Windows Server is completed, once the qualified login is realized, secondary authentication and login are not needed subsequently, and the user application interface can be directly accessed for operation;
GIS data users access various GIS data resources in a cloud platform database through a 128-bit RSA token which is unique to VPN through a third-party management middleware, a Java Filter technology is adopted between an application end and a client end as an interceptor, and each request of the users passes through the interceptor, so that the safety of the system is further ensured;
the active directory AD user is used as the only authentication user to provide user management for each GIS data client, the third party management middleware is used as a bridge to provide connection service for the active directory AD and each GIS data client, centralized management of user information and authority is achieved, each GIS data client does not need to pay attention to the user management problem of the third party management middleware, the third party management middleware provides a function issuing interface for the GIS data client to use, direct access of each GIS data client to the active directory AD is avoided, all requirements are completely met by calling the third party management middleware to access the active directory AD, the problems that GIS data access users are complex in maintenance, user management is dispersed, cooperativity between systems is poor, user information safety is poor and the like are solved, and the GIS data access method is suitable for being deployed and used in various heterogeneous environments.
As shown in fig. 4, which is a schematic diagram of an encryption management logic of the present invention, the encryption management mainly aims at data acquisition encryption, data analysis processing encryption, result archiving and publishing encryption, and secret-related information management encryption, and the data acquisition encryption includes vectorization software encryption, such as shp, gdh, and the like; importing encryption, such as DWG, DXE and the like, into CAD software; database encryption, such as dbf, mdb, xls; raster image encryption, such as tif, jpg; the analysis processing encryption comprises transparent encryption for workstation operation, transparent encryption for tool box import and export, and transparent encryption for database injection; the achievement filing and releasing comprises decryption approval, authorized data encryption used outside and gateway encryption; the external use is authorized, and comprises the steps of sending out, packaging and encrypting, installing an offline client and a USB flash disk plug-and-play client; the gateway encryption comprises the encryption of a trusted mail gateway and a GIS database server; the secret-related information management encryption comprises security domain division, authorization management, restrictive use and monitoring audit; the safety domain division comprises division according to departments, division according to security levels and division in production stages; the authorization management comprises read-only, editing, printing and application decryption; the restrictive use comprises outsourcing, borrowing and sending, permission application and permission recovery, the restrictive mode comprises people limitation, time limitation and frequency limitation, and the monitoring and auditing comprise pipe separation and log report forms.
As shown in fig. 5, encrypting the corresponding GIS data to directly embed the data encryption engine into the virtualization system specifically includes:
the data encryption virtual machine protects all virtual machines under one physical machine by deploying a special safe virtual device without installing a data encryption client on each virtual machine;
virtual machine compaction agent: the method comprises the steps that the privilege state information of each virtual machine is accessed by utilizing an API (application programming interface), wherein the privilege state information comprises a memory, a state and network communication flow of the virtual machine, and a comprehensive safety encryption function comprising antivirus, a firewall, IDS/IPS and system integrity monitoring is provided for each virtual machine;
a virtualization management module: the system comprises a data encryption virtual machine, a data encryption engine and a data processing module, wherein the data encryption virtual machine and the virtual machine are used for supporting the data encryption virtual machine and simplifying communication to be carried out on a virtualization management layer, monitoring file activity of the data encryption virtual machine, informing the data encryption engine, scanning by the data encryption engine and returning disposal information; when data encryption is required, the existing encryption and decryption tools are used to specify operations to be performed while affecting encryption and decryption operations in the data encryption virtual machine.
Various other modifications and changes may be made by those skilled in the art based on the above-described technical solutions and concepts, and all such modifications and changes should fall within the scope of the claims of the present invention.
Claims (7)
1. A GIS data management and processing method based on a cloud platform is characterized by comprising the following specific steps:
building a cloud platform database for storing GIS data;
the method comprises the steps of constructing a cloud platform database storage framework capable of realizing automatic layering of data blocks, dividing GIS data into three layers of hot data, warm data and cold data according to the number of times of data access, enabling the hot data to be migrated into an uppermost SSD of the cloud platform storage framework, migrating the cold data into a lowermost SATA disk of the cloud platform storage framework, storing the warm data in an optical fiber channel disk, and realizing cold and hot balance of the GIS data;
according to different types of GIS data, identity authentication is carried out on users accessing the GIS data, and corresponding GIS data are encrypted, so that data sharing is achieved on the premise that the GIS data in the cloud platform database are protected.
2. The method for managing and processing the GIS data based on the cloud platform according to claim 1, wherein the concrete steps of building the cloud platform database for storing the GIS data are as follows: an Orale large database and an ArcSDE spatial database engine are adopted, and the database is divided into three databases of a basic database, a process database and a result database according to functions;
the basic database supports data sources of various forms, can be accessed from a plurality of heterogeneous data sources, and supports data association access across databases; the ArcSDE spatial database engine can automatically optimize the merging and conversion process, and avoids performance loss caused by repeated data fetching, repeated association and repeated landing;
the process library formulates a data conversion standard and a calling rule according to the basic data collected by the basic library, compiles a data resource catalog, establishes a GIS data center, comprehensively manages and daily updates GIS data, processes the data and formulates a unified shared database, so that the data processing process is more efficient;
the result library is used as a core database of the GIS data center after integrating, classifying and processing the data in each basic library, and supports various planning services; the achievement library also takes the basic library as a surface and the process library as a line, collects business data dispersed in GIS data of the basic library and the process library, and realizes development and operation without delay.
3. The cloud platform-based GIS data management and processing method of claim 2, wherein the data processing supports batch creation, data sampling, and multi-person collaborative definition.
4. The cloud platform-based GIS data management and processing method of claim 1, wherein the construction of the cloud platform database storage architecture capable of realizing automatic layering of data blocks comprises the following specific steps:
s1: dividing GIS data in a cloud platform database into different data blocks according to data types, and simultaneously storing at least one copy in each data block;
s2: counting the access frequency of each data block;
s3: when the accessed frequency of the data block is higher than a hot data threshold value, the data block is judged to be hot data, the data block needing to be migrated is inquired, the position of the data block is found, the data block is migrated and stored in a copy pool in the SSD; if the accessed frequency of the data block is lower than the cold data threshold value, judging that the data block is cold data, and storing the data block in an archiving backup pool in the SATA disk; if the frequency of access of the data block is between the threshold frequency of access of cold data and hot data, the data is stored in a near-line or on-line backup environment.
5. The cloud platform-based GIS data management and processing method of claim 4, wherein the period of counting the access frequency of each data block is to monitor and migrate the data block every 128 MB.
6. The method for managing and processing the GIS data based on the cloud platform according to claim 1, wherein the step of authenticating the identity of the user accessing the GIS data comprises the following steps: adopting Microsoft active directory AD, GIS data users and third party middleware to realize user identity authentication for accessing GIS data;
a third-party management middleware for user authority distribution, user identity authentication, user information extraction and user authority extraction is arranged on the cloud platform;
the third-party management middleware divides users into a system administrator, a central administrator, a department administrator and a common user, simultaneously sets management groups of the system administrator, the central administrator, the department administrator and the common user, and sends a user name and a password for accessing GIS data to the third-party management middleware;
the third-party management middleware accesses the active directory AD by using a Kerberos protocol and sends user name and password information to a domain controller of the active directory AD to complete identity authentication;
the domain controller returns the verification result to a third-party management middleware, and the third-party management middleware extracts the GIS data operation authority of the user from an authorized database to complete the association among the database, the GIS data and the Windows Server;
GIS data users access various GIS data resources in the cloud platform database through a 128-bit RSA token unique to the VPN and a third-party management middleware.
7. The cloud platform-based GIS data management and processing method of claim 1, wherein encrypting the corresponding GIS data is to directly embed a data encryption engine into a virtualization system, and specifically comprises:
the data encryption virtual machine protects all virtual machines under one physical machine by deploying a special safe virtual device without installing a data encryption client on each virtual machine;
virtual machine compaction agent: the method comprises the steps that the privilege state information of each virtual machine is accessed by utilizing an API (application programming interface), wherein the privilege state information comprises a memory, a state and network communication flow of the virtual machine, and a comprehensive safety encryption function comprising antivirus, a firewall, IDS/IPS and system integrity monitoring is provided for each virtual machine;
a virtualization management module: the system comprises a data encryption virtual machine, a data encryption engine and a data processing module, wherein the data encryption virtual machine and the virtual machine are used for supporting the data encryption virtual machine and simplifying communication to be carried out on a virtualization management layer, monitoring file activity of the data encryption virtual machine, informing the data encryption engine, scanning by the data encryption engine and returning disposal information; when data encryption is required, the existing encryption and decryption tools are used to specify operations to be performed while affecting encryption and decryption operations in the data encryption virtual machine.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910995548.7A CN110765192A (en) | 2019-10-18 | 2019-10-18 | GIS data management and processing method based on cloud platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910995548.7A CN110765192A (en) | 2019-10-18 | 2019-10-18 | GIS data management and processing method based on cloud platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110765192A true CN110765192A (en) | 2020-02-07 |
Family
ID=69332782
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910995548.7A Pending CN110765192A (en) | 2019-10-18 | 2019-10-18 | GIS data management and processing method based on cloud platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110765192A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111831222A (en) * | 2020-06-15 | 2020-10-27 | 新浪网技术(中国)有限公司 | Distributed object storage method and system |
| CN112291269A (en) * | 2020-11-30 | 2021-01-29 | 南方电网科学研究院有限责任公司 | Cloud desktop authentication method and device, electronic equipment and readable storage medium |
| CN113645287A (en) * | 2021-07-29 | 2021-11-12 | 腾讯科技（深圳）有限公司 | Automobile message storage method and device and automobile message storage system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN205620984U (en) * | 2016-04-01 | 2016-10-05 | 南京紫光云信息科技有限公司 | Data layering storage device |
| CN106685994A (en) * | 2017-02-22 | 2017-05-17 | 河海大学 | Cloud GIS (Geographic Information System) resource access control method based on GIS role grade permission |
| CN108200107A (en) * | 2018-03-30 | 2018-06-22 | 浙江网新恒天软件有限公司 | A kind of method that single-sign-on is realized in multi-domain environment |
| US20180248683A1 (en) * | 2017-02-27 | 2018-08-30 | United States of America, as Represented by the Se cretary of the Navy | System and Method for Automating Indirect Fire Protocol Using Fully Homomorphic Encryption |
-
2019
- 2019-10-18 CN CN201910995548.7A patent/CN110765192A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN205620984U (en) * | 2016-04-01 | 2016-10-05 | 南京紫光云信息科技有限公司 | Data layering storage device |
| CN106685994A (en) * | 2017-02-22 | 2017-05-17 | 河海大学 | Cloud GIS (Geographic Information System) resource access control method based on GIS role grade permission |
| US20180248683A1 (en) * | 2017-02-27 | 2018-08-30 | United States of America, as Represented by the Se cretary of the Navy | System and Method for Automating Indirect Fire Protocol Using Fully Homomorphic Encryption |
| CN108200107A (en) * | 2018-03-30 | 2018-06-22 | 浙江网新恒天软件有限公司 | A kind of method that single-sign-on is realized in multi-domain environment |
Non-Patent Citations (3)
| Title |
|---|
| 唐国祥: ""基于开源软件的云端农业资源信息服务研究"", 《中国优秀硕士学位论文全文数据库 农业科技辑》 * |
| 胡杰 等: ""华南热区农业气候资源时空数据库构建与集成分析"", 《热带农业科学》 * |
| 苏弘逸: ""云计算数据隐私保护方法的研究"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111831222A (en) * | 2020-06-15 | 2020-10-27 | 新浪网技术(中国)有限公司 | Distributed object storage method and system |
| CN112291269A (en) * | 2020-11-30 | 2021-01-29 | 南方电网科学研究院有限责任公司 | Cloud desktop authentication method and device, electronic equipment and readable storage medium |
| CN113645287A (en) * | 2021-07-29 | 2021-11-12 | 腾讯科技（深圳）有限公司 | Automobile message storage method and device and automobile message storage system |
| CN113645287B (en) * | 2021-07-29 | 2022-09-20 | 腾讯科技（深圳）有限公司 | Automobile message storage method and device and automobile message storage system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022126968A1 (en) | Micro-service access method, apparatus and device, and storage medium | |
| CN110543464B (en) | Big data platform applied to intelligent park and operation method | |
| CN103812939B (en) | Big data storage system | |
| CN102651775B (en) | Based on method, the equipment and system of many tenants shared object management of cloud computing | |
| CN107315776A (en) | A kind of data management system based on cloud computing | |
| CN110765192A (en) | GIS data management and processing method based on cloud platform | |
| CN104063756A (en) | Electric power utilization information remote control system | |
| CN102096684A (en) | Grid real-time data integrating and sharing platform | |
| CN101901315A (en) | Security isolation and monitoring management method of USB mobile storage media | |
| CN102917006B (en) | A kind of unified control and management method and device realizing computational resource and object permission | |
| CN107330580A (en) | Power marketing Base data platform construction method | |
| CN102611699A (en) | Method and system for access control in cloud operation system | |
| CN107800808A (en) | A kind of data-storage system based on Hadoop framework | |
| CN103886104A (en) | Distributed real-time database management system and implementation method applicable to electric system | |
| CN108132775A (en) | A kind of tenant manages system and method | |
| US11138328B2 (en) | Controlling access to secure information resources using rotational datasets and dynamically configurable data containers | |
| CN103209189A (en) | Distributed file system-based mobile cloud storage safety access control method | |
| CN106993049A (en) | A kind of General Aviation enterprise operation total management system based on cloud computing | |
| US20220108031A1 (en) | Cloud Core Architecture for Managing Data Privacy | |
| CN110474897A (en) | A kind of file permission management system | |
| US11711369B2 (en) | Controlling access to secure information resources using rotational datasets and dynamically configurable data containers | |
| CN111177480B (en) | Block chain directory archive system | |
| US11165777B2 (en) | Controlling access to secure information resources using rotational datasets and dynamically configurable data containers | |
| CN101014044A (en) | Network GIS system and data transmitting method thereof | |
| CN100561516C (en) | Network gridding service system of national geolopy spatial data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200207 |