CN111046000B - Government data exchange sharing oriented security supervision metadata organization method - Google Patents

Government data exchange sharing oriented security supervision metadata organization method Download PDF

Info

Publication number
CN111046000B
CN111046000B CN201911349516.6A CN201911349516A CN111046000B CN 111046000 B CN111046000 B CN 111046000B CN 201911349516 A CN201911349516 A CN 201911349516A CN 111046000 B CN111046000 B CN 111046000B
Authority
CN
China
Prior art keywords
data
information
metadata
government
resource
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911349516.6A
Other languages
Chinese (zh)
Other versions
CN111046000A (en
Inventor
申国伟
李明前
雷吉成
王巍
普艳红
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guizhou University
Harbin Engineering University
CETC Big Data Research Institute Co Ltd
Original Assignee
Guizhou University
Harbin Engineering University
CETC Big Data Research Institute Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guizhou University, Harbin Engineering University, CETC Big Data Research Institute Co Ltd filed Critical Guizhou University
Priority to CN201911349516.6A priority Critical patent/CN111046000B/en
Publication of CN111046000A publication Critical patent/CN111046000A/en
Application granted granted Critical
Publication of CN111046000B publication Critical patent/CN111046000B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • G06F16/164—File meta data generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/176—Support for shared access to files; File sharing support
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/26—Government or public services
    • G06Q50/265—Personal security, identity or safety
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The invention provides a government data exchange sharing oriented security supervision metadata organization method, which comprises the following steps: preprocessing data; step 2: constructing a supervision metadata organization model; step 3: and (5) supervising the metadata storage. The metadata organization method conforms to national relevant standard specifications, realizes the organization and association of multidimensional information through a data exchange sharing model diagram, a data lineage diagram and a data version diagram, can support the safety analysis of multidimensional supervision data such as data tracing, safety monitoring analysis and compliance audit, and lays a foundation for the effective organization, high-efficiency analysis and application of the multi-source heterogeneous safety supervision data.

Description

Government data exchange sharing oriented security supervision metadata organization method
Technical Field
The invention belongs to the technical field of government data exchange sharing safety supervision, and particularly relates to a safety supervision metadata organization method oriented to government data exchange sharing.
Background
The government data exchange sharing platform supports government data sharing and business collaboration of cross-level, cross-region, cross-system, cross-department and cross-business, and is a basic interconnection platform for government data exchange sharing. At present, a government department deploys a government data exchange sharing platform, and a security module is realized inside the platform, however, how to independently and safely monitor the exchange sharing platform under a third view is a problem which needs to be solved in order to ensure government data security.
The security regulatory targets of government data exchange sharing platforms typically include whether a regulatory user is exchanging data in accordance with a sharing exchange specification, whether the exchange-shared data is reasonably used by the user, and the like. In order to achieve the above objective, the security supervision platform may collect and analyze supervision data such as real-time exchange sharing information in network traffic, log data of the exchange sharing platform itself, resource catalogue, and exchange sharing platform information. How to realize efficient organization of massive supervision data under a unified model, supporting multidimensional analysis and utilization are key problems to be solved by a government data exchange sharing safety supervision platform.
Organization of large amounts of data is a typical problem in the field of large data. The learner puts forward a relevant data organization association model in the data lake, and also puts forward the idea of the data context service, however, the metadata information extracted by the government data exchange sharing security supervision platform must conform to relevant specifications and standards. Therefore, the invention provides an efficient metadata organization method supporting multidimensional analysis aiming at mass supervision data of a government data exchange sharing safety supervision platform.
Disclosure of Invention
The invention aims to solve the problems in the prior art and provides a security supervision metadata organization method oriented to government data exchange sharing.
The invention is realized by the following technical scheme, and provides a security supervision metadata organization method oriented to government data exchange sharing, which comprises the following steps:
step 1: preprocessing data;
the data preprocessing specifically comprises the following steps:
step 101, data acquisition, namely acquiring log data, information resource catalogs and flow restoration data obtained by passive unauthorized acquisition of a government data exchange sharing platform;
102, extracting metadata information, namely extracting metadata information of government information resources from log data and flow restoration data of a platform, wherein the metadata of the government information resources comprise core metadata and extension metadata;
step 103, data coding analysis is carried out, data analysis is carried out according to coding specifications, and government information resource classification codes and government information resource sequence codes are extracted; the government information resource classification code consists of a class, an item, a project and a detail; obtaining a 10-bit hash value corresponding to the data version through data content hash as a platform code in the safety supervision platform; orderly forming a government information resource classification code, a government information resource sequence code and a platform code into a resource data code, wherein the resource data code is used as a unique identification resource object in a metadata organization model;
step 2: constructing a supervision metadata organization model;
the supervision metadata organization model is constructed specifically as follows:
step 201, constructing a government data exchange sharing model diagram to record information on initial metadata exchange sharing protocol, model and application;
step 202, constructing any change information of the recorded data of the data version chart, comparing the hash values of the data resource objects in the platform by taking the hash values of the data contents as the data version information, if the hash values are consistent, determining that the versions are the same, and if the hash values are inconsistent, adding a new data version;
step 203, constructing a data lineage diagram to record exchange information of any resource data among government departments;
step 3: monitoring metadata storage;
the supervision metadata storage is specifically as follows:
and combining the supervision metadata organization model, organizing the recorded supervision data into graph data, and finally storing the graph data in a graph database to provide data support for upper-layer data version inquiry, data lineage tracking and tracing and data exchange department statistical analysis.
Further, the information resource catalogs are divided into three main categories: subject classification, industry classification, and service classification.
Further, the core metadata and the extension metadata specifically include information resource classification, information resource name, information resource code, information resource provider code, information resource abstract, information resource format, information item information, shared attribute, open attribute, update period, release date, and associated resource code.
Further, the government data exchange sharing adopts an exchange method based on an HTTP protocol, an FTP protocol or a large file sharing protocol, and the exchange method is used as a government data exchange sharing model.
Further, the exchange information includes essential information of the demander, essential information of the data provider, and exchange date.
The invention has the beneficial effects that:
aiming at large-scale safety supervision data of a government data exchange sharing platform, the invention provides a multidimensional safety supervision metadata organization model comprising user behaviors, exchange sharing models, data lineages and the like, which can support upper-layer applications such as long-period data traceability evidence collection, multidimensional data analysis and the like. The supervision metadata organization method for the government exchange sharing platform constructed by the invention does not need to modify the existing sharing exchange platform and has good expandability.
Drawings
FIG. 1 is a flow chart of a method for organizing security supervision metadata for government data exchange sharing according to the invention;
FIG. 2 is a schematic diagram of a supervisory metadata organization model;
FIG. 3 is a schematic diagram of a supervisory metadata organization model deployment framework;
FIG. 4 is a schematic diagram of an exemplary method of organizing administrative metadata;
FIG. 5 is a data historic lineage diagram.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The large-scale heterogeneous safety supervision data comprises multi-dimensional safety supervision information such as user behaviors, exchange sharing models, data lineages and the like, and the invention provides a safety supervision metadata organization method for government data exchange sharing, which supports multi-dimensional supervision data safety analysis such as data tracing, safety monitoring analysis, compliance audit and the like and lays a foundation for effective organization, efficient analysis and application of the multi-source heterogeneous safety supervision data.
In combination with fig. 1 and 2, the present invention proposes a method for organizing security supervision metadata for government data exchange sharing, the method comprising the following steps:
step 1: preprocessing data;
the data preprocessing specifically comprises the following steps:
step 101, data acquisition, namely acquiring log data, information resource catalogs and flow restoration data obtained by passive unauthorized acquisition of a government data exchange sharing platform; the information resource catalog is divided into three main categories: subject classification, industry classification, and service classification.
102, extracting metadata information, namely extracting metadata information of government information resources from log data and flow restoration data of a platform, wherein the metadata of the government information resources comprise core metadata and extension metadata; the core metadata and the extension metadata specifically include information resource classification, information resource name, information resource code, information resource provider code, information resource abstract, information resource format, information item information, shared attribute, open attribute, update period, release date, and associated resource code.
Step 103, data coding analysis is carried out, data analysis is carried out according to coding specifications, and government information resource classification codes and government information resource sequence codes are extracted; the government information resource classification code consists of a class, an item, a project and a detail; obtaining a 10-bit hash value corresponding to the data version through data content hash as a platform code in the safety supervision platform; orderly forming a government information resource classification code, a government information resource sequence code and a platform code into a resource data code, wherein the resource data code is used as a unique identification resource object in a metadata organization model;
step 2: constructing a supervision metadata organization model;
the supervision metadata organization model is constructed specifically as follows:
step 201, constructing a government data exchange sharing model diagram to record information on initial metadata exchange sharing protocol, model and application; government data exchange sharing adopts an exchange method based on HTTP protocol, FTP protocol or large file sharing protocol, and the exchange method is used as a government data exchange sharing model.
Step 202, constructing any change information of the recorded data of the data version chart, comparing the hash values of the data resource objects in the platform by taking the hash values of the data contents as the data version information, if the hash values are consistent, determining that the versions are the same, and if the hash values are inconsistent, adding a new data version;
step 203, constructing a data lineage diagram to record exchange information of any resource data among government departments; the exchange information includes essential information of the demander, essential information of the data provider, and exchange date. In the data exchange sharing process, the hash value of the data content is used as the basis, whether the hash value of the data resource in the platform is consistent or not is compared, if so, an exchange information is recorded, and if not, a version information and an exchange information are recorded.
Step 3: monitoring metadata storage;
the supervision metadata storage is specifically as follows:
and combining a supervision metadata organization model, organizing the recorded large-scale supervision data into graph data, and finally storing the graph data in a graph database to provide high-quality data support for upper-layer data version inquiry, data lineage tracking and tracing and data exchange department statistical analysis.
Aiming at a typical government data exchange sharing platform deployment scene, the invention adopts a bypass mode to deploy a supervision metadata organization model, and verifies the effectiveness of a security supervision metadata organization method facing government data exchange sharing.
Method for carrying out the same
According to the supervision metadata organization flow shown in fig. 1 in the invention, the method mainly comprises data preprocessing, metadata organization model construction and metadata model storage.
In step 101, log data of the exchange sharing platform, configuration information of the exchange sharing resource directory and the platform itself are collected. In addition, under the condition of unauthorized, the access flow of the exchange sharing platform is passively analyzed, and the real-time data of exchange sharing is obtained in a flow restoration mode.
And in step 102, extracting metadata information such as data resource codes, departments, sharing exchange protocols and the like according to the collected log data and flow restoration data. And analyzing the exchange shared resource catalog according to the Json format to acquire information such as open sharing constraint conditions, exchange range and the like of the data resource.
In step 103, the data resource coding information is analyzed according to the coding specification in the government information resource catalog compiling guide (trial run), and the government information resource classification code and the government information resource sequence code are extracted. And obtaining a 10-bit hash value corresponding to the data version through data content hash as a platform code of the platform. In the invention, the resource object code consists of three parts of a resource classification code, a resource sequence code and a platform code, and the resource data code is used as a unique identification resource object in a metadata organization model.
The basic data of the metadata model construction is obtained in the data preprocessing, and a foundation is laid for constructing the supervision metadata organization model. The constructed administrative metadata organization model is shown in fig. 2. In step 201, a data exchange sharing model diagram is formed according to the exchange methods based on HTTP protocol, FTP protocol, large file sharing protocol, etc. adopted by the current government data exchange sharing, and information about initial metadata exchange sharing protocol, model, application, etc. is recorded.
Any change information of the data is recorded by the data version map in step 202. In step 203, exchange information of any resource data between government departments is recorded, including basic information of a demand side, basic information of a data provider, exchange date and the like, so that information on the operation behavior of a user on the data is saved.
The multisource supervision metadata is organized into graph data through construction of a metadata organization model. In the invention, a Neo4j graph database is taken as a storage example, and efficient graph data storage is realized through cyber language. And the supervision metadata storage based on Neo4j provides data support for applications such as data tracing and user behavior rules.
(II) experiment verification
In order to verify the effectiveness of the invention, the system architecture, network deployment and the like of the existing government data exchange sharing platform are not required to be changed, and the existing exchange sharing platform is only required to push the supervision data, so that the invention has good expandability and reliability. The generalized administrative metadata organization method is applied as shown in fig. 3. Furthermore, the invention takes a specific government data exchange sharing platform as an application scene, and verifies the validity of the supervision metadata organization method in the government data exchange sharing safety supervision platform. The deployed network topology is shown in fig. 4. In the deployment, the image original network flow is subjected to unauthorized analysis, so that completeness of supervision data can be guaranteed, and the method of data pushing is adopted for resource catalog updating, log data and the like, so that the influence on the original platform is reduced as much as possible. An application of the data history lineage diagram is shown in fig. 5, which shows changes between different data versions, government information of the data exchanged, etc., and lays a foundation for upper-layer applications such as data tracing analysis, etc.
The foregoing has outlined a detailed description of a method for organizing security supervision metadata for government data exchange sharing, wherein specific examples are provided herein to illustrate the principles and embodiments of the present invention, and the above examples are provided to assist in understanding the method and core ideas of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.

Claims (5)

1. A security supervision metadata organization method oriented to government data exchange sharing is characterized in that: the method comprises the following steps:
step 1: preprocessing data;
the data preprocessing specifically comprises the following steps:
step 101, data acquisition, namely acquiring log data, information resource catalogs and flow restoration data obtained by passive unauthorized acquisition of a government data exchange sharing platform;
102, extracting metadata information, namely extracting metadata information of government information resources from log data and flow restoration data of a platform, wherein the metadata of the government information resources comprise core metadata and extension metadata;
step 103, data coding analysis is carried out, data analysis is carried out according to coding specifications, and government information resource classification codes and government information resource sequence codes are extracted; the government information resource classification code consists of a class, an item, a project and a detail; obtaining a 10-bit hash value corresponding to the data version through data content hash as a platform code in the safety supervision platform; orderly forming a government information resource classification code, a government information resource sequence code and a platform code into a resource data code, wherein the resource data code is used as a unique identification resource object in a metadata organization model;
step 2: constructing a supervision metadata organization model;
the supervision metadata organization model is constructed specifically as follows:
step 201, constructing a government data exchange sharing model diagram to record information on initial metadata exchange sharing protocol, model and application;
step 202, constructing any change information of the recorded data of the data version chart, comparing the hash values of the data resource objects in the platform by taking the hash values of the data contents as the data version information, if the hash values are consistent, determining that the versions are the same, and if the hash values are inconsistent, adding a new data version;
step 203, constructing a data lineage diagram to record exchange information of any resource data among government departments;
step 3: monitoring metadata storage;
the supervision metadata storage is specifically as follows:
and combining the supervision metadata organization model, organizing the recorded supervision data into graph data, and finally storing the graph data in a graph database to provide data support for upper-layer data version inquiry, data lineage tracking and tracing and data exchange department statistical analysis.
2. The method according to claim 1, characterized in that: the information resource catalog is divided into three main categories: subject classification, industry classification, and service classification.
3. The method according to claim 2, characterized in that: the core metadata and the extension metadata specifically include information resource classification, information resource name, information resource code, information resource provider code, information resource abstract, information resource format, information item information, shared attribute, open attribute, update period, release date, and associated resource code.
4. A method according to claim 3, characterized in that: government data exchange sharing adopts an exchange method based on HTTP protocol, FTP protocol or large file sharing protocol, and the exchange method is used as a government data exchange sharing model.
5. The method according to claim 4, wherein: the exchange information includes essential information of the demander, essential information of the data provider, and exchange date.
CN201911349516.6A 2019-12-24 2019-12-24 Government data exchange sharing oriented security supervision metadata organization method Active CN111046000B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911349516.6A CN111046000B (en) 2019-12-24 2019-12-24 Government data exchange sharing oriented security supervision metadata organization method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911349516.6A CN111046000B (en) 2019-12-24 2019-12-24 Government data exchange sharing oriented security supervision metadata organization method

Publications (2)

Publication Number Publication Date
CN111046000A CN111046000A (en) 2020-04-21
CN111046000B true CN111046000B (en) 2023-06-27

Family

ID=70239054

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911349516.6A Active CN111046000B (en) 2019-12-24 2019-12-24 Government data exchange sharing oriented security supervision metadata organization method

Country Status (1)

Country Link
CN (1) CN111046000B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111984717A (en) * 2020-08-26 2020-11-24 江西微博科技有限公司 Big data intelligent government affair platform information management method
CN112052467A (en) * 2020-09-01 2020-12-08 华中农业大学 Food safety big data sharing method
CN112486764A (en) * 2020-11-24 2021-03-12 云南电网有限责任公司信息中心 System and method for issuing monitoring and changing content analysis
CN112800171A (en) * 2020-12-29 2021-05-14 勤智数码科技股份有限公司 Method for quickly forming catalog classification based on administrative regional relation

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101916396A (en) * 2010-06-12 2010-12-15 江西天恒电讯科技有限公司 Digital home government system and method
CN109167764A (en) * 2018-08-17 2019-01-08 广州韵成通信科技有限公司 A kind of electronic government affairs system network aware analysis platform system
CN109791594A (en) * 2016-08-12 2019-05-21 Altr解决方案公司 Data are segmented in order to persistently be stored in multiple immutable data structures

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7246144B2 (en) * 2002-03-25 2007-07-17 Data Quality Solutions Method and system for managing a plurality of enterprise business systems
ES2835784T3 (en) * 2016-04-05 2021-06-23 Zamna Tech Limited Method and system for managing personal information within independent computer systems and digital networks

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101916396A (en) * 2010-06-12 2010-12-15 江西天恒电讯科技有限公司 Digital home government system and method
CN109791594A (en) * 2016-08-12 2019-05-21 Altr解决方案公司 Data are segmented in order to persistently be stored in multiple immutable data structures
CN109167764A (en) * 2018-08-17 2019-01-08 广州韵成通信科技有限公司 A kind of electronic government affairs system network aware analysis platform system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
莫富传 ; .贵阳市政府数据开放共享体系研究.图书情报研究.2019,(第02期),全文. *

Also Published As

Publication number Publication date
CN111046000A (en) 2020-04-21

Similar Documents

Publication Publication Date Title
CN111046000B (en) Government data exchange sharing oriented security supervision metadata organization method
CN106557991B (en) Voltage monitoring data platform
CN107945086A (en) A kind of big data resource management system applied to smart city
CN102073579B (en) Method for merging and optimizing audit events of Linux file system
CN108039959A (en) Situation Awareness method, system and the relevant apparatus of a kind of data
CN107818024A (en) A kind of request ID transmission methods and system based on spring blockers
CN105427193A (en) Device and method for big data analysis based on distributed time sequence data service
CN105049247A (en) Network safety log template extraction method and device
CN112632135A (en) Big data platform
CN103618652A (en) Audit and depth analysis system and audit and depth analysis method of business data
CN114548706A (en) Early warning method for business risk and related equipment
CN106407429A (en) File tracking method, device and system
US20190050435A1 (en) Object data association index system and methods for the construction and applications thereof
CN104504014A (en) Data processing method and device based on large data platform
CN111930726B (en) Off-line form-based grade protection evaluation data acquisition and analysis method and system
CN114153920A (en) Big data edge platform and method
CN111383130A (en) Full life cycle management and control platform, mobile operation and maintenance client and terminal
CN112883001A (en) Data processing method, device and medium based on marketing and distribution through data visualization platform
CN106777265B (en) Service data processing method and device
CN108073582A (en) A kind of Computational frame selection method and device
CN111538720B (en) Method and system for cleaning basic data of power industry
CN112612802A (en) Real-time data middlebox processing method, device and platform
CN102521378A (en) Real-time intrusion detection method based on data mining
CN112465480A (en) A real name system management system of labor affairs for building trade
CN107423035A (en) A kind of software development process product data management system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant