CN116028574A - Government full life cycle big data management system and method thereof - Google Patents

Government full life cycle big data management system and method thereof Download PDF

Info

Publication number
CN116028574A
CN116028574A CN202211635301.2A CN202211635301A CN116028574A CN 116028574 A CN116028574 A CN 116028574A CN 202211635301 A CN202211635301 A CN 202211635301A CN 116028574 A CN116028574 A CN 116028574A
Authority
CN
China
Prior art keywords
data
government
government affair
module
cluster
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211635301.2A
Other languages
Chinese (zh)
Inventor
桑海翎
兰小刚
曾伟波
刘见奋
陈檀晟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujian Zefu Software Co ltd
Original Assignee
Fujian Zefu Software Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujian Zefu Software Co ltd filed Critical Fujian Zefu Software Co ltd
Priority to CN202211635301.2A priority Critical patent/CN116028574A/en
Publication of CN116028574A publication Critical patent/CN116028574A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application provides a government full life cycle big data management system and a method thereof, wherein the system comprises: an application layer, a service layer, a data acquisition and storage layer and a cluster management module; the business layer comprises an algorithm library and a component library; the cluster management module is used for creating and managing a distributed big data cluster; the data acquisition and storage layer is used for acquiring government affair data from one or more data sources, carrying out multi-table association, modeling analysis and data cleaning processing on the acquired government affair data according to analysis purposes, and storing the acquired government affair data into each cluster node of the distributed big data cluster; the business layer is used for managing government affair data stored by each cluster node; and the application layer is used for displaying the analyzed government affair data in a terminal in a chart form. The system forms a unified data acquisition, supports various data processing, data model visual analysis and a rich visual assembly library, realizes the efficient management of full life cycle data, and fully mines the value of government affair data.

Description

Government full life cycle big data management system and method thereof
Technical Field
The application relates to the field of big data management, in particular to a government full life cycle big data management system and a method thereof.
Background
Government has a strong incentive for government big data governance, but is also facing serious capability challenges in the practice of data governance. The main manifestations are:
1. data clutter island clusters
For historical reasons, government institutions independently develop informationized construction of the unit, data are very dispersed logically and physically, a large amount of the same information is repeatedly collected and stored in different departments, and formats and contents are different. Therefore, in the government data collection process, there are phenomena such as "data chimney" forestation and "data island" clustering.
2. Difficult fusion and data security problem in data analysis process
In the data analysis process of various government decisions, structured data and unstructured data which need to be summarized comprehensively are mixed, the data quality is low, and the data standards are not uniform, so that the data analysis process is difficult to complete by using a uniform data model or a data algorithm. At the same time, government sensitive data fusion processes may also present a safety hazard.
Disclosure of Invention
In view of the above problems, the present application provides a government full life cycle big data management system and a method thereof, by constructing a government unified full life cycle data management system, so as to dynamically and efficiently manage government data in full life cycle.
To achieve the above object, the present inventors provide a government full life cycle big data management system comprising: an application layer, a service layer, a data acquisition and storage layer and a cluster management module; the business layer comprises an algorithm library and a component library;
the cluster management module is used for creating and managing a distributed big data cluster;
the data acquisition and storage layer is used for acquiring government affair data from one or more data sources, performing multi-table association, modeling analysis and data cleaning processing on the acquired government affair data according to analysis purposes, and storing the acquired government affair data to each cluster node of the distributed big data cluster;
the business layer is used for managing government affair data stored by each cluster node; the algorithm library is used for training and analyzing government affair data; the component library is used for providing various chart components corresponding to the analyzed government affair data;
and the application layer is used for displaying the analyzed government affair data in a terminal in a chart form.
Compared with the prior art, the technical scheme is characterized in that a government affair unified full life cycle data management system is constructed to form unified data acquisition, support various data processing, data model visual analysis and rich visual component libraries, realize the efficient management of the full life cycle data and fully mine the value of government affair data.
In some embodiments, the application layer includes a BI module and an API module;
the BI module is used for providing business intelligent functions based on the analyzed government affair data at the terminal;
and the API module is used for providing a service development interface corresponding to the component library at the terminal.
In some embodiments, the business layer further comprises a user management module and a menu management module;
the user management module is used for managing the user basic information and acquiring the user basic information from a third-party system;
and the menu management module is used for maintaining a system menu and managing user rights.
In some embodiments, the data acquisition storage layer comprises a data acquisition module and a data storage module;
the data acquisition module comprises a timing scheduling unit, a data acquisition unit and a data cleaning unit;
the timing scheduling unit is used for setting a government affair data acquisition period;
the data acquisition unit is used for acquiring government affair data in a structured, unstructured or file form, and carrying out multi-table association processing and modeling analysis on the selected government affair data according to an analysis purpose;
the data cleaning unit is used for converting government affair data output by the data acquisition unit into a data format required by a business layer;
and the data storage module is used for storing the government affair data output by the data cleaning unit to each cluster node and providing an external interface.
In some embodiments, the system further comprises a data security module for cryptographically protecting government data in the data storage module.
In some embodiments, the system further comprises: and the operation monitoring module is used for carrying out visual monitoring on the application layer and the service layer.
In some embodiments, the operation monitoring module includes a report monitoring unit, a model monitoring unit, a user behavior monitoring unit, an index monitoring unit, an access statistics unit, and a management log unit.
In some embodiments, the system further comprises: a cluster visualization module;
the cluster visualization module is used for performing visual monitoring and visual management on the cluster management module.
The management system in the embodiments realizes multi-department, multi-service and cross-system data circulation, data analysis and service capacity based on high-quality data on the premise of ensuring safety compliance, can support rapid design and development of government big data application scenes, and maximizes the value of releasing government data.
To achieve the above object, the present inventors also provide a government full life cycle big data management method based on the above government full life cycle big data management system, comprising the steps of:
collecting government affair data from one or more data sources;
according to the analysis purpose, performing multi-table association, modeling analysis and data cleaning processing on the government affair data, and storing the government affair data into each cluster node;
managing government affair data stored by each cluster node; wherein the managing includes training and analyzing government data;
and displaying the government affair data after analysis in a terminal in a chart form.
In some embodiments, the method further comprises:
visual monitoring of user behavior, charts, models and access is performed.
Compared with the prior art, the technical scheme is based on a government affair unified full life cycle data management system, and realizes government affair big data full life cycle management through data analysis processes of data acquisition, data storage, data calculation, data preprocessing, data analysis modeling, data visual display and the like, and centralized management and control operation is carried out on multi-source government affair data, so that the government affair data can be efficiently stored, processed and analyzed, and the value of the government affair data is fully mined.
The foregoing summary is merely an overview of the present application, and is provided to enable one of ordinary skill in the art to make more clear the present application and to be practiced according to the teachings of the present application and to make more readily understood the above-described and other objects, features and advantages of the present application, as well as by reference to the following detailed description and accompanying drawings.
Drawings
The drawings are only for purposes of illustrating the principles, implementations, applications, features, and effects of the present application and are not to be construed as limiting the application.
In the drawings of the specification:
FIG. 1 is a block diagram illustrating an architecture of a government full life cycle big data management system in accordance with certain embodiments;
FIG. 2 is a flow diagram illustrating a government full life cycle big data management method according to an embodiment;
FIG. 3 is a workflow diagram illustrating the K-Means algorithm in the government full life cycle big data management method according to the embodiments;
FIG. 4 is a schematic diagram illustrating the operation of the KNN algorithm in the government full life cycle big data management method described in the embodiments;
FIG. 5 is a business flow diagram illustrating a government full life cycle big data management method according to an embodiment;
FIG. 6 is a schematic diagram illustrating information interaction when the government full life cycle big data management system performs system data analysis based on SPARK according to an embodiment.
Detailed Description
In order to describe the possible application scenarios, technical principles, practical embodiments, and the like of the present application in detail, the following description is made with reference to the specific embodiments and the accompanying drawings. The embodiments described herein are only used to more clearly illustrate the technical solutions of the present application, and are therefore only used as examples and are not intended to limit the scope of protection of the present application.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearances of the phrase "in various places in the specification are not necessarily all referring to the same embodiment, nor are they particularly limited to independence or relevance from other embodiments. In principle, in the present application, as long as there is no technical contradiction or conflict, the technical features mentioned in the embodiments may be combined in any manner to form a corresponding implementable technical solution.
Unless defined otherwise, technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application pertains; the use of related terms herein is for the description of specific embodiments only and is not intended to limit the present application.
In the description of the present application, the term "and/or" is a representation for describing a logical relationship between objects, which means that there may be three relationships, e.g., a and/or B, representing: there are three cases, a, B, and both a and B. In addition, the character "/" herein generally indicates that the front-to-back associated object is an "or" logical relationship.
In this application, terms such as "first" and "second" are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual number, order, or sequence of such entities or operations.
Without further limitation, the use of the terms "comprising," "including," "having," or other like terms in this application is intended to cover a non-exclusive inclusion, such that a process, method, or article of manufacture that comprises a list of elements does not include additional elements but may include other elements not expressly listed or inherent to such process, method, or article of manufacture.
As in the understanding of the "examination guideline," the expressions "greater than", "less than", "exceeding", and the like are understood to exclude the present number in this application; the expressions "above", "below", "within" and the like are understood to include this number. Furthermore, in the description of the embodiments of the present application, the meaning of "a plurality of" is two or more (including two), and similarly, the expression "a plurality of" is also to be understood as such, for example, "a plurality of groups", "a plurality of" and the like, unless specifically defined otherwise.
In the description of the embodiments of the present application, spatially relative terms such as "center," "longitudinal," "transverse," "length," "width," "thickness," "up," "down," "front," "back," "left," "right," "vertical," "horizontal," "vertical," "top," "bottom," "inner," "outer," "clockwise," "counter-clockwise," "axial," "radial," "circumferential," etc., are used herein as terms of orientation or positional relationship based on the specific embodiments or figures, and are merely for convenience of description of the specific embodiments of the present application or ease of understanding of the reader, and do not indicate or imply that the devices or components referred to must have a particular position, a particular orientation, or be configured or operated in a particular orientation, and therefore are not to be construed as limiting of the embodiments of the present application.
Unless specifically stated or limited otherwise, in the description of the embodiments of the present application, the terms "mounted," "connected," "affixed," "disposed," and the like are to be construed broadly. For example, the "connection" may be a fixed connection, a detachable connection, or an integral arrangement; the device can be mechanically connected, electrically connected and communicated; it can be directly connected or indirectly connected through an intermediate medium; which may be a communication between two elements or an interaction between two elements. The specific meanings of the above terms in the embodiments of the present application can be understood by those skilled in the art to which the present application pertains according to the specific circumstances.
Referring to FIG. 1, FIG. 1 is a block diagram illustrating an architecture of a government full life cycle big data management system in accordance with certain embodiments.
The embodiment provides a government full life cycle big data management system, which relies on the support of a bottom big data cluster architecture to dynamically regulate and manage the cluster environment according to the data volume. The management system consists of an application layer, a service layer, a data acquisition and storage layer and a cluster management module, wherein the service layer comprises an algorithm library and a component library.
The cluster management module is used for creating and managing a distributed big data cluster, namely, a base cluster dynamic management function for realizing large data volume analysis.
In some specific embodiments, the cluster management module realizes functions of automatic deployment, automatic expansion, easy maintenance and the like of the container clusters based on the K8s container cluster management system; meanwhile, the DOCKER technology is adopted to realize the penetration of resources such as CPU, memory, disk and the like; dynamic telescoping management is achieved based on the SPARKONYARN cluster. In addition, the cluster management module also creates a Hadoop distributed computing framework to construct a parallel distributed system with high reliability and good expansibility.
The data acquisition and storage layer is mainly used for realizing diversified data acquisition, analysis and storage; the system is particularly used for collecting government affair data from one or more data sources, carrying out multi-table association, modeling analysis and data cleaning processing on the collected government affair data according to analysis purposes, and storing the acquired government affair data into each cluster node of the distributed big data cluster.
The data source support structure (Oracle, mySQL, sqlserver and other databases), unstructured (Mongodb, excle file import and other types), API interfaces (SOAP, REST and other types) and the like, and one or more data sources are selected for collecting government affair data according to analysis purposes, so that support for various data sources is realized, and user requirements are fully met.
In some embodiments, the data acquisition and storage layer specifically includes a data acquisition module and a data storage module;
the data acquisition module comprises a timing scheduling unit, a data acquisition unit and a data cleaning unit;
the timing scheduling unit is used for supporting flexibly setting a government affair data acquisition period;
the data acquisition unit is used for acquiring government affair data in other forms including structured, unstructured or files and the like, and carrying out multi-table association processing and modeling analysis on the selected government affair data according to analysis purposes;
the data cleaning unit is used for converting government affair data output by the data acquisition unit into a data format required by a business layer so as to realize data conversion into a data format required by a presentation end;
and the data storage module is used for storing the government affair data output by the data cleaning unit to each cluster node and providing an external interface, so that the cleaned government affair data is subjected to distributed storage management and the data structure function is provided for other services to flexibly call.
As a specific embodiment, the data storage module is based on the HDFS and HBASE distributed database technology, and is combined with the cluster management module to build a large-scale structured storage cluster, so that a high-reliability, high-performance, column-oriented and scalable distributed storage system is realized.
The business layer is mainly used for realizing business management of basic government affair data, namely managing government affair data stored by each cluster node;
the algorithm library in the business layer is internally provided with algorithms such as a data analysis algorithm, a data encryption algorithm and the like, and is used for training, analyzing and encrypting government affair data. Preferably, the data encryption algorithm is an asymmetric encryption algorithm; the data analysis algorithm comprises K-MEANS, KNN and the like to realize data training and analysis.
The component library is used for providing various chart components corresponding to the analyzed government affair data so as to support rich and various visual effects at a presentation end (namely a terminal), wherein the various visual effects comprise various conventional charts (such as a histogram, a radar chart, a graph, a line graph and the like), various big data visual chart effects, customized charts and other non-traditional interactive visual effects, a business-oriented and role-oriented display interface is formed, multi-terminal display is supported, and the diversified data presentation requirements of users are met.
The application layer is mainly used for realizing presentation of government affair data and unifying an external interface of the government affair data; the method is particularly used for displaying the analyzed government affair data in a terminal (comprising a mobile terminal, a web terminal and the like) in a chart form and simultaneously providing an external interface.
In some embodiments, the application layer includes a BI module and an API module;
the BI module is used for providing business intelligent functions based on the analyzed government affair data at the terminal. The terminal provides data analysis functions such as modern data warehouse, on-line analysis processing, data mining and data presentation for users based on government affair data of a business layer so as to endow business value to government affair data.
Further, the application layer may further include a large screen analysis unit to provide a large screen presentation of the PC or mobile terminal.
And the API module is used for providing a service development interface corresponding to the component library at the terminal.
In some specific examples, the API module belongs to an open platform of the system that provides a way for user graph component development, in particular, provides both business API interfaces and SQL access ways.
Next, a specific composition of the service layer will be described in further detail in this embodiment based on the above-described embodiments.
As shown in fig. 1, the service layer may specifically include a user management module, a menu management module, an algorithm library, and a report component library.
The user management module is used for managing the basic information of the system user and providing the basic information of the user acquired from the third-party system.
In some embodiments, the subscriber management module includes a synchronization subscriber unit, an organization management unit, a self-building subscriber unit, and a role management unit; the synchronous user unit is used for synchronizing the basic information of the user; the organization management unit is used for managing the user organization architecture relationship in the system; the self-built user unit is used for newly adding relevant information of the system user; and the role management unit is used for managing the roles of the users in the system.
And the menu management module is used for maintaining a system menu and managing user rights.
In some embodiments, the menu management module includes a menu group and menu weightings; the menu group is used for managing menus; and the menu authority is used for managing the authorities of different users.
Referring to fig. 1, based on the foregoing embodiments, the government full life cycle big data management system of the present embodiment may further include a data security module for encrypting and protecting government affair data in the data storage module.
Specifically, the data security module comprises a symmetrical encryption unit and an asymmetrical encryption unit, so that corresponding encryption modes are selected according to different security level requirements to carry out encryption protection on government affair data, and secure transmission and secure application of the government affair data are ensured.
In some specific embodiments, the management system further includes an operation monitoring module, configured to visually monitor the application layer and the service layer.
Specifically, the operation monitoring module comprises a report monitoring unit, a model monitoring unit, a user behavior unit, an index monitoring unit, an access statistics unit and a management log unit;
the report monitoring unit is used for monitoring report data related to the system in a visual mode;
the model monitoring unit is used for monitoring model data related to the system in a visual mode;
the user behavior unit is used for monitoring the user behavior operation of the system in a visual form;
the index monitoring unit is used for monitoring various indexes of the system in a visual mode;
the access statistics unit is used for carrying out statistics and monitoring on the access of the system in a visual form;
the management log is used for monitoring the log of the system in a visual mode.
In some embodiments, the system further comprises a cluster visualization module; the cluster visualization module is connected with the cluster management module and is used for performing visual monitoring and visual management on the cluster management module.
Specifically, the cluster visualization module comprises a group-by-group allocation unit, a dynamic expansion unit and a cluster resource monitoring unit;
the group-by-group allocation unit is used for allocating cluster resources by groups;
the dynamic expansion unit is used for dynamically expanding cluster resources;
the cluster resource monitoring unit is used for monitoring cluster resources.
Referring to fig. 2, fig. 2 is a flow chart illustrating a government full life cycle big data management method according to an embodiment. The method comprises the following steps:
s01: collecting government affair data from one or more data sources;
s02: according to the analysis purpose, performing multi-table association, modeling analysis and data cleaning processing on the government affair data, and storing the government affair data into each cluster node;
s03: managing government affair data stored by each cluster node; wherein the managing includes training and analyzing government data;
in some embodiments, the algorithm involved in this step includes K-MEANS, KNN, etc. The K-Means algorithm divides a given sample set into K clusters according to the distance between samples. The points in the clusters are connected as closely as possible, and the distance between the clusters is as large as possible. The workflow of the K-Means algorithm is shown in FIG. 3. The KNN (K-Nearest Neighbor), also called as K Nearest Neighbor, is one of the simplest machine learning Q algorithms, is applicable to classification and return problems, and is a supervised learning algorithm. The basic assumption of the algorithm is that if we find K samples in the feature space that are most similar to it (i.e. nearest neighbors in the feature space), then most of the K samples belong to a certain class, and the samples also belong to this class. The principle of the KNN algorithm is shown in figure 4.
S04: and displaying the government affair data after analysis in a terminal in a chart form. The user can view the chart of the data concerned by the user at any time, and can also customize the display form of the drag setting data.
In some embodiments, the method further comprises:
s05: visual monitoring of user behavior, charts, models and access is performed.
Referring to fig. 5, fig. 5 is a schematic business flow diagram illustrating a government full life cycle big data management method according to an embodiment.
As shown in fig. 5, a terminal (PC, mobile terminal, etc.) located at a presentation layer, i.e., an application layer, supports report presentation and cluster management; the business system, namely the business layer, provides data acquisition, data cleaning, data storage, data encryption and data service; the external system provides basic government affair data for the business system as a data source and supports the OA office platform and the ERP enterprise resource planning.
Referring to fig. 6, fig. 6 is a schematic diagram illustrating information interaction when the government full life cycle big data management system according to the embodiment performs system data analysis based on the SPARK. In this embodiment, a description will be given of system data analysis by the government full life cycle big data management system described in the above embodiment.
The government full life cycle big data management system adopts a big data analysis technology SPARK when carrying out system data analysis. SPARK is a multilingual engine for performing data engineering, data science, and machine learning on a single-node computer or cluster.
As shown in fig. 6, in the yarn-client operation mode of Spark, that is, the local client, the interaction flow between the client and the yarn cluster during the system data analysis approximately includes:
1. the client submits an application to a resource manager;
2. the resource Manager forwards the application to the corresponding cluster Node agent Node Manager, and the cluster Node agent Node Manager starts the client application management soft device APPmaster;
3. the application manager replies the applied Container to the resource manager;
4. the resource manager forwards the application to the corresponding cluster node proxy NodeManager, and starts the executor.
5. The tasks are performed at the SparkYarn client.
6. After the task is completed, the SparkYarn client applies for logout from the resource manager.
Finally, it should be noted that, although the foregoing embodiments have been described in the text and the accompanying drawings of the present application, the scope of the patent protection of the present application is not limited thereby. All technical schemes generated by replacing or modifying equivalent structures or equivalent flows based on the essential idea of the application and by utilizing the contents recorded in the text and the drawings of the application, and the technical schemes of the embodiments are directly or indirectly implemented in other related technical fields, and the like, are included in the patent protection scope of the application.

Claims (10)

1. A government full life cycle big data management system, comprising: an application layer, a service layer, a data acquisition and storage layer and a cluster management module; the business layer comprises an algorithm library and a component library;
the cluster management module is used for creating and managing a distributed big data cluster;
the data acquisition and storage layer is used for acquiring government affair data from one or more data sources, performing multi-table association, modeling analysis and data cleaning processing on the acquired government affair data according to analysis purposes, and storing the acquired government affair data to each cluster node of the distributed big data cluster;
the business layer is used for managing government affair data stored by each cluster node; the algorithm library is used for training and analyzing government affair data; the component library is used for providing various chart components corresponding to the analyzed government affair data;
and the application layer is used for displaying the analyzed government affair data in a terminal in a chart form.
2. The government full life cycle big data management system of claim 1, wherein the application layer includes a BI module and an API module;
the BI module is used for providing business intelligent functions based on the analyzed government affair data at the terminal;
and the API module is used for providing a service development interface corresponding to the component library at the terminal.
3. The government full life cycle big data management system of claim 1, wherein said business layer further comprises a user management module and a menu management module;
the user management module is used for managing the user basic information and acquiring the user basic information from a third-party system;
and the menu management module is used for maintaining a system menu and managing user rights.
4. The government full life cycle big data management system of claim 1, wherein the data collection storage layer includes a data collection module and a data storage module;
the data acquisition module comprises a timing scheduling unit, a data acquisition unit and a data cleaning unit;
the timing scheduling unit is used for setting a government affair data acquisition period;
the data acquisition unit is used for acquiring government affair data in a structured, unstructured or file form, and carrying out multi-table association processing and modeling analysis on the selected government affair data according to an analysis purpose;
the data cleaning unit is used for converting government affair data output by the data acquisition unit into a data format required by a business layer;
and the data storage module is used for storing the government affair data output by the data cleaning unit to each cluster node and providing an external interface.
5. The government full life cycle big data management system of claim 4, further comprising a data security module for cryptographically protecting government data in the data storage module.
6. The government full life cycle big data management system of claim 1, wherein the system further comprises: and the operation monitoring module is used for carrying out visual monitoring on the application layer and the service layer.
7. The government full life cycle big data management system of claim 6, wherein the operation monitoring module includes a report monitoring unit, a model monitoring unit, a user behavior monitoring unit, an index monitoring unit, an access statistics unit, and a management log unit.
8. The government full life cycle big data management system of claim 1, wherein the system further comprises: a cluster visualization module;
the cluster visualization module is used for performing visual monitoring and visual management on the cluster management module.
9. A government full life cycle big data management method based on the government full life cycle big data management system of any of claims 1-8, comprising the steps of:
collecting government affair data from one or more data sources;
according to the analysis purpose, performing multi-table association, modeling analysis and data cleaning processing on the government affair data, and storing the government affair data into each cluster node;
managing government affair data stored by each cluster node; wherein the managing includes training and analyzing government data;
and displaying the government affair data after analysis in a terminal in a chart form.
10. The government full life cycle big data management method of claim 9, wherein said method further comprises:
visual monitoring of user behavior, charts, models and access is performed.
CN202211635301.2A 2022-12-19 2022-12-19 Government full life cycle big data management system and method thereof Pending CN116028574A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211635301.2A CN116028574A (en) 2022-12-19 2022-12-19 Government full life cycle big data management system and method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211635301.2A CN116028574A (en) 2022-12-19 2022-12-19 Government full life cycle big data management system and method thereof

Publications (1)

Publication Number Publication Date
CN116028574A true CN116028574A (en) 2023-04-28

Family

ID=86076997

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211635301.2A Pending CN116028574A (en) 2022-12-19 2022-12-19 Government full life cycle big data management system and method thereof

Country Status (1)

Country Link
CN (1) CN116028574A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117172410A (en) * 2023-09-04 2023-12-05 西南交通大学 Product service full life cycle value chain optimization system and method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117172410A (en) * 2023-09-04 2023-12-05 西南交通大学 Product service full life cycle value chain optimization system and method
CN117172410B (en) * 2023-09-04 2024-03-08 西南交通大学 Product service full life cycle value chain optimization system and method

Similar Documents

Publication Publication Date Title
CN107315776B (en) Data management system based on cloud computing
CN105809356A (en) Information system resource management method based on application integrated cloud platform
EP2717201A1 (en) Natural language metric condition alerts orchestration
CN109446274B (en) Method and device for managing BI metadata of big data platform
KR20080049757A (en) Template based management of services
Agrahari et al. A review paper on Big Data: technologies, tools and trends
CN110807026A (en) Automatic capture system for analyzing financial big data blood relationship
CN104781793A (en) Systems and methods for integrating storage usage information
CN116028574A (en) Government full life cycle big data management system and method thereof
CN112632025A (en) Power grid enterprise management decision support application system based on PAAS platform
Manikyam et al. Methods and techniques to deal with big data analytics and challenges in cloud computing environment
CN113542074A (en) Method and system for visually managing east-west network traffic of kubernets cluster
CN113987626A (en) Extensible building full life BIM modeling method
CN115168396A (en) Comprehensive intelligent platform data management method and system based on spatio-temporal system
CN110532058B (en) Management method, device and equipment of container cluster service and readable storage medium
CN110134511A (en) A kind of shared storage optimization method of OpenTSDB
CN112035555B (en) Information display method, device and equipment
CN113177088A (en) Multi-scale simulation big data management system for material irradiation damage
Polak et al. Organization of quality-oriented data access in modern distributed environments based on semantic interoperability of services and systems
CN110046150A (en) A kind of human resources monitoring analysis method and system
CN202584128U (en) Modifying and managing system of knowledge base of IT (Information Technology) operation and maintenance system
CN112199075A (en) Intelligent information processing method and framework based on micro-service
US20110158242A1 (en) Transporting object packets in a nested system landscape
CN113222563A (en) Audit data evidence storing and tracing system based on block chain
CN113391916A (en) Organization architecture data processing method, device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination