CN117632954A

CN117632954A - Method and system for managing data resources

Info

Publication number: CN117632954A
Application number: CN202311553972.9A
Authority: CN
Inventors: 李仁敏; 李相东
Original assignee: Inspur Communication Information System Tianjin Co Ltd
Current assignee: Inspur Communication Information System Tianjin Co Ltd
Priority date: 2023-11-21
Filing date: 2023-11-21
Publication date: 2024-03-01

Abstract

The invention provides a method and a system for managing data resources, comprising a data development center module, a data management center module, a data sharing center module, a data analysis center module, an operation management center module and a data open operation portal module. The invention is based on a multi-source heterogeneous data fusion technology, and by constructing a unified metadata warehouse and providing a perfect data management system, the invention realizes the efficient utilization of data resources, the structured management of data standards and the effective monitoring and improvement of data quality, and builds, shares and shares the data information inside the whole enterprise or government department from a unified service perspective.

Description

Method and system for managing data resources

Technical Field

The invention relates to the field of data management, in particular to a method and a system for managing data resources.

Background

The big data age is an age in which data is regarded as a core asset, and the data presents strategic, asset and socialization characteristics. IT systems of enterprises and government departments have experienced periods of high data volume expansion, and these massive amounts of data scattered in different corners have resulted in complexity of data resource utilization and high difficulty of management, and cannot be used to overview data information inside the whole enterprise or government departments from a unified business perspective. The relation between the systems is that standard data is obtained from where to have no slave knowledge, and the data quality is difficult to guarantee. Because of the system dispersion, the data standard lacks structured management, so that panoramic retrieval cannot be provided; meanwhile, the data confusion is caused by the source data change, the data problem cannot be rapidly defined among all systems, and the data quality cannot be effectively monitored and improved.

The existing big data platform and the construction method thereof still have the following defects:

1. guaranteeing data cross-system fusion to improve data value: the traditional method and system have a plurality of defects in the aspect of realizing data cross-system communication. First, they tend to rely on manual operations, resulting in inefficiency and susceptibility to error. Second, the conventional method lacks automation capability, and is difficult to rapidly and accurately process large-scale data conversion and comparison work. In addition, conventional methods often have insufficient flexibility and are difficult to accommodate for the data sharing requirements between different systems and databases. Finally, the traditional method also has data security problems, such as risks of data leakage, data tampering and the like.

2. Providing unified data service capability to improve service enabling efficiency: conventional methods and systems have drawbacks in providing unified data service capabilities and improving service enabling efficiency. First, data tends to be scattered and difficult to integrate in conventional systems, resulting in data being difficult to share and utilize, limiting the efficiency of service enablement. Secondly, the traditional method has low response speed, is difficult to quickly respond to market demands, and cannot make business decisions and adjustments in time. In addition, the conventional method has limited decision support capability, and it is difficult to fully utilize data to improve business efficiency and competitiveness. In addition, the shortage of standardization and standardization of data is a great disadvantage of the traditional method, increases the threshold for data use, and can influence the quality and accuracy of the data. Finally, the techniques and tools involved in conventional approaches may form a technical barrier, increasing technical thresholds and learning costs, making it difficult for non-technicians to understand and use.

3. Lowering the technology usage threshold to break the usage technology barrier: conventional methods and systems have significant drawbacks in terms of lowering the technical use threshold and breaking the technical barriers to use. First, they often involve complex techniques and tools, resulting in a high technical threshold, and many business and technical personnel have difficulty leveraging data and information technology to support business development. Secondly, the traditional method lacks usability, the use process is complex and complicated, and staff is difficult to get up quickly. This increases the learning costs and burden for the staff and may cause them to experience difficulties in understanding and applying data and information technology. In addition, the lack of flexibility in conventional methods may not be able to accommodate different business requirements and technical environments, making it difficult for enterprises to achieve rapid iterations and optimizations in terms of data utilization and information technology applications. In addition, the traditional method has security problems such as data leakage, system loopholes and the like, and risks are brought to the data security of enterprises.

Disclosure of Invention

A method and system for managing data resources, comprising the steps of:

step S1: the data development center module is used for realizing the work of data batch processing, data stream processing, data internet acquisition and data modeling;

step S2: the data management center module is responsible for data standard definition, metadata management, data index management, data asset maintenance and data quality monitoring;

step S3: the data sharing center module is responsible for data FTP service management, data API service management, system message service management, data service subscription management and data service state monitoring work;

step S4: the data analysis center module is responsible for the self-defining data report design and presentation, the online data impromptu query visualization, the visual large-screen integrated management and the automatic generation of data analysis reports;

step S5: the operation management and control center module is in charge of system task work order management, data asset panoramic view display, system operation state monitoring and system multi-tenant management work;

step S6: the data open operation portal module is responsible for map navigation of data asset indexes, data catalog management, external service catalog management, system authority setting and system operation log audit work.

Further, in step S1, the data development center module is a key component for supporting data development and processing work, and includes data batch processing, data stream processing, data internet collection and data modeling work, which means that it can process batch operation of large-scale data, real-time data stream processing, connect with external data sources for collection, and support data modeling work, in addition, the module also provides task scheduling function, so that data processing tasks can be arranged, connected in series or parallel as required, thereby realizing complex data processing flow, in terms of data collection, the module supports off-line and real-time modes, can perform efficient collection on data through various protocols such as JDBC, SFTP, socket, and meets data acquisition requirements in different scenes, and the data development center module aims to help data developers to perform data processing and modeling work efficiently.

Further, in step S2, the data management center module plays an important role in data management, whose responsibility covers data standard definition, metadata management, data index management, data asset maintenance and data quality monitoring, and this module supports defining various types of data standards including layering, table names, fields, etc., so as to help enterprise specifications and specifications in terms of naming, format, structure, etc. of unified data, through metadata management, this module can effectively record and track attributes, relationships and business meanings of data, and provide basic support for maintenance and utilization of data assets, and in addition, the data index management function enables enterprises to define and monitor data quality indexes, understand data quality status in real time and take corresponding improvement measures, and in view of synthesis, the data management center module aims to ensure consistency, accuracy and reliability of data, and provides important support and guarantee for management and value implementation of enterprise data assets.

Further, in step S3, the data sharing center module is an important component responsible for data sharing and service management, and its responsibilities include data FTP service management, data API service management, system message service management, data service subscription management, and data service status monitoring, which means that the module can provide multiple data sharing modes, including FTP service and API service, and implement secure data transmission and sharing between different systems.

Further, in step S4, the user-defined data report design and presentation function enables the user to flexibly design and customize the data report according to specific business requirements and analysis purposes, and presents the data in an intuitive manner, so as to help the user to better understand the data, find rules and make decisions, meanwhile, the data online on-line prompt query visualization function allows the user to perform real-time data query and analysis through an online tool, and intuitively presents the query result in the form of a chart and the like, thereby facilitating the user to quickly acquire required information, the visual large-screen comprehensive management function provides support for large-screen data display and management for the user, can help the user to construct and manage various data visualized large-screen displays and real-time monitoring key business indexes, has important significance for real-time monitoring and decision support of enterprises, generates a data analysis report through an automatic flow, saves time and energy of the user on report generation, and ensures accuracy and consistency of the report.

Further, in step S5, the system task work order management function allows the user to effectively manage and schedule the system tasks and work orders, including the creation, allocation, monitoring and feedback of execution conditions of the tasks, thereby ensuring efficient operation and timely processing of the system tasks, the data asset panoramic view display function provides the overall overview and detailed information display of the enterprise data assets, helps the user to comprehensively understand and grasp the conditions of the enterprise data assets, thereby pertinently performing data management and utilization, the system operation state monitoring function timely finds and solves the problems in system operation by monitoring the operation state, performance index and abnormal conditions of the system in real time, ensuring the stability and reliability of the system, and is significant for ensuring the continuity and stability of the service flow, the system multi-tenant management work is responsible for managing the multi-tenant environment of the system, including the division of tenants, the allocation of resources and the control of rights, so as to meet the individual requirements of different tenants on the system resources and services, the operation management and control center module aims at providing comprehensive control and monitoring of the information system for the enterprise, and timely discovering and solving the problems in system operation, simultaneously meeting the requirements of different service scenes, providing flexible protocols under various requirements, such as the requirements of the enterprise interface, and the requirements of the system is further adapted to the requirements of the system, and the system is suitable for the requirements of the various types WebService, FTP.

Further, in step S6, the data asset index map navigation function allows the user to intuitively understand the distribution condition and index information of the data asset through a map or other visual modes, helps the user to quickly locate and access the required data asset, thereby improving the efficiency and accuracy of data acquisition, the data directory management function is used for managing and displaying the directory structure and detailed information of the data resource in the enterprise, including description of the data set, data format, data quality and the like, helping the user comprehensively understand and utilize the data asset of the enterprise, the external service directory management function focuses on the data service directory of the enterprise which is open to the outside, including information such as interfaces, service description, data format and the like which are provided to the outside, and facilitates the external user or partner to find and use the data service which is open to the enterprise, the system authority setting function is used for managing and configuring authorities of different users or roles, ensuring the safety and compliance of data, simultaneously meeting the operation requirements of different users in the data opening and sharing process, enabling system operation log audit work to be responsible for recording and auditing operation logs of a system, including user operation logs, system event logs and the like, so as to monitor and track the operation and abnormal conditions of the system, provide support for the investigation and the solution of safety problems, and enable a data opening operation portal module to aim at providing a unified data opening and sharing platform for enterprises, provide flexible data classification and custom management so as to adapt to the requirements in different client scenes, simultaneously ensure the safety and compliance of the data and promote the maximization and sustainable operation of the value of data assets.

Compared with the prior art, the invention has the following advantages:

1. first, the conventional method can provide a possibility of manual operation, which means that enterprises can process data manually to make up for the lack of automation capability in the face of large-scale, complex data conversion and comparison works. Although this approach is relatively inefficient, the accuracy and quality of the data can be guaranteed. Second, conventional methods generally have better flexibility and adaptability. There may be differences between different systems and databases, and conventional methods may be adjusted and optimized according to different situations and requirements to meet the data sharing requirements between different systems and databases. Furthermore, conventional approaches may have a relative advantage in terms of data security. While conventional approaches present the risk of data leakage and data tampering, they tend to provide more security control and regulatory mechanisms, such as data encryption, access control, and audit trails, than fully automated systems. These mechanisms help to reduce data security risks. In general, while conventional methods and systems suffer from drawbacks such as inefficiency, lack of automation capability, lack of flexibility, etc., they also have advantages in protecting data from cross-system fusion to enhance data value. These advantages include providing possibilities for manual operation, better flexibility and adaptability, and relatively better data security. However, to better enhance data value and overcome the drawbacks of conventional methods, enterprises are continually exploring and innovating to employ more advanced data management and analysis tools to improve efficiency, automation, and data quality.

2. First, the conventional method can realize centralized management and integration of data. In conventional systems, data is often stored in various departments and systems, and sharing and utilization of the data is difficult to achieve. By adopting the traditional method, enterprises can realize unified management and integration of the data, thereby better utilizing the data and improving the efficiency of service energization. Second, the conventional method can quickly respond to market demands. In a strong market competition, a fast response to market changes is crucial. The conventional method can provide rapid data processing and analysis capabilities, helping enterprises to make business decisions and adjustments quickly to capture market opportunities. In addition, conventional approaches may provide comprehensive decision support capabilities. Data is an important basis for decision making, and traditional methods can provide comprehensive data analysis and visualization tools to help enterprises make more intelligent and accurate business decisions. In addition, the conventional method can also promote standardization and normalization of data. Through standardized and normalized processing of the data, the data use threshold can be reduced, the quality and accuracy of the data are improved, and meanwhile, the cooperation and data sharing in enterprises are facilitated. Finally, the techniques and tools of the conventional methods are relatively mature and stable. Compared with modern data management and analysis tools, the technology and tools in the traditional method have higher reliability and stability after time and practical inspection. This makes it easier for non-technicians to understand and use these tools and techniques to better participate in data utilization and information technology applications. In general, although conventional methods and systems have some drawbacks such as data dispersion, slow response speed, limited decision support capability, etc., they have some advantages in providing uniform data service capability and improving service enabling efficiency. These advantages include centralized management and integration of data, quick response to market demands, comprehensive decision support capabilities, promotion of standardization and normalization of data, and maturation and stability of technologies and tools. However, to better promote business enabling efficiency and data service capabilities, businesses need to continually explore and innovate, employing more advanced data management and analysis tools to increase efficiency, flexibility, and degree of intelligence.

3. First, conventional approaches can provide a clearer technical path. In the presence of complex technologies and tools, the explicit technology path can reduce the use threshold, helping business and technical personnel to better understand and apply data and information technology. By providing detailed technical guidelines, training courses, operating manuals, and the like, conventional methods can help employees to quickly get on hand and reduce learning costs. Second, traditional approaches focus on ease of use designs. In order to solve the problems of complicated and complex use process, the traditional method focuses on improving the usability of the product, simplifying the operation flow and reducing the use difficulty. This enables employees to use data and information technology more quickly and easily, improving work efficiency and reducing error rates. In addition, the traditional method has better flexibility and adaptability. Different business requirements and technical environments may require different solutions. The traditional method can provide a flexible solution according to the actual demands of enterprises, and meets the personalized demands of the enterprises in the aspects of data utilization and information technology application. This facilitates fast iteration and optimization for enterprises in data utilization and information technology applications. In addition, the conventional method also focuses on security issues. Security problems such as data leakage and system loopholes are important aspects of enterprises that need to pay attention to. The traditional method generally adopts a series of security measures such as data encryption, access control, bug repair and the like to ensure the data security of enterprises. This helps to reduce the data security risk for the enterprise. In general, while conventional methods and systems suffer from drawbacks such as excessive technical thresholds, lack of ease of use, insufficient flexibility, etc., they also have advantages in lowering technical use thresholds and breaking technical barriers to use. These advantages include providing a clearer technical path, focusing on ease of use design, having better flexibility and adaptability, and focusing on security issues. However, to better break the usage technology barriers and improve the efficiency and effectiveness of data utilization and information technology applications, businesses need to continually explore and innovate, employing more advanced data management and analysis tools to improve efficiency, user experience, and security.

Detailed Description

The subject matter described herein will now be discussed with reference to example embodiments. It should be appreciated that these embodiments are discussed only to enable a person skilled in the art to better understand and thereby practice the subject matter described herein, and are not limiting of the scope, applicability, or examples set forth in the claims. Changes may be made in the function and arrangement of elements discussed without departing from the scope of the disclosure as set forth in the specification. Various examples may omit, replace, or add various procedures or components as desired. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. In addition, features described with respect to some examples may be combined in other examples as well.

As used herein, the term "comprising" and variations thereof mean open-ended terms, meaning "including, but not limited to. The term "based on" means "based at least in part on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment. The term "another embodiment" means "at least one other embodiment". The terms "first," "second," and the like, may refer to different or the same object. Other definitions, whether explicit or implicit, may be included below. Unless the context clearly indicates otherwise, the definition of a term is consistent throughout this specification.

Examples

A method and system for managing data resources, comprising the steps of:

In step S1, first, the data development center module performs efficient processing on large-scale data through a data batch processing function. The processing mode is suitable for carrying out batch operation on the historical data, such as carrying out batch analysis, cleaning and storage on the historical sales data, so as to facilitate subsequent data analysis and decision support. In the batch processing process, the data development center module can automatically process data in batches, so that the pressure of data processing is reduced, and the data processing efficiency is improved. And the data development center module also has a real-time data stream processing function. The processing mode is suitable for carrying out instant analysis and processing on the real-time data, and the real-time data stream processing can rapidly capture and analyze the real-time data, thereby providing more timely and accurate data support for decision makers. In addition, the data development center module also has an internet acquisition function. By connecting to an external data source, the module can efficiently collect various data on the internet, such as social media data, e-commerce data, and the like. The acquisition mode can rapidly acquire a large amount of data, and provides important support for subsequent data analysis and modeling. Meanwhile, the data development center module also supports data modeling work. By providing various data modeling tools and algorithm libraries, the module can help a user quickly build various data models, such as machine learning models, statistical models, and the like. These models may help users better understand and analyze data, providing more accurate and reliable support for decisions. In addition to powerful data processing capabilities, the data development center module also has flexible task orchestration functionality. The functions enable a user to perform arrangement, serial or parallel operation as required, and complex data processing flows are realized. For example, a user may combine and arrange data batch processing tasks and real-time data stream processing tasks through a task orchestration function to implement more complex data processing flows. In terms of data collection, the data development center module supports both off-line and real-time modes. The module can collect data efficiently through a variety of protocols, such as JDBC, SFTP, socket, etc. These protocol supports enable the module to adapt to different scenarios and requirements, and enable efficient and stable data acquisition, whether structured or unstructured data acquisition. In a word, the data development center module is used as a key component of data development and processing work, and comprehensive processing and analysis of mass data are realized by integrating various data processing and modeling functions. The data development center module can provide powerful and flexible support for users, whether large-scale data batch processing, real-time data stream processing, internet data acquisition or data modeling work. Meanwhile, the module can realize complex data processing flow and adapt to different scene demands through task arrangement functions and multiple protocol support. Integration and optimization of these functions makes the data development center module a powerful tool in the data field, helping users to perform data processing and modeling work more efficiently.

In step S2, the data development center module further has an internet collection function, and can be easily connected with an external data source to obtain various online data, such as social media data, e-commerce data, and the like. This not only enriches the data sources, but also makes the data acquisition speed faster. By providing various data modeling tools and algorithm libraries, the module can help a user to easily construct various data models, so that data can be better understood and analyzed, and decisions can be more accurate and reliable. It should be noted that the data development center module also has a task orchestration function. This means that the user can arrange, serially connect or operate different data processing tasks in parallel as required, and implement a complex data processing flow. The function greatly improves the data processing efficiency and meets different business requirements. In terms of data acquisition, the module supports both offline and real-time modes. Whether the historical data is collected in batches or the real-time data is collected in a streaming mode, the method can be used for coping with the historical data easily. This benefits from the fact that it supports multiple protocols, such as JDBC, SFTP, socket, etc., making the acquisition of data more efficient and stable. It is excellent in both structured and unstructured data acquisition. In general, the data development center module enables comprehensive processing and analysis of mass data by integrating a variety of data processing and modeling functions, as well as providing flexible task orchestration and multi-protocol support. Whether mass processing of large-scale data, real-time data stream processing, internet data acquisition or data modeling work, the system can provide powerful and flexible support for users. The optimization and integration of these functions makes the data development center module a powerful tool for the data field, greatly helping users to efficiently perform data processing and modeling work.

In step S3, the data sharing center module is a key component for data interaction and circulation, and performs tasks such as data service management, data subscription management, and data status monitoring. The module can safely transmit and share data among different systems by providing various data sharing modes such as FTP service, API service and the like. The module supports message communication and data exchange between systems through the system message service management function, and meets the requirement of asynchronous data exchange. In addition, the data service subscription management function helps the user manage the data subscription relationship, and ensures the compliance and safety of the data service. And finally, the data service state monitoring function monitors the running state of the data service in real time, and ensures the stability and reliability of the data sharing service. In short, the data sharing center module aims to help users to more efficiently conduct data interaction and circulation through flexible and comprehensive data sharing and service management functions

In step S4, the data report and the presentation module endow the user with a strong self-defining capability, and according to the business requirements and the analysis targets, the user can freely design and customize the data report to present the data in the most intuitive way. The module can provide all-round support for users, both for large-scale data processing and analysis and real-time data monitoring and decision making. Through the online impromptu query tool, a user can query data in real time and intuitively display query results in the form of charts and the like, so that the time for acquiring key information is greatly shortened. For the integrated management of large screen display, the module provides convenient construction and management functions for users, and the users can easily deal with the large screen display of data visualization or the real-time monitoring of key business indexes. This enables the enterprise to monitor the operating conditions in real time and make decision-making reactions quickly. Finally, the automatic data analysis report generation function releases the user from the tedious report generation work, thereby saving a great deal of time and energy. More importantly, the automation flow ensures the accuracy and consistency of the report, and avoids human errors and omission. In general, the data report and presentation module provides comprehensive and efficient data processing and analysis support for users through flexible design and customization functions, on-line impromptu query tools, visual large-screen integrated management, automatic generation of data analysis reports and other functions. Whether processing and monitoring of real-time data, or analysis and decision-making of large-scale data, the module can help users better understand and utilize the data to make informed business decisions.

In step S5, first, the system task work order management function provides the user with an end-to-end management tool from task creation, distribution to monitoring and execution feedback. The user can quickly create tasks and flexibly allocate to different teams or individuals according to the requirements. Meanwhile, through real-time monitoring of the execution condition of the task, the user can timely find and solve the problem, and high-efficiency operation and timely processing of the task are ensured. This not only improves the working efficiency, but also reduces the potential risk. The data asset panoramic view presentation function provides an intuitive enterprise data asset overview for the user. Through this function, a user can quickly learn about the overall status of the enterprise data asset, including the type, source, storage and usage of the data. This provides the user with a comprehensive and accurate view of data, helping them make intelligent decisions in a complex business environment. The system running state monitoring function ensures the stability and reliability of the system by monitoring the running state, performance index and abnormal condition of the system in real time. Once an abnormality or performance bottleneck is found, the system can immediately give an alarm to inform a user to timely process, so that the continuity and stability of the business process are ensured. The system multi-tenant management function is responsible for comprehensively managing the multi-tenant environment of the system. Through the division of tenants, the allocation of resources and the control of authority, enterprises can meet the personalized demands of different tenants on system resources and services. The flexible management mode enables enterprises to better serve different customer groups, and customer satisfaction is improved. Finally, the operation management and control center module provides an enterprise with a comprehensive management and control and monitoring tool for the information system. The system not only ensures the stability and the efficient operation of the system, but also meets the business requirements of enterprises in diversity by providing flexible and various interface protocol types, such as RESTful API, webService, FTP and the like. This enables the enterprise to adapt more flexibly to market changes and business needs, enabling continued innovation and development. In general, the synergy of these functional modules enables an enterprise to more efficiently manage information systems, monitor business processes, master data asset profiles, and meet multi-tenant needs. Through flexible interface protocol types, enterprises can realize seamless integration with various business systems, thereby improving working efficiency and reducing risks. The integrated management tools of these modules enable businesses to better serve customers, master market dynamics, and enable continued innovation and development.

In step S6, first, the data asset index map navigation function enables the user to intuitively understand the distribution situation and index information of the data asset through a map or other visual manner. This provides a user with a means to quickly locate and access desired data assets, thereby improving the efficiency and accuracy of data acquisition. For example, by clicking on a specific area on the map, a user can quickly find the data set and index information related to the area, and timely support is provided for analysis decision. The data catalog management functions then assist the user in comprehensively knowing and utilizing the data assets of the enterprise. Through managing and displaying the directory structure of the data resources in the enterprise, the user can easily acquire detailed information such as description, data format, data quality and the like of the data set. The user can search and use the required data resources more conveniently, and the working efficiency is improved. The external service directory management function focuses on the data service directory of the enterprise which is open to the outside. The function facilitates external users or partners to find and use the data service opened by the enterprise by managing and displaying information such as interfaces, service descriptions, data formats and the like provided by the enterprise. This provides powerful support for data interaction and collaboration between the enterprise and external users. The system rights setting function is responsible for managing and configuring rights of different users or roles. According to the requirements and regulations of enterprises, users can use the functions to give different rights to different users or roles, so that the safety and compliance of data are ensured. Meanwhile, the function also meets the operation requirements of different users in the data opening and sharing process, and the flexibility and usability of the data are guaranteed. System operation log audit work record and audit system operation log. This includes user operation logs, system event logs, etc. to monitor and track the operation and abnormal conditions of the system. Once abnormal or illegal operation is found, the system can immediately give an alarm to remind an administrator of timely processing and provide support for the investigation and the solution of the security problem. And finally, the data open operation portal module provides a unified data open and share platform for enterprises. By providing flexible data classification and custom management, the module adapts to the requirements of different customer scenes, and simultaneously ensures the safety and compliance of the data. In addition, the module is also directed to maximizing the value and sustainable operation of the data asset, creating more business opportunities and value for the enterprise. In general, the series of functional modules together build an efficient, secure, flexible, and sustainable data asset management and operation architecture. Through the functions of data asset index map navigation, data catalog management, external service catalog management and the like, enterprises can better know and utilize own data assets; and the functions of system authority setting, system running log audit, data open operation portal and the like further ensure the safety and compliance of the data, and realize flexible management and operation of the data. Such integrated solutions help businesses to improve business insights, innovation ability, and competitiveness, thereby gaining greater advantages in increasingly aggressive market competition.

It will be appreciated by those skilled in the art that various changes and modifications can be made to the embodiments disclosed above without departing from the spirit of the invention. Accordingly, the scope of the invention should be limited only by the attached claims.

It should be noted that not all the steps and units in the above-mentioned processes are necessary, and some steps or units may be omitted according to actual needs. The order of execution of the steps is not fixed and may be determined as desired. The apparatus structures described in the above embodiments may be physical structures or logical structures, that is, some units may be implemented by the same physical entity, or some units may be implemented by multiple physical entities, or may be implemented jointly by some components in multiple independent devices.

The detailed description set forth above describes exemplary embodiments, but does not represent all embodiments that may be implemented or fall within the scope of the claims. The term "exemplary" used throughout this specification means "serving as an example, instance, or illustration," and does not mean "preferred" or "advantageous over other embodiments. The detailed description includes specific details for the purpose of providing an understanding of the described technology. However, the techniques may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form in order to avoid obscuring the concepts of the described embodiments.

The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other variations without departing from the scope of the disclosure. Thus, the disclosure is not limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method and system for managing data resources, characterized by: the system comprises a data development center module, a data management center module, a data sharing center module, a data analysis center module, an operation management control center module and a data open operation portal module.

2. A method and system for managing data resources according to claim 1, characterized in that: in step S1, the data development center module is a key component for supporting data development and processing work, and includes data batch processing, data stream processing, data internet acquisition and data modeling work, which means that it can process batch operation of large-scale data, real-time data stream processing, connect with external data sources for acquisition, and support data modeling work, and in addition, the module provides task scheduling function, so that data processing tasks can be arranged, connected in series or operated in parallel as required, thereby realizing complex data processing flow, in terms of data acquisition, the module supports off-line and real-time modes, can perform efficient acquisition of data through multiple protocols such as JDBC, SFTP, socket, and meets data acquisition requirements in different scenes, and the data development center module aims to help data developers perform data processing and modeling work efficiently.

3. A method and system for managing data resources according to claim 1, characterized in that: in step S2, the data management center module plays an important role in data management, whose responsibility covers data standard definition, metadata management, data index management, data asset maintenance and data quality monitoring, and this module supports defining multiple types of data standards including layering, table names, fields, etc., thereby helping enterprise specifications and unified data naming, format, structure, etc., through metadata management, this module can effectively record and track data attributes, relationships and business meanings, and provide basic support for data asset maintenance and utilization, and in addition, the data index management function enables enterprises to define and monitor data quality indexes, understand data quality status in real time and take corresponding improvement measures, comprehensively, the data management center module aims to ensure data consistency, accuracy and reliability, and provides important support and guarantee for enterprise data asset management and value implementation.

4. A method and system for managing data resources according to claim 1, characterized in that: in step S3, the data sharing center module is an important component responsible for data sharing and service management, and its responsibilities include data FTP service management, data API service management, system message service management, data service subscription management, and data service status monitoring, which means that the module can provide multiple data sharing modes, including FTP service and API service, and implement secure data transmission and sharing between different systems.

5. A method and system for managing data resources according to claim 1, characterized in that: in step S4, the user-defined data report design and presentation function enables the user to flexibly design and customize the data report according to specific business requirements and analysis purposes, and presents the data in an intuitive manner, thereby helping the user to better understand the data, find rules and make decisions, meanwhile, the data online ad hoc query visualization function allows the user to perform real-time data query and analysis through online tools, and intuitively presents the query result in the form of a chart and the like, thereby facilitating the user to quickly acquire required information, the visual large screen integrated management function provides support for large screen data display and management for the user, can help the user to construct and manage various large screen display of data visualization, and monitor key business indexes in real time, has important significance for the real-time monitoring and decision support of enterprises, and the data analysis report automatically generates a data analysis report through an automated process, saves time and energy of the user in report generation, and ensures accuracy and consistency of the report.

6. A method and system for managing data resources according to claim 1, characterized in that: in step S5, the system task work order management function allows the user to effectively manage and schedule the system tasks and work orders, including the creation, distribution, monitoring and feedback of execution conditions of the tasks, thereby ensuring the efficient operation and timely processing of the system tasks, the data asset panoramic view display function provides the overall overview and detailed information display of the enterprise data assets, helps the user to comprehensively understand and grasp the conditions of the enterprise data assets, thereby pertinently performing data management and utilization, the system operation state monitoring function timely discovers and solves the problems in system operation by monitoring the operation state, performance index and abnormal conditions of the system in real time, and ensures the stability and reliability of the system.

7. A method and system for managing data resources according to claim 1, characterized in that: in step S6, the data asset index map navigation function allows the user to intuitively understand the distribution situation and index information of the data asset through a map or other visual modes, helps the user to quickly locate and access the required data asset, thereby improving the efficiency and accuracy of data acquisition, the data catalog management function is used for managing and displaying the catalog structure and detailed information of the data resource in the enterprise, including description of a data set, data format, data quality and the like, helps the user comprehensively understand and utilize the data asset of the enterprise, the external service catalog management function focuses on the data service catalog of the enterprise which is externally opened, including information such as externally provided interfaces, service description, data format and the like, facilitates the external user or partner to search and use the data service opened by the enterprise, and the system authority setting function is used for managing and configuring authorities of different users or roles, ensures the safety and compliance of the data, meets the operation requirements of different users in the data opening and sharing process, and the system operation log work is responsible for recording and auditing operation logs of the system, including user operation log, system event log and the like, so as to monitor and track the operation and abnormal situation of the system, and provide information for the system, and provide a unified service for the security problem, and provide a unified service module for the data with the user-opening and the data service, and a user-opening and a data-sharing system, and a data-opening service module is provided with the data-opening and a data-opening-supporting system.