CN117792902A - Data management method, device, equipment and medium - Google Patents
Data management method, device, equipment and medium Download PDFInfo
- Publication number
- CN117792902A CN117792902A CN202311623864.4A CN202311623864A CN117792902A CN 117792902 A CN117792902 A CN 117792902A CN 202311623864 A CN202311623864 A CN 202311623864A CN 117792902 A CN117792902 A CN 117792902A
- Authority
- CN
- China
- Prior art keywords
- data
- interface
- function configuration
- management system
- data management
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013523 data management Methods 0.000 title claims abstract description 126
- 238000000034 method Methods 0.000 title claims abstract description 36
- 230000006870 function Effects 0.000 claims abstract description 146
- 238000013500 data storage Methods 0.000 claims description 12
- 238000007726 management method Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 5
- 230000014509 gene expression Effects 0.000 claims description 5
- 238000011161 development Methods 0.000 abstract description 7
- 238000012423 maintenance Methods 0.000 abstract description 4
- 238000012545 processing Methods 0.000 description 7
- 230000018109 developmental process Effects 0.000 description 6
- 230000007246 mechanism Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000012795 verification Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 230000014759 maintenance of location Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000033772 system development Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data management method, a device, equipment and a medium, and relates to the technical field of information. The method comprises the following steps: the interface module provides an access interface for the external application and receives an access request sent by the external application based on the access interface; the service module is provided with a function configuration interface, receives function configuration information sent by an access user based on the function configuration interface, and changes the function configuration of the data management system based on the function configuration information so that the data management system realizes dynamic configuration of system functions based on the function configuration interface. The problems of large development workload, low efficiency and high cost caused by the fact that a traditional decentralised data management system needs to modify codes when the demands are changed are solved; therefore, the system functions can be dynamically configured according to the requirements by a user on the basis of not modifying codes, the efficiency is high, the cost is low, the maintenance is convenient, and the system is suitable for large-scale application and popularization.
Description
Technical Field
The present invention relates to the field of information technologies, and in particular, to a data management method, apparatus, device, and medium.
Background
At present, the de-centralized data system is still in an early development stage, but has incomparable advantages with the traditional data management system, so that the future development potential of the de-centralized data management system is huge; when in practical application, developing a data application, and needing a matched data view to realize corresponding functional configuration; however, because the development of the data view is very complicated, the data needs to be queried or crawled from a blockchain browser or a quotation website, different data filtering rules, storage rules and the like, and the code needs to be modified once the requirement is increased or modified, the processing steps are complicated, the efficiency is low, a large amount of manpower and material resources are consumed, and the system cost is increased; based on this, how to provide a data management system capable of dynamically configuring functions according to requirements has become an urgent problem to be solved.
Disclosure of Invention
The invention aims to provide a data management method which is used for solving the problems of complicated processing steps, low efficiency and high cost caused by the need of modifying codes when the demand of an decentralized data management system in the prior art is increased or modified.
In a first aspect, an embodiment of the present invention provides a data management method, including:
the interface module provides an access interface for the external application and receives an access request sent by the external application based on the access interface, wherein the access request comprises access user information;
the service module is provided with a function configuration interface, wherein the service module communicates with the interface module, determines the access authority of the access user based on the access user information in the access request sent by the interface module, and opens the function configuration interface to the access user when the access authority is the management configuration authority;
and receiving function configuration information sent by the access user based on the function configuration interface, and changing the function configuration of the data management system based on the function configuration information so that the data management system realizes dynamic configuration of system functions based on the function configuration interface.
Preferably, the service module alters the function configuration of the data management system based on the function configuration information, including:
the service module configures a data source of the data acquisition by the data management system based on the functional configuration information, configures a data acquisition strategy of the data management system, configures a data consumption interface of the data management system and/or configures a data storage strategy of the data management system.
Preferably, the service module configures a data source of the data acquisition by the data management system based on the function configuration information, and the service module includes:
the service module modifies information of the data source and/or adds information of the data source based on the function configuration information, and the information of the data source comprises URL address, access token, user information, data source name and/or system to which the data source belongs.
Preferably, the service module configures a data acquisition policy of the data management system based on the function configuration information, including:
the service module adds data acquisition policies and/or modifies the data acquisition policies based on the functional configuration information, and the data acquisition policies include a data acquisition policy enforcement engine, data acquisition policy enforcement rules, parameter definitions for data acquisition, and/or regular expressions for data filtering.
Preferably, the service module configures a data consumption interface of the data management system based on the function configuration information, including: the RPC interface and/or the sql query interface are configured based on the functional configuration information.
In a second aspect, an embodiment of the present invention provides a data management system, including:
the interface module is used for providing an access interface for the external application and receiving an access request sent by the external application based on the access interface, wherein the access request comprises access user information;
the service module is provided with a function configuration interface, wherein the service module is used for communicating with the interface module, determining the access authority of the access user based on the access user information in the access request sent by the interface module, and opening the function configuration interface to the access user when the access authority is the management configuration authority;
the service module is also used for receiving the function configuration information sent by the access user based on the function configuration interface and changing the function configuration of the data management system based on the function configuration information so that the data management system realizes the dynamic configuration of the system functions based on the function configuration interface.
Preferably, the service module alters the function configuration of the data management system based on the function configuration information, including:
the service module is used for configuring a data source for data acquisition of the data management system based on the functional configuration information, configuring a data acquisition strategy of the data management system, configuring a data consumption interface of the data management system and/or configuring a data storage strategy of the data management system.
Preferably, the service module is configured to configure a data source of the data acquisition by the data management system based on the function configuration information, and includes:
and the service module is used for modifying the information of the data source and/or adding the information of the data source based on the function configuration information, and the information of the data source comprises a URL address, an access token, user information, a data source name and/or a system to which the data source belongs.
In a third aspect, an embodiment of the present invention provides a computer device, including a memory and a processor communicatively connected, where the memory is configured to store a computer program, and the processor is configured to read the computer program and execute the data management method as set forth in the foregoing embodiment.
In a fourth aspect, an embodiment of the present invention proposes a computer readable storage medium having stored thereon instructions that, when run on a computer, perform a data management method as proposed by the above embodiment.
The beneficial effects are that: the invention realizes the function modularization of the data management system and solves the problems of large development workload, low efficiency and high cost caused by the fact that the traditional decentralised data management system needs to modify codes when the demands are changed; therefore, the user can dynamically configure the system functions according to the requirements on the basis of not modifying codes, the efficiency is high, the cost is low, and the maintenance is convenient.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
fig. 1 is a flowchart of a data management method according to an embodiment of the present invention.
Detailed Description
In order to more clearly illustrate the embodiments of the invention or the solutions of the prior art, the invention will be briefly described below with reference to the accompanying drawings and the description of the embodiments or the prior art, it being apparent that the following description of the structure of the drawings is only some embodiments of the invention and that other drawings can be obtained from these drawings by a person skilled in the art without inventive effort. The description of the embodiments is provided to assist understanding of the present invention, but is not intended to limit the present invention.
It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of example embodiments of the present invention.
It should be understood that for the term "and/or" that may appear herein, it is merely one association relationship that describes an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a alone, B alone, and both a and B; for the term "/and" that may appear herein, which is descriptive of another associative object relationship, it means that there may be two relationships, e.g., a/and B, it may be expressed that: a alone, a alone and B alone; in addition, for the character "/" that may appear herein, it is generally indicated that the context associated object is an "or" relationship.
In a first aspect, please refer to fig. 1, which illustrates a data management method according to an embodiment of the present invention, which is applied to a data management system; the data management system may be, but is not limited to, executed by a computer device with a certain computing resource, for example, a personal computer (Personal Computer, PC, refer to a multipurpose computer with a size, price and performance suitable for personal use, and desktop, notebook, tablet, and ultrabook all belong to personal computers), a smart phone, a personal digital assistant (Personal digital assistant, PAD), or an electronic device such as a system server. The storage end comprises a main board, a memory and an external data exchange module, wherein the external data exchange module and the memory are respectively connected with the main board, an encryption and decryption unit and an identity data read-only memory unit are arranged in the memory, and the method comprises the following steps: acquiring multi-modal data from a plurality of data sources and cleaning the multi-modal data, wherein the multi-modal data comprises: the interface module provides an access interface for the external application and receives an access request sent by the external application based on the access interface, wherein the access request comprises access user information; the service module is provided with a function configuration interface, wherein the service module communicates with the interface module, determines the access authority of the access user based on the access user information in the access request sent by the interface module, and opens the function configuration interface to the access user when the access authority is the management configuration authority; and receiving function configuration information sent by the access user based on the function configuration interface, and changing the function configuration of the decentralised data management system based on the function configuration information so that the decentralised data management system realizes the dynamic configuration of the system functions based on the function configuration interface. Therefore, the problems of complicated processing steps, low efficiency and high cost caused by the need of modifying codes when the demand of the decentralization data management system is increased or modified in the prior art are solved.
It should be understood that the foregoing execution subject is not limited to the embodiments of the present application, and accordingly, the operation steps of the present method may be, but are not limited to, those shown in the following steps S11 to S13.
Step S11, an interface module provides an access interface for an external application and receives an access request sent by the external application based on the access interface, wherein the access request comprises access user information;
determining an interface specification: the URL defining the interface (Uniform Resource Locator address, an identifier for locating and accessing a website, file, or other resource), the method of the request (e.g., GET, POST, PUT, etc.), parameters, and the structure of the returned results. Realizing interface service: developing server-side programs of the interface module by using a proper back end (such as node. Js, python Django, java Spring and the like); wherein these server-side programs will listen for and process requests from external applications. Identity verification and authentication: an authentication mechanism is implemented in the interface service, for example using JWT (JSON Web Tokens) to authenticate the user identity. Ensuring that only authenticated users can use the interface; furthermore, the access user may be associated with its corresponding rights.
By defining interface specifications, a standardized interface mode is provided, so that an external application program can conveniently interact with the data management system; by implementing an identity verification mechanism, only authorized users can use the interface, and the safety of the system is improved; the design of the interface module allows the system to add new functions and services as needed while also supporting integration and interoperability with other systems.
Step S12, the service module is provided with a function configuration interface, wherein the service module communicates with the interface module, determines the access authority of the access user based on the access user information in the access request sent by the interface module, and opens the function configuration interface to the access user when the access authority is the management configuration authority;
determining a permission rule: defining and establishing a hierarchical structure of access rights, and determining the access rights of each user role; for example, an administrator has administrative configuration rights, and an average user may have only partial read rights. Authentication and verification: in the function configuration interface, access user information in the access request is verified and access rights of the user are determined. This can be verified by an access token (token) and then checking the role and rights of the user. Open function configuration interface: if the accessing user is granted the right to manage the configuration, the functional configuration interface is exposed to the user, allowing the user to modify the configuration of the system using the functional configuration interface.
Fine-grained access control to the function configuration interface is realized by determining and distributing authorities of different user roles; the functional configuration interface can be adjusted according to the needs, so that the workload of code modification and system deployment is reduced; the function configuration interface is opened to the user with the management configuration authority, so that the user can independently configure the system function when the system runs, and the satisfaction degree of the user and the flexibility of the system are improved.
Step S13, the function configuration information sent by the access user is received based on the function configuration interface, and the function configuration of the data management system is changed based on the function configuration information, so that the data management system realizes the dynamic configuration of the system functions based on the function configuration interface.
Defining function configuration information: the data structure and format of the functional configuration are determined to receive configuration information sent by the user. This may be in JSON format or other suitable data format. Receiving and parsing configuration information: in the function configuration interface, function configuration information sent by an access user is received and parsed to obtain specific details of the configuration. Modifying the system configuration: and using the analyzed configuration information to change the functional configuration of the data management system by means of updating a database, modifying a configuration file, adjusting system parameters and the like. Dynamic configuration is realized: according to the new system configuration, the data management system can reload related modules, readjust workflow or apply new rules to realize dynamic configuration, avoiding the need to modify code and redeploy.
By receiving and analyzing the function configuration information from the user, the system can change the function configuration as required, thereby realizing the dynamics and flexibility; the complicated processes of modifying codes and redeploying the system are avoided, and the development and operation cost of the system is reduced; through flexible functional configuration, the system can adjust the running mechanism according to different requirements and scenes, and the response capability and efficiency of the system are improved. The system and the method are beneficial to improving the flexibility, maintainability and user satisfaction of the decentralised data management system, and simultaneously reducing the cost of system development and operation and maintenance. Therefore, the problems of complicated processing steps, low efficiency and high cost caused by the need of modifying codes when the demand of the decentralization data management system is increased or modified in the prior art are solved.
Preferably, the service module alters the function configuration of the data management system based on the function configuration information, including:
the service module configures a data source of the data acquisition by the data management system based on the functional configuration information, configures a data acquisition strategy of the data management system, configures a data consumption interface of the data management system and/or configures a data storage strategy of the data management system.
Configuring a data source for data acquisition of a data management system according to the function configuration information; this may include connecting to external data sources such as a blockchain browser or a ticker website, or configuring internal data sources of the system; and configuring a data acquisition strategy of the data management system according to the function configuration information. This may include configuring the acquisition frequency of data, the acquired data range, the condition of data acquisition, etc.; for example, the real-time data, the history data or the data of the specific block can be configured and acquired according to the configuration requirement of the user; configuring a data consumption interface of the data management system according to the functional configuration information, which may include what format and manner the configuration data is provided to an external application or other system, such as providing data by way of an API interface, message queue, etc.; and configuring a data storage strategy of the data management system according to the function configuration information. This may include selecting an appropriate data storage scheme, such as a relational database, a distributed storage system, etc., and setting corresponding storage rules, such as retention deadlines for data, backup policies, etc.
Through the configuration, the data management system can flexibly configure the source, the acquisition strategy, the consumption interface and the data storage mode of the data according to the requirements and the configuration of the user, thereby meeting the requirements of different business requirements and data management. The specific embodiments may vary depending on the system design, the technical architecture of the data management system, and the business requirements. It is clear that the above configuration items provide general examples, and the implementation needs to be adjusted and expanded according to actual situations.
Preferably, the service module configures a data source of the data acquisition by the data management system based on the function configuration information, and the service module includes:
the service module modifies information of the data source and/or adds information of the data source based on the function configuration information, and the information of the data source comprises URL address, access token, user information, data source name and/or system to which the data source belongs.
When the service module configures the data source for data acquisition by the data management system based on the function configuration information, the service module may modify related information of the existing data source, such as updating URL address, access token, data source name, and the like, according to the function configuration information. This enables the system to dynamically modify existing data source configurations as required. Based on the functional configuration information, the service module may add a new data source configuration. This may involve adding a new data source to the data management system and providing the corresponding URL address, access token, user information, data source name, and information about the system to which it belongs.
The configuration items can provide flexibility and expandability, so that the service module can dynamically configure data sources for data acquisition according to specific requirements, access corresponding URL addresses according to requirements of different data sources, and acquire data by using correct access tokens and user information. In addition, by assigning each data source a name and system to which it belongs, it is possible to facilitate the management and identification of different data sources. The particular implementation may vary, among other things, depending on the architecture and technology choice of the data management system.
Preferably, the service module configures a data acquisition policy of the data management system based on the function configuration information, including:
the service module adds data acquisition policies and/or modifies the data acquisition policies based on the functional configuration information, and the data acquisition policies include a data acquisition policy enforcement engine, data acquisition policy enforcement rules, parameter definitions for data acquisition, and/or regular expressions for data filtering.
When the service module configures the data acquisition policy of the data management system based on the function configuration information, the service module may add a new data acquisition policy based on the function configuration information. This means that a set of rules and conditions can be defined to determine when and how to obtain data. These policies may be determined based on specific business needs and user configurations. In addition to adding new policies, the service module may also modify existing data acquisition policies; this may involve altering the enforcement engine of the policy, updating the enforcement rules of the policy, or adjusting the parameter definitions for data acquisition. The enforcement engine that determines the data acquisition policy may be a rule engine, a workflow engine, or a custom enforcement engine. The enforcement engine is used to interpret and enforce the data acquisition policy. Defining execution rules of the data acquisition policy that describe when and how to trigger data acquisition; for example, the data acquisition may be triggered based on time, or it may be decided whether to acquire data based on certain conditions. Specifying parameter definitions for data acquisition, which parameters are configurable according to the function configuration information; for example, it may be defined to acquire data within a particular time frame, or to acquire data of a particular type or tag. Regular expressions for data filtering are provided. These regular expressions can be used to filter and match the acquired data to meet specific needs.
By configuring the data acquisition strategy, the service module can define and adjust the data acquisition behavior according to the function configuration information, flexibility and customizability are provided, and the system can acquire data according to specific service requirements and configuration requirements of users.
Preferably, the service module configures a data consumption interface of the data management system based on the function configuration information, including: the RPC interface and/or the sql query interface are configured based on the functional configuration information.
When the service module configures the data consumption interface of the data management system based on the function configuration information, the service module may configure an RPC (remote procedure call) interface based on the function configuration information to provide a manner of data consumption. The RPC interface may employ standardized protocols, such as RESTful API, gRPC, etc., for providing access and interaction of data; according to the function configuration information, the service module can configure an SQL query interface, wherein the SQL query interface is a program interface for interacting with the database so as to support query operation of the database. This may be accomplished by defining the query parameters, query statements, and data formats of the returned results.
By configuring the two types of data consumption interfaces, the data management system can provide different data consumption modes, so that external applications can select interfaces suitable for themselves to access and consume data according to the function configuration information.
In a second aspect, an embodiment of the present invention provides a data management system, including:
the interface module is used for providing an access interface for the external application and receiving an access request sent by the external application based on the access interface, wherein the access request comprises access user information;
determining an interface specification: the URL of the interface, the request method (e.g., GET, POST, PUT, etc.), parameters, and the structure of the returned results are defined. Realizing interface service: developing server-side programs of the interface module by using a proper back end (such as node. Js, python Django, java Spring and the like); wherein these server-side programs will listen for and process requests from external applications. Identity verification and authentication: an authentication mechanism is implemented in the interface service, for example using JWT (JSON Web Tokens) to authenticate the user identity. Ensuring that only authenticated users can use the interface; furthermore, the access user may be associated with its corresponding rights.
By defining interface specifications, a standardized interface mode is provided, so that an external application program can conveniently interact with the data management system; by implementing an identity verification mechanism, only authorized users can use the interface, and the safety of the system is improved; the design of the interface module allows the system to add new functions and services as needed while also supporting integration and interoperability with other systems.
The service module is provided with a function configuration interface, wherein the service module is used for communicating with the interface module, determining the access authority of the access user based on the access user information in the access request sent by the interface module, and opening the function configuration interface to the access user when the access authority is the management configuration authority;
determining a permission rule: defining and establishing a hierarchical structure of access rights, and determining the access rights of each user role; for example, an administrator has administrative configuration rights, and an average user may have only partial read rights. Authentication and verification: in the function configuration interface, access user information in the access request is verified and access rights of the user are determined. This can be verified by an access token (token) and then checking the role and rights of the user. Open function configuration interface: if the accessing user is granted the right to manage the configuration, the functional configuration interface is exposed to the user, allowing the user to modify the configuration of the system using the functional configuration interface.
Fine-grained access control to the function configuration interface is realized by determining and distributing authorities of different user roles; the functional configuration interface can be adjusted according to the needs, so that the workload of code modification and system deployment is reduced; the function configuration interface is opened to the user with the management configuration authority, so that the user can independently configure the system function when the system runs, and the satisfaction degree of the user and the flexibility of the system are improved.
The service module is also used for receiving the function configuration information sent by the access user based on the function configuration interface and changing the function configuration of the data management system based on the function configuration information so that the data management system realizes the dynamic configuration of the system functions based on the function configuration interface.
Defining function configuration information: the data structure and format of the functional configuration are determined to receive configuration information sent by the user. This may be in JSON format or other suitable data format. Receiving and parsing configuration information: in the function configuration interface, function configuration information sent by an access user is received and parsed to obtain specific details of the configuration. Modifying the system configuration: and using the analyzed configuration information to change the functional configuration of the data management system by means of updating a database, modifying a configuration file, adjusting system parameters and the like. Dynamic configuration is realized: according to the new system configuration, the data management system can reload related modules, readjust workflow or apply new rules to realize dynamic configuration, avoiding the need to modify code and redeploy.
By receiving and analyzing the function configuration information from the user, the system can change the function configuration as required, thereby realizing the dynamics and flexibility; the complicated processes of modifying codes and redeploying the system are avoided, and the development and operation cost of the system is reduced; through flexible functional configuration, the system can adjust the running mechanism according to different requirements and scenes, and the response capability and efficiency of the system are improved.
The system and the method are beneficial to improving the flexibility, maintainability and user satisfaction of the decentralised data management system, and simultaneously reducing the cost of system development and operation and maintenance. Therefore, the problems of complicated processing steps, low efficiency and high cost caused by the need of modifying codes when the demand of the decentralization data management system is increased or modified in the prior art are solved.
Preferably, the service module alters the function configuration of the data management system based on the function configuration information, including:
the service module is used for configuring a data source for data acquisition of the data management system based on the functional configuration information, configuring a data acquisition strategy of the data management system, configuring a data consumption interface of the data management system and/or configuring a data storage strategy of the data management system.
Configuring a data source for data acquisition of a data management system according to the function configuration information; this may include connecting to external data sources such as a blockchain browser or a ticker website, or configuring internal data sources of the system; and configuring a data acquisition strategy of the data management system according to the function configuration information. This may include configuring the acquisition frequency of data, the acquired data range, the condition of data acquisition, etc.; for example, the real-time data, the history data or the data of the specific block can be configured and acquired according to the configuration requirement of the user; configuring a data consumption interface of the data management system according to the functional configuration information, which may include what format and manner the configuration data is provided to an external application or other system, such as providing data by way of an API interface, message queue, etc.; and configuring a data storage strategy of the data management system according to the function configuration information. This may include selecting an appropriate data storage scheme, such as a relational database, a distributed storage system, etc., and setting corresponding storage rules, such as retention deadlines for data, backup policies, etc.
Through the configuration, the data management system can flexibly configure the source, the acquisition strategy, the consumption interface and the data storage mode of the data according to the requirements and the configuration of the user, thereby meeting the requirements of different business requirements and data management. The specific embodiments may vary depending on the system design, the technical architecture of the data management system, and the business requirements. The above configuration items provide general examples, and the implementation needs to be adjusted and expanded according to actual situations.
Preferably, the service module is configured to configure a data source of the data acquisition by the data management system based on the function configuration information, and includes:
and the service module is used for modifying the information of the data source and/or adding the information of the data source based on the function configuration information, and the information of the data source comprises a URL address, an access token, user information, a data source name and/or a system to which the data source belongs.
When the service module configures the data source for data acquisition by the data management system based on the function configuration information, the service module may modify related information of the existing data source, such as updating URL address, access token, data source name, and the like, according to the function configuration information. This enables the system to dynamically modify existing data source configurations as required. Based on the functional configuration information, the service module may add a new data source configuration. This may involve adding a new data source to the data management system and providing the corresponding URL address, access token, user information, data source name, and information about the system to which it belongs.
The configuration items can provide flexibility and expandability, so that the service module can dynamically configure data sources for data acquisition according to specific requirements, access corresponding URL addresses according to requirements of different data sources, and acquire data by using correct access tokens and user information. In addition, by assigning each data source a name and system to which it belongs, it is possible to facilitate the management and identification of different data sources. The particular implementation may vary, among other things, depending on the architecture and technology choice of the data management system.
In a third aspect, there is also provided a data management computer device, comprising, at a hardware level: a data interface; a memory for storing instructions; a processor, configured to read an instruction stored in the memory, and execute the data management method according to the first embodiment according to the instruction: step S11, an interface module provides an access interface for an external application and receives an access request sent by the external application based on the access interface, wherein the access request comprises access user information; step S12, the service module is provided with a function configuration interface, wherein the service module communicates with the interface module, determines the access authority of the access user based on the access user information in the access request sent by the interface module, and opens the function configuration interface to the access user when the access authority is the management configuration authority; step S13, the function configuration information sent by the access user is received based on the function configuration interface, and the function configuration of the data management system is changed based on the function configuration information, so that the data management system realizes the dynamic configuration of the system functions based on the function configuration interface.
Optionally, the device further comprises an internal bus. The processor and memory and data interfaces may be interconnected by an internal bus, which may be an ISA (Industry Standard Architecture ) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus, or an EISA (Extended Industry Standard Architecture ) bus, among others. The buses may be divided into address buses, data buses, control buses, etc.
The Memory may include, but is not limited to, random access Memory (Random Access Memory, RAM), read Only Memory (ROM), flash Memory (Flash Memory), first-in first-out Memory (First Input First Output, FIFO), and/or first-in last-out Memory (First In Last Out, FILO), etc. The processor may be a general-purpose processor including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
The working process, working details and technical effects of the device according to the fourth embodiment of the present invention can be referred to the first embodiment, and are not described herein.
A fourth embodiment of the present invention provides a computer-readable storage medium storing instructions comprising the data management method of the first embodiment, i.e. the computer-readable storage medium has instructions stored thereon which, when executed on a computer, perform the data management method as the first aspect. The computer readable storage medium refers to a carrier for storing data, and may include, but is not limited to, a floppy disk, an optical disk, a hard disk, a flash Memory, and/or a Memory Stick (Memory Stick), etc., where the computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable devices.
The working process, working details and technical effects of the computer readable storage medium according to the fourth embodiment of the present invention can be referred to the first embodiment, and are not described herein.
A fifth aspect of the present embodiment provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform a data management method as in the first embodiment, wherein the computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus.
The various embodiments described above are illustrative only, in that elements illustrated as separate elements may or may not be physically separate, and elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware systems, or of course by means of hardware. Based on this understanding, the above-described technical solutions may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., comprising several instructions for causing a merging means of warehouse codes to perform the various embodiments or methods of certain parts of the embodiments.
Finally, it should be noted that: the above is only a preferred embodiment of the invention and is not intended to limit the scope of the invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the invention should be included in the protection scope of the invention.
Claims (10)
1. A method of data management, comprising:
the interface module provides an access interface for an external application and receives an access request sent by the external application based on the access interface, wherein the access request comprises access user information;
the service module is provided with a function configuration interface, wherein the service module is in communication connection with the interface module, determines the access right of an access user based on the access user information in the access request sent by the interface module, and opens the function configuration interface to the access user when the access right is a management configuration right;
and receiving function configuration information sent by an access user based on the function configuration interface, and changing the function configuration of the data management system based on the function configuration information so that the data management system realizes dynamic configuration of system functions based on the function configuration interface.
2. The method according to claim 1, wherein the service module changes the function configuration of the data management system based on the function configuration information, comprising:
the service module configures a data source of the data management system for data acquisition based on the function configuration information, configures a data acquisition strategy of the data management system, configures a data consumption interface of the data management system and/or configures a data storage strategy of the data management system.
3. The method of claim 2, wherein the service module configures a data source of the data acquisition by the data management system based on the function configuration information, comprising:
the service module modifies information of the data source and/or adds information of the data source based on the function configuration information, and the information of the data source comprises URL address, access token, user information, data source name and/or system to which the data source belongs.
4. The method according to claim 2, wherein a service module configures a data acquisition policy of the data management system based on the function configuration information, comprising:
the service module adds a data acquisition strategy and/or modifies the data acquisition strategy based on the function configuration information, and the data acquisition strategy comprises a data acquisition strategy execution engine, a data acquisition strategy execution rule, parameter definition for data acquisition and/or a regular expression for data filtering.
5. The method of claim 2, wherein a service module configures a data consumption interface of the data management system based on the function configuration information, comprising: and configuring an RPC interface and/or an sql query interface based on the function configuration information.
6. A data management system, comprising:
the interface module is used for providing an access interface for an external application and receiving an access request sent by the external application based on the access interface, wherein the access request comprises access user information;
the service module is provided with a function configuration interface, wherein the service module is in communication connection with the interface module and is used for determining the access authority of an access user based on the access user information in the access request sent by the interface module, and opening the function configuration interface to the access user when the access authority is the management configuration authority;
the service module is also used for receiving the function configuration information sent by the access user based on the function configuration interface, and changing the function configuration of the data management system based on the function configuration information so that the data management system realizes the dynamic configuration of the system functions based on the function configuration interface.
7. The data management system of claim 6, wherein the service module alters a functional configuration of the data management system based on the functional configuration information, comprising:
the service module is used for configuring a data source of the data management system for data acquisition based on the function configuration information, configuring a data acquisition strategy of the data management system, configuring a data consumption interface of the data management system and/or configuring a data storage strategy of the data management system.
8. The data management system of claim 7, wherein the service module for configuring the data source of the data acquisition by the data management system based on the functional configuration information comprises:
and the service module is used for modifying the information of the data source and/or adding the information of the data source based on the function configuration information, and the information of the data source comprises a URL address, an access token, user information, a data source name and/or a system to which the data source belongs.
9. A computer device comprising a memory and a processor in communication, wherein the memory is adapted to store a computer program and the processor is adapted to read the computer program and to perform a data management method as claimed in any one of claims 1 to 5.
10. A computer readable storage medium having instructions stored thereon which, when run on a computer, perform the data management method according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311623864.4A CN117792902A (en) | 2023-11-30 | 2023-11-30 | Data management method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311623864.4A CN117792902A (en) | 2023-11-30 | 2023-11-30 | Data management method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117792902A true CN117792902A (en) | 2024-03-29 |
Family
ID=90386188
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311623864.4A Pending CN117792902A (en) | 2023-11-30 | 2023-11-30 | Data management method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117792902A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116049860A (en) * | 2023-03-06 | 2023-05-02 | 深圳前海环融联易信息科技服务有限公司 | Access control method, device, computer equipment and storage medium |
| CN116707839A (en) * | 2022-02-24 | 2023-09-05 | 中移(上海)信息通信科技有限公司 | Resource access management method, network device, system and storage medium |
| CN116910336A (en) * | 2023-08-02 | 2023-10-20 | 上海戳记科技有限公司深圳分公司 | Dynamic encrypted data acquisition method, system, computer equipment and storage medium |
-
2023
- 2023-11-30 CN CN202311623864.4A patent/CN117792902A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116707839A (en) * | 2022-02-24 | 2023-09-05 | 中移(上海)信息通信科技有限公司 | Resource access management method, network device, system and storage medium |
| CN116049860A (en) * | 2023-03-06 | 2023-05-02 | 深圳前海环融联易信息科技服务有限公司 | Access control method, device, computer equipment and storage medium |
| CN116910336A (en) * | 2023-08-02 | 2023-10-20 | 上海戳记科技有限公司深圳分公司 | Dynamic encrypted data acquisition method, system, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230396669A1 (en) | Technologies for transparent function as a service arbitration for edge systems | |
| US20140289243A1 (en) | Method and system for centralized issue tracking | |
| CN113407997A (en) | Data desensitization method and device, computer equipment and storage medium | |
| US9832322B1 (en) | Techniques to transform network resource requests to zero rated network requests | |
| CN111782652B (en) | Data calling method, device, computer equipment and storage medium | |
| CN112930530B (en) | Client application for network application execution | |
| WO2022156087A1 (en) | Data blood relationship establishing method and apparatus, computer device, and storage medium | |
| CN111190750B (en) | Data processing method and system | |
| US20070245314A1 (en) | Method, system and computer program for collecting inventory information through a data mover | |
| CN112650804B (en) | Big data access method, device, system and storage medium | |
| CN117792902A (en) | Data management method, device, equipment and medium | |
| CN116186649A (en) | Cross-system access method, device, computer equipment and storage medium | |
| US10897463B2 (en) | Managing and securing manageable resources in stateless web server architecture using servlet filters | |
| CN111353766A (en) | Service process processing system and method of distributed service system | |
| CN112953996B (en) | Service method and system thereof | |
| US20230247111A1 (en) | Runtime module conversion | |
| US20240103818A1 (en) | Annotation driven just in time and state-based rbac policy control | |
| CN112799775B (en) | Node attribute transfer method and related device | |
| CN115361285B (en) | Method, device, equipment and medium for realizing off-line service mixed deployment | |
| EP3023891B1 (en) | Techniques to transform network resource requests to zero rated network requests | |
| CN116776842A (en) | Data processing method, device, computer equipment and readable storage medium | |
| CN116795492A (en) | Resource scheduling method, device and equipment of cloud platform and readable storage medium | |
| CN113778392A (en) | Method and device for realizing streaming development based on configuration | |
| CN114579085A (en) | API (application program interface) development method and device of gateway | |
| CN115526731A (en) | Task batch processing method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |