CN115982692A - Data management application method based on big data technology - Google Patents
Data management application method based on big data technology Download PDFInfo
- Publication number
- CN115982692A CN115982692A CN202211468781.8A CN202211468781A CN115982692A CN 115982692 A CN115982692 A CN 115982692A CN 202211468781 A CN202211468781 A CN 202211468781A CN 115982692 A CN115982692 A CN 115982692A
- Authority
- CN
- China
- Prior art keywords
- data
- user
- node
- module
- data source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Storage Device Security (AREA)
Abstract
The invention discloses a data management application method based on big data technology, which comprises the following steps: the user logs in for verification after identity registration in the system and carries out access verification according to the set authority; the user identity registration module is used for acquiring the identity and the authority of accessing the data management system; the authority setting module divides user roles and corresponding authority functions according to the functional requirements of different users on the system; after acquiring the role users of the system, the data supplier creates the data nodes and adds the data nodes into the system cluster network so as to use the system to manage and share the data source; the data supplier issues the data source after establishing the data source connection information; the application developer can check the published data source information through the client and submit a request for applying the data source to develop the data application, and the application developer can use the data source after the application is successful; the invention has the characteristics of data security management and control and improvement of effective data management capacity.
Description
Technical Field
The invention relates to the technical field of data management, in particular to a data management application method based on a big data technology.
Background
With the rapid development of information technology, big data is not shared by a certain part of people any more, and everyone is put into the production and use of data, so that the concept of big data is generated, the big data enables the collection of data to be more convenient, the processing of data to be faster, and the application of data to be wider.
The data attention of individuals or enterprises is gradually deepened, and finally the data barrier problem is revealed. In order to break through the data barrier and fully exploit the potential value of the data stored in a distributed manner, a plurality of researchers are invested in a data sharing solution, but the current data sharing solution focuses on ensuring that the data is not exported to realize the combined application of the data, but a data provider lacks effective management capacity in the whole data sharing process and has security threat caused by data deviation from control. Therefore, it is necessary to design a data management application method based on big data technology for data security management and data effective management capability improvement.
Disclosure of Invention
The present invention aims to provide a data management application method based on big data technology to solve the problems proposed in the background art.
In order to solve the technical problems, the invention provides the following technical scheme: the data management application method based on the big data technology comprises the following operation steps:
the user logs in for verification after identity registration in the system and carries out access verification according to the set authority;
after acquiring the role users of the system, the data provider creates a data node and adds the data node into a system cluster network so as to use the system to manage and share a data source;
the data supplier establishes data source connection information and then issues the data source;
the application developer can check the published data source information through the client and submit a request for applying the data source to develop the data application, and the application developer can use the data source after the application is successful.
According to the technical scheme, the steps that the user logs in for verification after identity registration in the system and performs access verification according to the set authority comprise:
the user identity registration module is used for acquiring the identity and the authority of accessing the data management system;
the authority setting module divides user roles and corresponding authority functions according to the functional requirements of different users on the system;
the user accesses the system through the system client, and the access verification module performs safe access verification on the user access according to the user identity authentication.
According to the technical scheme, the step of using the system to manage the shared data source by creating the data node and adding the data node into the system cluster network after the data supplier acquires the role user of the system comprises the following steps:
a data supplier applies for creating a new data node through a data node registration function provided by a system data node management module;
a data supplier starts a data node based on a data node network access toolkit, so that the data node is added into a system cluster;
the data provider may have selective control over the deactivation or use of data resources.
According to the technical scheme, the step of issuing the data source after the data supplier creates the data source connection information comprises the following steps:
a data supplier user initiates a request for establishing data source connection information;
the data supplier sends a request for confirming the data resource issuing to the data resource issuing module through the system client.
According to the technical scheme, the application developer can check the published data source information through the client and submit a request for applying the data source to develop the data application, and the application developer can use the data source information after the application is successful, and the application developer comprises the following steps:
the application developer can check the published data source information through the system client and submit a request for applying for data resource development data application.
According to the above technical solution, the system applied by the method comprises:
the user management module is used for managing users using the system;
the data node management module is used for managing the data nodes;
and the data source management module is used for managing the data source.
According to the above technical solution, the user management module includes:
the identity registration module is used for the user to register the identity;
the user authentication module is used for authenticating when a user logs in the system;
the authority setting module is used for setting the authority according to the user identity;
and the user access authentication module is used for carrying out identity verification on the access user.
According to the above technical solution, the data node management module includes:
the node registration module is used for carrying out node registration;
the node state detection module is used for detecting the node state;
and the node operation management module is used for managing the node operation state.
According to the above technical solution, the data source management module includes:
the data resource connection module is used for creating data source connection information;
the data resource publishing module is used for publishing the data resources;
and the user application and use module is used for applying and using the data source by the user.
Compared with the prior art, the invention has the following beneficial effects: according to the invention, through the arrangement of the user management module, the data node management module and the data source management module, a user logs in for verification after identity registration in the system, performs access verification according to set permissions, and divides user roles and corresponding permission functions according to the functional requirements of different users on the system; after acquiring the role users of the system, the data supplier creates the data nodes and adds the data nodes into the system cluster network so as to use the system to manage and share the data source; the application developer can check the published data source information through the client and submit a request for applying the data source to develop the data application, and the application developer can use the data source after the application is successful; the description information of the data is comprehensively published as much as possible, the data is still stored in a database controlled by a data supplier and is not streamed outwards, the requirement of the data supplier for sharing the data is met, the safety of the original data is guaranteed, the safety threat caused by the fact that the data is separated from the control due to the lack of effective management capacity in the data sharing process is avoided, the safety control of the data is realized, and the data management capacity is effectively improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
fig. 1 is a flowchart of a data management application method based on big data technology according to an embodiment of the present invention;
fig. 2 is a schematic block diagram of a data management application system based on big data technology according to a second embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first embodiment is as follows:
fig. 1 is a flowchart of a data management application method based on a big data technology according to an embodiment of the present invention, where the present embodiment is applicable to an environment of data development and sharing, and the method can be executed by the data management application method based on the big data technology according to the embodiment of the present invention, and the system is composed of a plurality of software and hardware modules, and the method specifically includes the following steps:
s101, a user logs in for verification after identity registration in a system and carries out access verification according to set authority;
in some embodiments of the present invention, the user identity registration module obtains the identity and the authority to access the data management system, and only the user registered with the identity can access the system resources in the role authority allocated to the user management module.
Specifically, in the embodiment of the present invention, a user initiates a user registration request to a system through a system client interface, where the user registration request includes a user name, a user login password, and an application role; the system user management module receives a user identity registration request, creates a unique identifier for the user, encrypts a user login password through an MD5 algorithm to generate a ciphertext, and prevents user privacy information from being leaked; and coordinating other modules of the system according to the roles applied by the user, creating user information required by different roles, binding role authority, and storing the user information.
In some embodiments of the present invention, the permission setting module divides the user roles and the corresponding permission functions according to the functional requirements of different users on the system.
Illustratively, in the embodiment of the present invention, users accessing the system are divided into four categories: a system administrator, a data supplier, an application developer and a data user; the system administrator is a user role generally trusted by all data suppliers, and has a decisive role in the safety of the whole system to a certain extent, and the authority position of the user role is in the highest level in all roles and has all the authorities of all subordinate user roles; the data supplier role user not only provides the public shared data source in the system, but also has the functions of managing data nodes and data sources, and the data application is used as an application program for using the data source, and the data supplier also needs to manage the distribution of the data application using data authority and the user access verification; the application developer is the one who mines the value of the data, designs the data application mode and develops the data application program according to the data which is publicly shared by the data supplier; the data user uses the API provided by the data application to obtain the role of the corresponding data service.
In some embodiments of the invention, a user accesses the system through a system client, the access verification module performs security access verification on user access according to user identity authentication, responds to a user request after checking and judging whether the user has information access authority, and feeds back a request result to the user.
S102, after acquiring the role users of the system, the data provider creates the data nodes and adds the data nodes into a system cluster network to use the system to manage and share the data source;
in some embodiments of the present invention, a data supplier applies for creating a new data node through a data node registration function provided by a system data node management module.
Specifically, in the embodiment of the present invention, a data provider user logs in the system, sends a request for applying for registering a data node to the data node management module, creates a unique identity for the data node after authentication of an identity authority, issues a node digital certificate, and stores the node digital certificate in the data node table.
In some embodiments of the present invention, the data provider starts the data node to join the system cluster based on a data node networking toolkit, where the networking toolkit includes a networking script file, a node digital certificate, and a system trusted root certificate.
Specifically, in the embodiment of the present invention, after the system management end and the data node end complete the bidirectional identity authentication and establish the connection through the certificate, the node identity authentication information is verified to determine that the node is legally registered by the data provider; after receiving the node heartbeat sent by the node state detector running at the data node end, the data node management module confirms that the data node is successfully accessed to the network and is in a healthy running state, and updates the node state according to the node heartbeat; the data node management module inquires the node state periodically through a node state detector and updates the node state in a data node table; when a subsequent data node accesses a system management terminal interface, node identity authentication carried in a valid period is required, the valid period can be set by an administrator, and a node which is not reconnected after exceeding the valid period is judged to be invalid and needs a data supplier to operate the data node again to access the network.
In some embodiments of the present invention, after the data node is deactivated by the data provider, the data node management module updates the node status to deactivated, and clears the node authentication until the data provider activates the data node to reapply the node authentication, updating the authentication validity period and the node status.
S103, the data supplier issues the data source after establishing the data source connection information;
in some embodiments of the present invention, a data provider user initiates a request for creating data source connection information, including database access information and a data source connection name, a data resource connection module receives the request, tests a corresponding database state according to the database access information, checks whether a database user in the access information has a sub-account for creating a database, gives an authority to acquire data to the sub-account, and then returns a result to the data resource connection module, if the result is that the database exists and can be connected, the request is allowed, otherwise, the user request is rejected.
Illustratively, in the embodiment of the present invention, the database access information is used as the privacy data of the data provider user, and after receiving the database access information of the data provider, the data source connection management module of the system encodes the information into a byte stream, and then stores the byte stream in a storage and transmission mode in a character string mode, so as to ensure that the data information is stored and transmitted safely.
In some embodiments of the present invention, a data supplier initiates a request for confirming issuing of data resources to a data resource issuing module through a system client, and after receiving a request for confirming issuing of data resources submitted by the data supplier, the data description information is collated into data sources, and then is persistently stored; the data source is not original data, and the original data is also stored in a database of a data supplier and is not delivered out of the database, so that the requirement of the data supplier for sharing data is met, and the safety of the original data is guaranteed.
S104, the application developer can check the published data source information through the client and submit a request for applying the data source to develop data application, and the application developer can use the data source after the application is successful;
in some embodiments of the invention, the application developer may view the published data source information through the system client and submit a request for applying for data resources to develop a data application.
Specifically, in the embodiment of the present invention, it is first verified whether the user role has the authority to apply for the data source, whether the data source exists is checked according to the data source identifier sent by the data provider along with the request, and finally, the data application use information is recorded and stored in a storage, and the data provider to which the data source belongs waits for the audit, so as to obtain the certificate of applying the data source, and if the audit is passed, the connection information of the corresponding data source is obtained, and the certificate of using the data source is applied to the corresponding data node side, and the application use information is recorded and stored.
Example two:
an embodiment of the present invention provides a data management application system based on a big data technology, and fig. 2 is a schematic diagram of a module configuration of the data management application system based on the big data technology provided in the embodiment two, as shown in fig. 2, the system includes:
the user management module is used for managing users using the system;
the data node management module is used for managing the data nodes;
the communication connection module is used for establishing communication connection with a system management end, managing resource exchange between the node end and the management end and calling interfaces among all components in the node;
and the data source management module is used for managing the data source.
In some embodiments of the invention, the user management module comprises:
the identity registration module is used for the user to register the identity;
the user authentication module is used for authenticating when a user logs in the system;
the authority setting module is used for setting the authority according to the user identity;
and the user access authentication module is used for carrying out identity verification on the access user.
In some embodiments of the invention, the data node management module comprises:
the node registration module is used for carrying out node registration;
the node state detection module is used for detecting the node state;
and the node operation management module is used for managing the node operation state.
In some embodiments of the invention, the data source management module comprises:
the data resource connection module is used for creating data source connection information;
the data resource publishing module is used for publishing the data resources;
and the user application and use module is used for applying and using the data source by the user.
It should be noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. The data management application method based on the big data technology is characterized in that: the method comprises the following steps:
the user logs in for verification after identity registration in the system and carries out access verification according to the set authority;
after acquiring the role users of the system, the data provider creates a data node and adds the data node into a system cluster network so as to use the system to manage and share a data source;
the data supplier issues the data source after establishing the data source connection information;
the application developer can check the published data source information through the client and submit a request for applying the data source to develop the data application, and the application developer can use the data source after the application is successful.
2. The big data technology based data management application method according to claim 1, wherein: the steps that the user logs in for verification after identity registration in the system and carries out access verification according to the set authority comprise:
the user identity registration module is used for acquiring the identity and the authority of accessing the data management system;
the authority setting module divides user roles and corresponding authority functions according to the functional requirements of different users on the system;
the user accesses the system through the system client, and the access verification module performs safe access verification on the user access according to the user identity authentication.
3. The big data technology based data management application method according to claim 1, wherein: after the data supplier acquires the role user of the system, the step of using the system to manage the shared data source by creating the data node and adding the data node into the system cluster network comprises the following steps:
a data supplier applies for creating a new data node through a data node registration function provided by a system data node management module;
the data supplier starts the data nodes based on the data node network access toolkit, so that the data nodes are added into the system cluster;
the data provider may have selective control over the deactivation or use of data resources.
4. The big data technology-based data management application method according to claim 1, wherein: the step of issuing the data source after the data supplier creates the data source connection information comprises the following steps:
a data supplier user initiates a request for establishing data source connection information;
the data supplier sends a request for confirming the data resource issuing to the data resource issuing module through the system client.
5. The big data technology-based data management application method according to claim 1, wherein: the application developer can check the published data source information through the client and submit a request for applying the data source to develop the data application, and the application developer can use the data source information after the application is successful, and the application developer comprises the following steps:
the application developer can check the published data source information through the system client and submit a request for applying for data resource development data application.
6. The big data technology-based data management application method according to claim 1, wherein: the system applied by the method comprises the following steps:
the user management module is used for managing users using the system;
the data node management module is used for managing the data nodes;
and the data source management module is used for managing the data source.
7. The big data technology based data management application system of claim 6, wherein: the user management module comprises:
the identity registration module is used for the user to register the identity;
the user authentication module is used for authenticating when a user logs in the system;
the authority setting module is used for setting the authority according to the user identity;
and the user access authentication module is used for carrying out identity verification on the access user.
8. The big data technology-based data management application system of claim 6, wherein: the data node management module comprises:
the node registration module is used for carrying out node registration;
the node state detection module is used for detecting the node state;
and the node operation management module is used for managing the node operation state.
9. The big data technology-based data management application system of claim 6, wherein: the data source management module comprises:
the data resource connection module is used for creating data source connection information;
the data resource publishing module is used for publishing the data resources;
and the user application and use module is used for applying and using the data source by the user.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211468781.8A CN115982692A (en) | 2022-11-22 | 2022-11-22 | Data management application method based on big data technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211468781.8A CN115982692A (en) | 2022-11-22 | 2022-11-22 | Data management application method based on big data technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115982692A true CN115982692A (en) | 2023-04-18 |
Family
ID=85961413
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211468781.8A Pending CN115982692A (en) | 2022-11-22 | 2022-11-22 | Data management application method based on big data technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115982692A (en) |
-
2022
- 2022-11-22 CN CN202211468781.8A patent/CN115982692A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111541656B (en) | Identity authentication method and system based on converged media cloud platform | |
| US20210314312A1 (en) | System and method for transferring device identifying information | |
| US7308502B2 (en) | Method and architecture to provide client session failover | |
| US6490679B1 (en) | Seamless integration of application programs with security key infrastructure | |
| US8726342B1 (en) | Keystore access control system | |
| RU2670778C1 (en) | Forming the key depending on the parameter | |
| CN112422532A (en) | Business communication method, system, device and electronic equipment | |
| CN111314340B (en) | Authentication method and authentication platform | |
| CN112632164B (en) | Universal cross-chain programming interface method for realizing trusted authority access | |
| CN110856174B (en) | Access authentication system, method, device, computer equipment and storage medium | |
| CN108965342B (en) | Authentication method and system for data requester to access data source | |
| CN103152179A (en) | Uniform identity authentication method suitable for multiple application systems | |
| CN101729541B (en) | Method and system for accessing resources of multi-service platform | |
| CN106341428A (en) | Cross-domain access control method and system | |
| CN113378227A (en) | Information data resource sharing system and method | |
| CN114928460A (en) | Multi-tenant application integration framework system based on micro-service architecture | |
| RU2415466C1 (en) | Method of controlling identification of users of information resources of heterogeneous computer network | |
| CN114117264A (en) | Illegal website identification method, device, equipment and storage medium based on block chain | |
| CN114338242A (en) | Cross-domain single sign-on access method and system based on block chain technology | |
| CN115842680A (en) | Network identity authentication management method and system | |
| US20110321134A1 (en) | Consigning Authentication Method | |
| US8726335B2 (en) | Consigning authentication method | |
| KR101736157B1 (en) | Method and Apparatus for Federated Authentication | |
| CN110740122B (en) | Method and device for improving safety of data warehouse | |
| CN112291244A (en) | Multi-tenant method for industrial production data real-time processing platform system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |