CN106599195B - Metadata synchronization method and system under massive network data environment - Google Patents
Metadata synchronization method and system under massive network data environment Download PDFInfo
- Publication number
- CN106599195B CN106599195B CN201611155171.7A CN201611155171A CN106599195B CN 106599195 B CN106599195 B CN 106599195B CN 201611155171 A CN201611155171 A CN 201611155171A CN 106599195 B CN106599195 B CN 106599195B
- Authority
- CN
- China
- Prior art keywords
- metadata
- node server
- version number
- child node
- central
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the invention provides a metadata synchronization method and a metadata synchronization system under a massive network data environment, wherein the method comprises the following steps: after the metadata in a metadata database table in the central node server is updated from a first stable version number to a second stable version number, a notification message is sent to the child node management equipment; wherein the second stable version number is higher than the first stable version number; after receiving the notification message, the child node management device acquires the metadata database table from the central node server, wherein the version number of metadata in the metadata database table is the second stable version number; and the child node management equipment sends the acquired metadata database table to a child node server so as to synchronize metadata in the metadata database tables of the central node server and the child node server. According to the embodiment of the invention, the metadata in the metadata database table in the metadata synchronization system can be synchronized in time.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a metadata synchronization method and a metadata synchronization system under a massive network data environment.
Background
Metadata, broadly speaking, refers to all basic resource data. In the field of metadata management, there have been some relatively sophisticated management schemes applied to metadata management systems with different backgrounds, different types of data stored, and different requirements on data accuracy.
At present, the method for managing metadata mainly includes: maintenance personnel of the metadata database table perform operations of adding, deleting, modifying, inquiring and the like on the metadata of the metadata database table stored in the central node server according to the change of the metadata, and a local administrator downloads the metadata database table from the central node server at a preset period and stores the downloaded metadata database table to the child node servers; when a user needs to obtain the metadata database table in the child node server, the metadata database table is obtained from the child node server, and data analysis is carried out.
It can be seen that after the maintenance personnel perform operations such as adding, deleting, modifying, querying and the like on the metadata of the metadata database table, the local administrator obtains the metadata database table in a preset period, and when the time interval of the preset period is long, the local administrator cannot obtain the metadata database table of the latest version in time, so that the metadata database tables in the child node servers are inconsistent with the metadata in the metadata database table in the central node server.
Disclosure of Invention
The embodiment of the invention aims to provide a metadata synchronization method and a metadata synchronization system under a massive network data environment so as to realize timely synchronization of metadata in a metadata database table in a metadata synchronization system. The specific technical scheme is as follows:
on one hand, the embodiment of the invention discloses a metadata synchronization method under a massive network data environment, which is applied to a metadata synchronization system, wherein the metadata synchronization system comprises: the method comprises the following steps of:
after the metadata in the metadata database table in the central node server is updated from a first stable version number to a second stable version number, a notification message is sent to the child node management equipment; wherein the second stable version number is higher than the first stable version number;
after receiving the notification message, the child node management device acquires the metadata database table from the central node server, wherein the version number of metadata in the metadata database table is the second stable version number;
and the child node management equipment sends the acquired metadata database table to a child node server so as to synchronize metadata in the metadata database tables of the central node server and the child node server.
Optionally, before the metadata in the metadata database table in the central node server is updated from the first stable version number to the second stable version number, the method further includes:
after first metadata in a metadata database table in the central node server is updated to second metadata, the central node server updates a first temporary version number of the first metadata to a second temporary version number, wherein the first metadata is: metadata having a first stable version number, the second temporary version number being higher than the first temporary version number.
Optionally, after the central node server updates the first temporary version number of the first metadata to the second temporary version number, the method further includes:
the central node server sends an authentication request to central management equipment;
updating the metadata in the metadata database table in the central node server from a first stable version number to a second stable version number, wherein the updating comprises the following steps:
when the central management equipment receives the authentication request, the central management equipment sends a first message to the central node server, wherein the first message is a message for receiving the first metadata and updating the first metadata into the second metadata;
and the central node server updates the metadata in the metadata database table from the first stable version number to the second stable version number according to the first message.
Optionally, before the metadata in the metadata database table in the central node server is updated from the first stable version number to the second stable version number, the method further includes:
when the central management equipment receives the authentication request, the central management equipment sends a second message to the central node server, wherein the second message is a message for refusing the first metadata to be updated into the second metadata;
and the central management equipment maintains a first stable version number of the metadata in the metadata table according to the second message, and updates the second metadata into the first metadata.
Optionally, after the updating the metadata in the metadata library table from the first stable version number to the second stable version number, the method further includes:
the central node server sets the first metadata from the second temporary version number to an initial temporary version number, wherein the initial temporary version number is a temporary version number of the first metadata before the first metadata is updated, and the value of the initial temporary version number is a preset value;
after the central node server maintains a first stable version number of metadata within the metadata database table, the method further comprises:
and the central node server updates the first metadata from the second temporary version number to the initial temporary version number.
Optionally, the method further includes:
when the child node server detects that a user accesses the child node server through terminal equipment, recording operation information of the user for operating the terminal equipment in a stored first log information table; the operation information includes: login time, logout time, login IP address;
the central management equipment acquires the first log information base table in the child node server;
and the central management equipment sends the first log information table to the central node server so as to synchronize the operation information of the first log information tables of the central node server and the child node servers.
Optionally, the method further includes:
the child node management equipment allocates a user name and a password to the terminal equipment, so that the user can access the child node server through the terminal equipment according to the user name and the password;
when the child node server receives an access request of a user for accessing the child node server through the terminal equipment, matching the user name and the password in a user account information table in the child node server; when the user name is matched with the user name in the user account information base table and the password is matched with the password in the user account information base table, the child node server receives the access request; the access request carries the user name and the password;
and the central management equipment acquires the user account information base table in the child node server and sends the user account information base table to the central node server so as to synchronize the user account information base tables of the central node server and the child node server.
Optionally, after the first metadata in the metadata database table in the central node server is updated to the second metadata, the method further includes:
the central node server records the updating information of the first metadata updated into the second metadata in a stored second log information base table; the update information includes: time updating and IP address updating.
On the other hand, the embodiment of the invention also discloses a metadata synchronization system under the environment of mass network data, which comprises: the system comprises a central node server, child node management equipment and child node servers;
wherein the content of the first and second substances,
the central node server is used for updating the temporary version number and the stable version number of the metadata in the metadata database table; and a notification message is sent to the child node management device; and updating the second log information base table;
the child node management device is configured to receive a notification message sent by the central node server, and obtain the metadata database table from the central node server according to the notification message; the metadata database table is used for importing the metadata database table to the child node server; the system comprises a child node server, a central node server and a sub-node server, wherein the child node server is used for acquiring a first log information base table and a user account information base table from the child node server and importing the first log information base table and the user account information base table into the central node server;
the child node server is used for storing the metadata database table; and the database table is used for storing and updating the first log information database table and the user account information database table.
Optionally, the system is further specifically adapted to,
after the first metadata in the metadata database table in the central node server is updated to the second metadata, the central node server updates the first temporary version number of the first metadata to a second temporary version number.
The embodiment of the invention provides a metadata synchronization method and a metadata synchronization system under a massive network data environment, wherein the method comprises the following steps: when the stable version number of the metadata in the metadata database table in the central node server is updated in the metadata synchronization system, a notification message is sent to the child node management device, so that the child node management device obtains the metadata database table after the metadata update from the central node server, and the metadata database table is imported into the child node server after the child node management device obtains the metadata database table, so that the metadata in the metadata database table in the child node server is the same as the metadata in the metadata database table in the central node server. Therefore, according to the scheme, when the stable version of the metadata in the metadata database table is updated, the user can timely acquire the updated stable version of the metadata in the metadata database table, and the metadata database table is stored between the child node server and the central node server, so that backup disaster tolerance is facilitated. When a maintainer operates the metadata in the metadata database table in the central node server, the authority of authentication management is increased, and the security of the metadata is ensured. Therefore, the embodiment of the invention ensures the consistency, reliability, accuracy and timeliness of the metadata. It is not necessary for any product or method of practicing the invention to achieve all of the above-described advantages at the same time.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is an architecture diagram of a metadata synchronization system in a massive network data environment according to an embodiment of the present invention;
FIG. 2 is a flowchart of a metadata synchronization method in a massive network data environment according to an embodiment of the present invention;
fig. 3 is a software architecture diagram of a central node server in a metadata synchronization system in a massive network data environment according to an embodiment of the present invention;
FIG. 4 is another flowchart of a metadata synchronization method in a massive network data environment according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating a metadata synchronization method in a massive network data environment according to an embodiment of the present invention;
fig. 6 is a flowchart of a metadata synchronization method in a massive network data environment according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a metadata synchronization system in a massive network data environment according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to solve the problem of the prior art, embodiments of the present invention provide a metadata synchronization method and system in a massive network data environment, so as to implement timely synchronization of metadata in a metadata database table in a metadata synchronization system.
First, a metadata synchronization method in a massive network data environment according to an embodiment of the present invention is described below.
It should be noted that, the metadata synchronization method in the massive network data environment provided by the embodiment of the present invention is applied to a metadata synchronization system, as shown in fig. 1, an architecture of the metadata synchronization system in the massive network data environment provided by the embodiment of the present invention, where the metadata synchronization system includes: the system comprises a central node server 101, at least one child node management device 102 and at least one child node server 103, wherein the central node server can be a high-performance computer, the central node server serves as a node of a network and stores and processes a large amount of data and information on the network, the child node server can have the same or similar performance and function as the central node server, and the child node management device can be a notebook computer, a tablet computer or a desktop computer.
As shown in fig. 2, a metadata synchronization method in a massive network data environment according to an embodiment of the present invention is applied to a metadata synchronization system, where the metadata synchronization system includes: the method comprises the following steps:
s201, after the metadata in a metadata database table in a central node server is updated from a first stable version number to a second stable version number, a notification message is sent to the child node management equipment;
wherein the second stable version number is higher than the first stable version number;
when the metadata in the metadata database table in the central node server is changed, namely the metadata is updated to some extent, the stable version number of the metadata is updated, and therefore when the metadata is kept unchanged, the stable version number corresponding to the metadata is not changed, and when the metadata is operated, namely the metadata is updated, the stable version number of the metadata is changed from the first stable version number to the second stable version number. It is understood that the first stable version number is a version number before updating the metadata, and the second stable version number is a version number after updating the metadata, for example, the first stable version number is "3", the second stable version number is "4", the stable version number of the metadata is "3" before updating the metadata, the stable version number of the metadata is "4" after updating the metadata, and in general, the stable version number of the metadata after updating the metadata is larger than the stable version number of the metadata before updating the metadata.
It should be noted that, the central node server plays a main function in the metadata synchronization system, and referring to fig. 3, fig. 3 is a software architecture diagram of the central node server in the metadata synchronization system in the massive network data environment provided by the embodiment of the present invention, where the software architecture includes: client software architecture 310 and server software architecture 320, where the client software architecture 310 is used for interacting with a user using a central node server, and the server software architecture 320 is used for receiving a request from the client software architecture 310, processing specific service logic, and reading and writing data in the database 330 by the user.
Specifically, the client software architecture 310 includes: a Web UI (Web User Interface) Interface 311, a control layer 312, and a data model layer 313, where the Web UI Interface is a part that a User using a central node server can directly operate on a Web page, and the Web UI Interface can realize purposeful, organized, and normalized display of data and information. Here, the user using the center node server includes: the method comprises the following steps that maintenance personnel of a metadata database table, center management personnel and child node equipment management personnel can operate the metadata database table through a Web UI interface, and the operation comprises the following steps: view, add, delete, or modify metadata in the metadata database table. The central manager can authenticate the operation of adding, modifying and deleting the metadata database table by the maintainer through a Web UI interface, and the child node equipment manager can download the metadata database table in the central node server through the Web UI interface.
The data model layer is used for encapsulating metadata related to business logic of the application program and a processing method of the metadata, and the data model layer is related to the metadata and an organization form of the metadata. The data model layer obtains metadata required by a user of the central node server from a server software architecture through the Internet, organizes the metadata into a corresponding data model, and simultaneously provides the data model for a Web UI interface. For example, the Web UI interfaces of all the maintenance personnel operating the metadata database table in a related summary and the detailed Web UI interfaces of the maintenance personnel operating the metadata database table may share one data model layer, that is, the data provides the organized metadata to the two Web UI interfaces at the same time.
The control layer is positioned between the Web UI interface and the data model layer, serves as a bridge and a link of the two layers, and is mainly used for processing events and responding. The "event" herein includes both a user operation that occurs on the Web UI interface and a change in data itself or data organization form that occurs on the data model layer. When these events occur, the control layer must respond at a first time, either by passing a user request for the Web UI interface to the data model layer, or by feeding back changes to the data occurring at the data model layer to the Web UI interface for display. The control layer essentially completes all the business logic of the client 340.
The server-side software architecture 320 may include: presentation layer 321, logical layer 322, and persistent layer 323. And each metadatabase table corresponds to a server-side software architecture of a presentation layer, a logic layer and a persistence layer respectively.
The method for the HTTP (HyperText Transfer Protocol) request of the client may include GET, PUT, POST, DE L ETE, etc., the presentation layer directs different requests to corresponding PATHs through a preset method (e.g., @ PATH annotation, also called PATH annotation), and receives and processes the requests through different API (Application Programming Interface) interfaces.
The logic layer is the core of the server-side software architecture, and is responsible for processing specific service logic, so that the main functions of the system are realized. The service logic of the metadata synchronization system is quite complex, for example, a service of querying the metadata in each metadata database table, a service of adding, modifying, deleting, etc. the metadata in each metadata database table, a metadata database table maintenance person submitting an authentication request for operating the metadata in the metadata database table to a central management device, a central administrator receiving or rejecting the authentication request through the central management device, updating the version number of the metadata, etc. It should be noted that the logical layer cannot interact directly with the metadatabase tables.
The persistence layer encapsulates all operations of the metadata base table, including queries, additions, modifications, deletions, and the like. The persistence layer is responsible for interacting with the metadata base tables, i.e., reading the metadata in the database tables. The persistent layer maintains an entity manager 360, and implements operations such as reading and writing of the metadata database table by manipulating different entities 331. One for each entity, metadata base table 332. A row of mapped metadata in the metadata base table corresponding to each entity is an object. The persistent layer realizes the operation on the row of the corresponding metadata in the metadata base table by manipulating the object and the attribute of the object.
In order to ensure data security, presentation layers corresponding to different metadata database tables cannot be accessed mutually, and persistent layers corresponding to different metadata database tables cannot be accessed mutually. If a business logic involves multiple metadata base tables, the required operation modules for the metadata base tables of the persistence tier can be called at the logic tier to manipulate the different metadata base tables.
The purpose of the software architecture of the central node server is as follows: the robustness and code reusability of the metadata synchronization system are improved. Each layer plays its own role, and when a problem occurs in a certain layer, only the code of the layer needs to be changed, and the codes of other layers do not need to be changed.
Specifically, before the metadata in the metadata database table in the central node server is updated from the first stable version number to the second stable version number, the method further includes:
and after the first metadata in the metadata database table in the central node server is updated into the second metadata, the central node server updates the first temporary version number of the first metadata into a second temporary version number.
It should be noted that each piece of metadata in the metadata library table corresponds to a stable version number and a temporary version number. When the maintainer operates on the first metadata in the metadata base table in the central node server, that is, the maintainer updates the first metadata in the metadata base table in the central node server to the second metadata, the temporary version number of the first metadata is updated from the first temporary version number to the second temporary version number, for example, after the first metadata is updated to the second metadata, the temporary version number of the first metadata is updated from "0" to "1", and after the first metadata is updated to the second metadata, the maintainer updates the second metadata to the third metadata again, the temporary version number of the second metadata is updated from the second temporary version number to the third temporary version number, that is, the temporary version number of the second metadata is updated from "1" to "2". It should be noted that the first metadata is any metadata in the metadata base table.
Wherein the first metadata is: metadata having a first stable version number, the second temporary version number being higher than the first temporary version number.
Specifically, after the central node server updates the first temporary version number of the first metadata to the second temporary version number, the method further includes:
the central node server sends an authentication request to the central management equipment;
it can be understood that, when the first metadata is updated to the second metadata, the first temporary version number of the first metadata is updated to the second temporary version number, the first stable version number of the first metadata remains unchanged, and the central node server sends the authentication request to the central node management device.
It should be noted that, before the central node server sends the authentication request to the central management device, the number of times of the maintenance personnel operating on the first metadata is the number of times of updating the temporary version number of the first metadata, for example, if the maintenance personnel operating on the first metadata 5 times, the temporary version number of the first metadata is updated to "5". Typically, the temporary version number of the metadata in the metadata base table is "0" before the maintenance person operates the first metadata.
Specifically, in an implementation manner of the embodiment of the present invention, updating metadata in a metadata database table in the central node server from a first stable version number to a second stable version number includes:
when the central management equipment receives the authentication request, the central management equipment sends a first message to the central node server, wherein the first message is a message for receiving the first metadata and updating the first metadata into the second metadata;
and the central node server updates the metadata in the metadata database table from the first stable version number to the second stable version number according to the first message.
It can be understood that, after the maintenance personnel operates the metadata, that is, after the first metadata is updated to the second metadata, the central node server sends an authentication request to the central node management device, and at the same time, the central node management device returns a message, that is, the first message, to the central node server. Since the first message is a message for receiving the update of the first metadata to the second metadata, the central node server updates the first stable version number of the first metadata to the second stable version number. When the first stable version number of the first metadata is updated to the second stable version number, the first metadata is updated to the second metadata in the true sense.
Specifically, in another implementation manner of the embodiment of the present invention, before the metadata in the metadata database table in the central node server is updated from the first stable version number to the second stable version number, the method further includes:
when the central management equipment receives the authentication request, the central management equipment sends a second message to the central node server, wherein the second message is a message for refusing the first metadata to be updated into the second metadata;
and the central management equipment maintains a first stable version number of the metadata in the metadata table according to the second message, and updates the second metadata into the first metadata.
After the maintenance personnel operate the metadata, the central node server sends an authentication request to the central node management equipment, meanwhile, the central node management equipment returns a message, namely a second message, to the central node server, and as the second message is a message for refusing the first metadata to be updated into the second metadata, the stable version number of the first metadata is not changed, namely the first stable version number of the first metadata is kept. Meanwhile, the central node server updates the second metadata to the first metadata.
For example, as shown in fig. 4, a metadata synchronization method in a massive network data environment according to an embodiment of the present invention includes:
step 1, a central node server sends an authentication request to a central management device;
after a maintenance person operates the metadata in the metadata database table, the central node server sends an authentication request to the central node management device according to the operation of the metadata, wherein when one piece of metadata in the metadata database table is operated, the central node server sends the authentication request to the central node management device according to the operated metadata, or the central node server sends the authentication requests of a plurality of pieces of metadata operated in a preset period to the central management device, so that the central management device processes the authentication requests of the plurality of pieces of metadata operated.
Step 2, the central management equipment receives the authentication request;
specifically, after receiving an authentication request sent by the central node server, the central management device waits for a central administrator to log in the central management device, and processes the authentication request.
Step 3, the central management equipment sends a first message to the central node server;
wherein the first message is a message for receiving the first metadata and updating the first metadata into the second metadata;
the central administrator processes the authentication request and sends a first message to the central node server, that is, the central administrator allows the maintenance personnel to update the first metadata to the second metadata, and the first stable version number of the first metadata is updated to the second stable version number.
Step 4, the central management equipment sends a second message to the central node server;
wherein the second message is a message rejecting the update of the first metadata to the second metadata.
The central administrator processes the authentication request and sends a second message to the central node server, that is, the central administrator does not allow the maintenance personnel to update the first metadata into the second metadata, the second metadata is reset into the first metadata, and the first stable version number of the first metadata is maintained.
Specifically, in an implementation manner of the embodiment of the present invention, after the updating the metadata in the metadata database table from the first stable version number to the second stable version number, the method further includes:
the central node server sets the first metadata from the second temporary version number to an initial temporary version number, wherein the initial temporary version number is a temporary version number of the first metadata before the first metadata is updated, and the value of the initial temporary version number is a preset value;
after the central node server maintains a first stable version number of metadata within the metadata database table, the method further comprises: and the central node server updates the first metadata from the second temporary version number to the initial temporary version number.
In general, the initial temporary version number of each piece of metadata in the metadata library table is a preset value that is set in advance for the temporary version number of each piece of metadata, and the preset value may be "0", for example. When a maintainer operates on the metadata, the initial temporary version number of the metadata starts to be updated, and after the stable version number of the metadata is updated from a first stable version number to a second stable version number, the temporary version number of the metadata is set to be the initial temporary version number, for example, when the stable version number of the metadata is updated from the first stable version number to the second stable version number, the temporary version number of the metadata is "4", the temporary version number "4" of the metadata is updated to be "0", that is, after the stable version number of the metadata is updated, the temporary version number of the metadata is updated to be a preset value, for example, the preset value is "0". Meanwhile, after the central node server maintains the first stable version number of the metadata within the metadata repository table, the temporary version number of the metadata is set to the initial temporary version number. For example, when the central node server maintains a stable version number of a piece of metadata as "5" and a temporary version number of the piece of metadata is "4", the temporary version number of the piece of metadata is updated from "4" to "0", and at this time, the stable version number of the piece of metadata is "5" and the temporary version number of the piece of metadata is "0".
S202, after receiving the notification message, the child node management device acquires the metadata database table from the central node server, wherein the version number of metadata in the metadata database table is the second stable version number;
when the stable version of one piece of metadata in the metadata database table changes, a notification message is sent to the child node management device, wherein the notification message is as follows: and if the stable version number of the metadata in the metadata database table in the central node server has a message of updating, the child node management equipment acquires the metadata database table with the updated version number from the central node server according to the notification message.
It should be noted that, in the central node server, the metadata base table including the metadata of different stable versions is stored, that is, the central node server stores both the metadata base table including the metadata before updating the stable version and the metadata base table including the metadata of the updated stable version. In this way, the user can query the metadata base table where all the stable versions of metadata are located on the Web IU interface.
S203, the child node management device sends the obtained metadata database table to the child node server, so that metadata in the metadata database tables of the central node server and the child node server are synchronized.
And the child node management equipment sends the acquired metadata database table to the child node server in time, so that the stable version numbers of the metadata in the metadata database tables in the central node server and the child node server are consistent, and the contents of the metadata in the metadata database tables in the central node server and the child node server are the same.
In the embodiment of the invention, when the stable version number of the metadata in the metadata database table in the central node server is updated, the metadata database table where the metadata with the stable version number is updated is sent to the child node server in time through the child node management equipment, so that the timeliness of the synchronization of the metadata in the metadata database table in the central node server and the metadata in the child node server is improved.
In an implementation manner of the embodiment of the present invention, after the first metadata in the metadata database table in the central node server is updated to the second metadata, the method further includes:
the central node server records the updating information of the first metadata updated into the second metadata in a stored second log information base table; the update information includes: time updating and IP address updating.
Wherein the update information further comprises: the user name of a maintainer operating the metadata database table and the unique identity ID of a central administrator receiving the authentication request.
Specifically, the second log information base table is configured to record update information for updating the first metadata into the second metadata, where the update information includes: update time, update IP (Internet Protocol Address) Address. The update time includes: the time when the maintenance person operates the first metadata, for example, the time when the maintenance person adds metadata in the metadata database table, the time when the maintenance person modifies the first metadata into the second metadata, the time when the metadata is deleted, and the like. Updating the IP address may include: and the maintenance personnel operate the IP address of the terminal equipment used when the metadata in the database table. Each maintainer who has the authority to operate the metadata base table has a fixed user name, when the maintainer operates the metadata base table, the user name of the maintainer is recorded in the second log information base table, and meanwhile, no matter whether the center manager audits the authentication request of the maintainer operating the metadata base table, the ID of the center manager is recorded in the second log information base table.
Wherein the update information may further include: the time when the central management apparatus receives the first message that the first metadata is updated to the second metadata or rejects the second message that the first metadata is updated to the second metadata is not limited thereto.
As shown in fig. 5, a metadata synchronization method in a massive network data environment according to an embodiment of the present invention includes the following steps:
s501, when detecting that a user accesses the child node server through a terminal device, the child node server records operation information of the user for operating the terminal device in a stored first log information table; the operation information includes: login time, logout time, and login IP address.
It can be understood that, when each user accesses the child node server by using the terminal device, the child node server detects that there is a user accessing the child node server, and then the child node server records the time when each user logs in the terminal device, the time when the user logs out of the terminal device, and the IP address of the terminal device used by the user in the first log information table.
It should be noted that the terminal device may include: the electronic device such as a notebook computer, a desktop computer, a tablet computer, etc. may access the child node server, but is not limited thereto. The user views the metadata stored in the metadata database table in the child node server by accessing the child node server.
S502, the central management equipment acquires the first log information table in the child node server;
the central node management device can acquire the first log information table from the child node server according to a preset period.
And S503, the central management equipment sends the first log information table to the central node server so as to synchronize the operation information of the first log information tables of the central node server and the child node servers.
The central management equipment sends the first log information base table acquired according to the preset period to the central node server, so that the first log information base table is stored in the child node server and also stored in the central node server, and even if the child node server loses the first log information base table due to accidents (such as fire, earthquake and the like), the central node server also stores the first log information base table, so that the first log information base table is ensured not to be lost or damaged.
As shown in fig. 6, a metadata synchronization method in a massive network data environment according to an embodiment of the present invention includes the following steps:
s601, the child node management equipment distributes a user name and a password to the terminal equipment so that the user can access the child node server through the terminal equipment according to the user name and the password;
it can be understood that the child node management device assigns a user name and a password to the user terminal device before the user accesses the child node server through the terminal device. Therefore, after the terminal device is allocated with the corresponding user name and the corresponding password, the user accesses the child node server according to the user name and the password.
S602, when a child node server receives an access request of a user for accessing the child node server through the terminal equipment, matching the user name and the password in a user account information table in the child node server; when the user name is matched with the user name in the user account information base table and the password is matched with the password in the user account information base table, the child node server receives the access request;
the access request carries the user name and the password;
generally, a user account name and a password are stored in a user account information base table, the password is an encrypted password, when a child node server receives an access request of any user accessing the child node server through a terminal device used by the user, whether a user name and the password carried by the access request are matched with the account name and the password stored in the user account information base table is judged, if yes, the child node server receives the access request, and if not, the child node server rejects the access request. The safety of the user accessing the child node server is guaranteed.
S603, the central management equipment acquires the user account information base table in the child node server and sends the user account information base table to the central node server so as to synchronize the user account information base tables of the central node server and the child node server.
Therefore, the user account information base table is stored in the central node server and the child node servers, the method ensures the safety of the account information base table and avoids the loss of the account information base table.
As shown in fig. 7, a metadata synchronization system in a massive network data environment according to an embodiment of the present invention, where the metadata synchronization system 700 includes: a center node server 710, a child node management apparatus 720, and a child node server 730;
wherein the content of the first and second substances,
the central node server 710 is configured to update the temporary version number and the stable version number of the metadata in the metadata database table; and a notification message is sent to the child node management device; and updating the second log information base table;
the child node management device 720 is configured to receive a notification message sent by the central node server, and obtain the metadata database table from the central node server according to the notification message; the metadata database table is used for importing the metadata database table to the child node server; the system comprises a child node server, a central node server and a sub-node server, wherein the child node server is used for acquiring a first log information base table and a user account information base table from the child node server and importing the first log information base table and the user account information base table into the central node server;
the child node server 730 is configured to store the metadata database table; and the database table is used for storing and updating the first log information database table and the user account information database table.
Optionally, the system is further specifically configured to, after the first metadata in the metadata database table in the central node server is updated to the second metadata, update, by the central node server, the first temporary version number of the first metadata to the second temporary version number, where the first metadata is: metadata having a first stable version number, the second temporary version number being higher than the first temporary version number.
Optionally, the system is further specifically adapted to,
the central node server sends an authentication request to central management equipment;
updating the metadata in the metadata database table in the central node server from a first stable version number to a second stable version number, wherein the updating comprises the following steps:
when the central management equipment receives the authentication request, the central management equipment sends a first message to the central node server, wherein the first message is a message for receiving the first metadata and updating the first metadata into the second metadata;
and the central node server updates the metadata in the metadata database table from the first stable version number to the second stable version number according to the first message.
Optionally, the system is further specifically configured to:
when the central management equipment receives the authentication request, the central management equipment sends a second message to the central node server, wherein the second message is a message for refusing the first metadata to be updated into the second metadata;
and the central management equipment maintains a first stable version number of the metadata in the metadata table according to the second message, and updates the second metadata into the first metadata.
The system is further specifically configured to set, by the central node server, the first metadata from the second temporary version number to an initial temporary version number, where the initial temporary version number is a temporary version number of the first metadata before the first metadata is updated, and a value of the initial temporary version number is a preset value;
the system is further specifically configured to update, by the central node server, the first metadata from the second temporary version number to the initial temporary version number.
The system is specifically further configured to record, in a stored first log information table, operation information of the user operating the terminal device when the child node server detects that the user accesses the child node server through the terminal device; the operation information includes: login time, logout time, login IP.
The central management equipment acquires the first log information base table in the child node server;
and the central management equipment sends the first log information table to the central node server so as to synchronize the operation information of the first log information tables of the central node server and the child node servers.
The system is further specifically configured to allocate, by the child node management device, a user name and a password to the terminal device, so that the user accesses the child node server through the terminal device according to the user name and the password;
when the child node server receives an access request of a user for accessing the child node server through the terminal equipment, matching the user name and the password in a user account information table in the child node server; when the user name is matched with the user name in the user account information base table and the password is matched with the password in the user account information base table, the child node server receives the access request; the access request carries the user name and the password;
and the central management equipment acquires the user account information base table in the child node server and sends the user account information base table to the central node server so as to synchronize the user account information base tables of the central node server and the child node server.
The system is specifically further configured to record, by the central node server, update information of the first metadata updated to the second metadata in a stored second log information base table; the update information includes: time updating and IP address updating.
In the embodiment of the invention, when the stable version number of the metadata in the metadata database table in the central node server is updated, the metadata database table where the metadata with the stable version number is updated is sent to the child node server in time through the child node management equipment, so that the timeliness of the synchronization of the metadata in the metadata database table in the central node server and the metadata in the child node server is improved.
For the system embodiment, since it is basically similar to the method embodiment, the description is simple, and the relevant points can be referred to the partial description of the method embodiment.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (8)
1. A metadata synchronization method under a massive network data environment is characterized by being applied to a metadata synchronization system, wherein the metadata synchronization system comprises: the method comprises the following steps of:
after the metadata in the metadata database table in the central node server is updated from a first stable version number to a second stable version number, a notification message is sent to the child node management equipment; wherein the second stable version number is higher than the first stable version number;
after receiving the notification message, the child node management device acquires the metadata database table from the central node server, wherein the version number of metadata in the metadata database table is the second stable version number;
the child node management equipment sends the obtained metadata database table to a child node server so as to synchronize metadata in the metadata database tables of the central node server and the child node server;
the central node server sends an authentication request to central management equipment;
updating the metadata in the metadata database table in the central node server from a first stable version number to a second stable version number, wherein the updating comprises the following steps:
when the central management equipment receives the authentication request, the central management equipment sends a first message to the central node server, wherein the first message is a message for receiving first metadata and updating the first metadata into second metadata;
the central node server updates the metadata in the metadata database table from the first stable version number to the second stable version number according to the first message;
when the child node server detects that a user accesses the child node server through terminal equipment, recording operation information of the user for operating the terminal equipment in a stored first log information table; the operation information includes: login time, logout time, login IP address;
the central management equipment acquires the first log information base table in the child node server;
and the central management equipment sends the first log information table to the central node server so as to synchronize the operation information of the first log information tables of the central node server and the child node servers.
2. The method of claim 1, wherein before the metadata in the metadata repository table in the central node server is updated from a first stable version number to a second stable version number, the method further comprises:
after first metadata in a metadata database table in the central node server is updated to second metadata, the central node server updates a first temporary version number of the first metadata to a second temporary version number, wherein the first metadata is: metadata having a first stable version number, the second temporary version number being higher than the first temporary version number.
3. The method of claim 1, wherein before updating the metadata in the metadata repository table in the central node server from a first stable version number to a second stable version number, the method further comprises:
when the central management equipment receives the authentication request, the central management equipment sends a second message to the central node server, wherein the second message is a message for refusing the first metadata to be updated into the second metadata;
and the central management equipment maintains a first stable version number of the metadata in the metadata database table according to the second message, and updates the second metadata into the first metadata.
4. The method of claim 2, wherein after said updating the metadata in the metadata library table from the first stable version number to the second stable version number, the method further comprises:
the central node server sets the first metadata from the second temporary version number to an initial temporary version number, wherein the initial temporary version number is a temporary version number of the first metadata before the first metadata is updated, and the value of the initial temporary version number is a preset value;
after the central node server maintains a first stable version number of metadata within the metadata database table, the method further comprises:
and the central node server updates the first metadata from the second temporary version number to the initial temporary version number.
5. The method of claim 1, further comprising:
the child node management equipment allocates a user name and a password to the terminal equipment, so that the user can access the child node server through the terminal equipment according to the user name and the password;
when the child node server receives an access request of a user for accessing the child node server through the terminal equipment, matching the user name and the password in a user account information table in the child node server; when the user name is matched with the user name in the user account information base table and the password is matched with the password in the user account information base table, the child node server receives the access request; the access request carries the user name and the password;
and the central management equipment acquires the user account information base table in the child node server and sends the user account information base table to the central node server so as to synchronize the user account information base tables of the central node server and the child node server.
6. The method of claim 2, wherein after the first metadata in the metadata database table in the central node server is updated to the second metadata, the method further comprises:
the central node server records the updating information of the first metadata updated into the second metadata in a stored second log information base table; the update information includes: time updating and IP address updating.
7. A metadata synchronization system in a mass network data environment, the system comprising: the system comprises a central node server, child node management equipment and child node servers;
wherein the content of the first and second substances,
the central node server is used for updating the temporary version number and the stable version number of the metadata in the metadata database table; the method comprises the steps that a first stable version number of metadata in a metadata database table is updated to a second stable version number, and then a notification message is sent to child node management equipment; and updating the second log information base table;
the child node management device is configured to receive a notification message sent by the central node server, and obtain the metadata database table from the central node server according to the notification message; the metadata database table is imported to the child node server so as to synchronize metadata in the metadata database tables of the central node server and the child node server; the system comprises a child node server, a central node server and a sub-node server, wherein the child node server is used for acquiring a first log information base table and a user account information base table from the child node server and importing the first log information base table and the user account information base table into the central node server;
the child node server is used for storing the metadata database table; the first log information base table and the user account information base table are used for storing and updating; the first log information base table records operation information of a user operating terminal device; the operation information includes: login time, logout time, login IP address.
8. The system according to claim 7, characterized in that the system is in particular also adapted for,
after the first metadata in the metadata database table in the central node server is updated to the second metadata, the central node server updates the first temporary version number of the first metadata to a second temporary version number.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611155171.7A CN106599195B (en) | 2016-12-14 | 2016-12-14 | Metadata synchronization method and system under massive network data environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611155171.7A CN106599195B (en) | 2016-12-14 | 2016-12-14 | Metadata synchronization method and system under massive network data environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106599195A CN106599195A (en) | 2017-04-26 |
| CN106599195B true CN106599195B (en) | 2020-07-31 |
Family
ID=58801982
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611155171.7A Active CN106599195B (en) | 2016-12-14 | 2016-12-14 | Metadata synchronization method and system under massive network data environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106599195B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109684342A (en) * | 2018-12-21 | 2019-04-26 | 成都四方伟业软件股份有限公司 | Data managing method and device |
| CN111726375A (en) * | 2019-03-19 | 2020-09-29 | 北京数聚鑫云信息技术有限公司 | Data synchronization method and device and computer equipment |
| US10979300B2 (en) | 2019-06-20 | 2021-04-13 | Kaloom Inc. | Computing device and methods for synchronizing networking information with a topology server |
| US11075812B2 (en) * | 2019-06-20 | 2021-07-27 | Kaloom Inc. | Server and methods for synchronizing networking information with client devices |
| CN111147560B (en) * | 2019-12-18 | 2022-05-17 | 河北远东通信系统工程有限公司 | Data synchronization method based on HTTP (hyper text transport protocol) and breakpoint continuous transmission |
| CN111984730A (en) * | 2020-08-24 | 2020-11-24 | 中电九天智能科技有限公司 | Centralized data synchronization method for industrial internet |
| CN113472856A (en) * | 2021-06-08 | 2021-10-01 | 北京城市网邻信息技术有限公司 | Message pushing processing method and device, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1480859A (en) * | 2002-09-03 | 2004-03-10 | 鸿富锦精密工业（深圳）有限公司 | Synchronous system in distributed files and method |
| CN101076793A (en) * | 2004-08-31 | 2007-11-21 | 国际商业机器公司 | System structure for enterprise data integrated system |
| CN102289460A (en) * | 2011-07-13 | 2011-12-21 | 中国工商银行股份有限公司 | Method and system for performing report metadata synchronization on testing environments |
| CN103795754A (en) * | 2012-10-31 | 2014-05-14 | 中国电信股份有限公司 | Method and system for data synchronization among multiple systems |
| CN104348913A (en) * | 2014-10-28 | 2015-02-11 | 浪潮电子信息产业股份有限公司 | Tight-coupling extensible big data interaction method |
| CN104506625A (en) * | 2014-12-22 | 2015-04-08 | 国云科技股份有限公司 | Method for improving reliability of metadata nodes of cloud databases |
-
2016
- 2016-12-14 CN CN201611155171.7A patent/CN106599195B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1480859A (en) * | 2002-09-03 | 2004-03-10 | 鸿富锦精密工业（深圳）有限公司 | Synchronous system in distributed files and method |
| CN101076793A (en) * | 2004-08-31 | 2007-11-21 | 国际商业机器公司 | System structure for enterprise data integrated system |
| CN102289460A (en) * | 2011-07-13 | 2011-12-21 | 中国工商银行股份有限公司 | Method and system for performing report metadata synchronization on testing environments |
| CN103795754A (en) * | 2012-10-31 | 2014-05-14 | 中国电信股份有限公司 | Method and system for data synchronization among multiple systems |
| CN104348913A (en) * | 2014-10-28 | 2015-02-11 | 浪潮电子信息产业股份有限公司 | Tight-coupling extensible big data interaction method |
| CN104506625A (en) * | 2014-12-22 | 2015-04-08 | 国云科技股份有限公司 | Method for improving reliability of metadata nodes of cloud databases |
Non-Patent Citations (2)
| Title |
|---|
| "基于云存储服务的文件系统设计与实现";王俊龙;《中国优秀硕士学位论文全文数据库 信息科技辑》;20150415;第19-29页 * |
| "福建农业银行数据仓库元数据管理系统设计与实现";周微;《中国优秀硕士学位论文全文数据库 信息科技辑》;20160315;第12-15页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106599195A (en) | 2017-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106599195B (en) | Metadata synchronization method and system under massive network data environment | |
| US10747841B2 (en) | Systems and methods for modifying and restoring website content via a website directory | |
| Arnold | Openstack swift: Using, administering, and developing for swift object storage | |
| TWI473029B (en) | Extensible and programmable multi-tenant service architecture | |
| AU2015363218B2 (en) | No password user account access | |
| US6490624B1 (en) | Session management in a stateless network system | |
| US9053117B2 (en) | Storing application data with a unique ID | |
| AU2015289740B2 (en) | Unregistered user account generation for content item sharing | |
| US11948473B2 (en) | Assignments for classrooms | |
| US20050055382A1 (en) | Universal synchronization | |
| US9955444B1 (en) | Data synchronization across multiple devices connecting to multiple data centers | |
| TW200845657A (en) | Techniques for a web services data access layer | |
| CN109408523B (en) | Data sharing method and device, data sharing platform and electronic equipment | |
| JP2010129028A (en) | Gateway server, system for managing file, file management method, and program | |
| US7533157B2 (en) | Method for delegation of administrative operations in user enrollment tasks | |
| US10607498B2 (en) | Releasing assignments to students | |
| WO2007111751A2 (en) | Architecture for a smart enterprise framework and methods thereof | |
| US9495421B1 (en) | Bidirectional data portability | |
| US20140344267A1 (en) | Storing, Accessing and Restoring Website Content via a Website Repository | |
| US11675927B2 (en) | System and method for external users in groups of a multitenant system | |
| US20220318227A1 (en) | Content management system for a distributed key-value database | |
| US10078433B2 (en) | Sharing a template file | |
| KR20160052524A (en) | Multi work space system for supporting software as a service | |
| Kumar et al. | Modern Big Data processing with Hadoop: Expert techniques for architecting end-to-end Big Data solutions to get valuable insights | |
| US20230359611A1 (en) | Verifying data consistency using verifiers in a content management system for a distributed key-value database |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |