WO2017080362A1

WO2017080362A1 - Data managing method and device

Info

Publication number: WO2017080362A1
Application number: PCT/CN2016/103529
Authority: WO
Inventors: 张朋飞
Original assignee: 中兴通讯股份有限公司
Priority date: 2015-11-11
Filing date: 2016-10-27
Publication date: 2017-05-18
Also published as: CN106682040A

Abstract

An ENUM-DNS big data processing method. The method manages ENUM-DNS background data via a distributed cache system. The method mainly comprises: a method for ensuring consistency of ENUM-DNS foreground-background data. The method ensures consistency of the foreground-background data of an ENUM-DNS system by regularly detecting file identifiers in a first configuration file and a second configuration file, improving maintenance efficiency for data consistency in an ENUM-DNS server and the distributed cache system.

Description

Data management method and device

Technical field

The present application relates to the field of data management technologies in an IMS (IP Multimedia Subsystem) network, for example, to a data management method and apparatus.

Background technique

IMS is a new form of multimedia service that meets the needs of users for more innovative and diverse multimedia services. The ENUM-DNS (E.164 Number URI Mapping-Domain Name System) server provides the IMS with a phone number and domain name and a host resource correspondence query service. With the development of technology, IMS has higher and higher performance requirements for ENUM-DNS servers, such as ensuring data consistency between ENUM-DNS servers and distributed cache systems. In the related art, when maintaining data consistency between the ENUM-DNS server and the distributed cache system, the ENUM-DNS server is usually compared with the data in the distributed cache system, and the data is compared when the comparison result is different. Consistency maintenance. Because of the long time taken to compare data, it is less efficient to maintain data consistency between the ENUM-DNS server and the distributed cache system.

Summary of the invention

The embodiment of the invention provides a data management method and device, which aims to solve the technical problem of low maintenance efficiency of data consistency in the ENUM-DNS server and the distributed cache system.

The data management method is implemented by software, and the method includes the following steps:

Generating a data file for the received data every first preset duration, setting a file identifier corresponding to the data file, saving the data file, and writing the file identifier corresponding to the data file into the first configuration file;

Writing the data file to the distributed cache system, and writing the file identifier corresponding to the saved data file to the second configuration file when the writing is successful;

Detecting, differently, the file identifiers in the first configuration file and the second configuration file every second preset duration, and writing different data files corresponding to the file identifiers to the distributed cache In the system.

The file identifier may be a file number added to the data file in the order of receiving, and the step of writing the file identifier corresponding to the data file into the first configuration file may include:

Writing, by using the file identifier corresponding to the data file, the first configuration file to update the file identifier saved in the first configuration file;

The steps of writing the file identifier corresponding to the data file successfully written into the distributed cache system into the second configuration file include:

Writing, by using a file identifier corresponding to the data file successfully written in the distributed cache system, to the second configuration file, to update the file identifier saved in the second configuration file; wherein when a data file is not successfully written , repeatedly write the data file until the write is successful, or report the alarm when the write fails;

The step of detecting the file identifiers in the first configuration file and the second configuration file, and writing the data files corresponding to the different file identifiers to the distributed cache system includes:

Detecting whether the file identifier saved in the first configuration file is consistent with the file identifier saved in the second configuration file;

And deleting, when the file identifier saved in the first configuration file is different from the file identifier saved in the second configuration file, the file identifier saved in the first configuration file and the file saved in the second configuration file A data file corresponding to the file identifier of the difference between the identifiers is written into the distributed cache system.

While performing the step of generating the data file by using the first preset duration, the received data, and setting the file identifier corresponding to the data file, the following steps may be performed:

Sending a detection message to a node corresponding to each preset space in the distributed cache system every third preset duration, and detecting whether a response message fed back by the node corresponding to each preset space is received;

When the response message fed back by the node corresponding to the preset space is not received, determining that the node corresponding to the preset space is an abnormal node;

The number of the abnormal nodes in the distributed cache system is counted. If the number of the abnormal nodes is greater than the preset number threshold, the alarm information is sent to the operation and maintenance system, and the function of receiving data is closed, and the first preset is separated. The duration no longer generates new data files until the system returns to normal.

When the response message of the node corresponding to the preset space is not received, the step of determining that the node corresponding to the preset space is an abnormal node may include:

When the response message fed back by the node corresponding to a certain preset space is not received, the update does not receive the received message. The number of times the response message is reported;

Determining whether the number of times is greater than or equal to a preset number of times, wherein the node corresponding to the preset space is an abnormal node when the number of times the response message is not received is greater than or equal to a preset number of times.

The data management method may further include:

When the data file of the node in the distributed cache system is faulty, the data file corresponding to the file directory is sequentially copied from the standby node to the node according to the file directory of the node.

In addition, an embodiment of the present invention further provides a data management apparatus, where the data management apparatus includes:

Generating a module, configured to generate a data file for the received data every other first preset duration, set a file identifier corresponding to the data file, save the data file, and write the file identifier corresponding to the data file In the first configuration file;

a processing module, configured to write the data file into the distributed cache system, and write the file identifier corresponding to the saved data file to the second configuration file when the writing is successful;

The storage module is configured to detect that the first configuration file and the data file are saved to the distributed cache system every second preset duration.

The file identifier may be a file number added to the data file in the order of receiving, and the generating module may be further configured to write the file identifier corresponding to the generated data file into the first configuration file to update a file identifier saved in the first configuration file;

The processing module is further configured to write the file identifier corresponding to the successfully written data file into the second configuration file to update the file identifier saved in the second configuration file; wherein when a certain data file is not saved On success, the data file is continuously written repeatedly until the write is successful, or the alarm is reported when the write fails;

The storage module is further configured to: detect whether the file identifier saved in the first configuration file is consistent with the file identifier saved in the second configuration file; and the file identifier and the file saved in the first configuration file When the file identifiers saved in the second configuration file are inconsistent, the data files corresponding to the file identifiers of the differences between the file identifiers saved in the first configuration file and the file identifiers saved in the second configuration file are written to In the distributed cache system.

The data management device may further include:

The sending module is configured to send a detection message to a node corresponding to each preset space in the distributed cache system every third preset duration, and detect whether a response of the node corresponding to each preset space is received. Message

The determining module is configured to receive a response message of the node corresponding to a certain preset space Determining, that the node corresponding to the preset space is an abnormal node;

The alarm module is configured to count the number of abnormal nodes in the distributed cache system. If the number of the abnormal nodes is greater than a preset threshold, the alarm information is sent to the operation and maintenance system, and the function of receiving data is closed. The new data file is no longer generated every first preset interval until the system returns to normal.

The determining module can be configured to:

Updating the number of times the response message is not received when the response message fed back by the node corresponding to the preset space is not received;

The data management device may further include:

The data recovery module is configured to, when the data file of the node in the distributed cache system is faulty, sequentially copy the data file corresponding to the file directory from the standby node to the node according to the file directory of the node.

The embodiment of the invention further provides a non-transitory computer readable storage medium storing computer executable instructions, the computer executable instructions being arranged to perform the above method.

An embodiment of the present invention further provides an electronic device, including:

At least one processor;

a memory communicatively coupled to the at least one processor; wherein

The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform the method described above.

A data management method and apparatus according to an embodiment of the present invention, in the data management method, periodically generating a data file from the received data, and writing a file identifier corresponding to the data file into the first configuration file, and successfully writing the data file When entering the distributed cache system, the file identifier corresponding to the successfully written data file is written into the second configuration file, and the file identifiers of the first configuration file and the second configuration file are compared, and By writing data files corresponding to different file identifiers to the distributed cache system, data synchronization can be realized without sequentially comparing the data files, thereby improving the consistency of data maintenance between the ENUM-DNS server and the distributed cache system. effectiveness.

BRIEF abstract

1 is a schematic flowchart of a first embodiment of a data management method according to the present invention;

2 is a schematic flowchart of a third embodiment of a data management method according to the present invention;

3 is a schematic diagram of functional modules of a first embodiment of a data management apparatus according to the present invention;

4 is a schematic diagram of functional modules of a third embodiment of a data management apparatus according to the present invention;

FIG. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Embodiments of the invention

It is understood that the embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The embodiment of the present invention provides a data management method. Referring to FIG. 1, FIG. 1 is a schematic flowchart of a first embodiment of a data management method according to the present invention.

In this embodiment, the data management method includes the following steps:

Step S10: Generate a data file for the received data every first preset duration, set a file identifier corresponding to the data file, save the data file, and write the file identifier corresponding to the data file into the first configuration file. in;

The O&M system or the SPN (Service Provisioning Node) delivers the data of the overhead household to the ENUM-DNS (E.164Number URI Mapping-Domain Name System) server. The ENUM-DNS server is timed. The data to be received is generated into a data file. In this embodiment, the first preset duration is set in advance, and the ENUM-DNS server generates a data file for the data received within the first preset duration every time the first preset duration is separated. And set the file identifier corresponding to the generated file. For example, if the first preset duration is 2 minutes, the ENUM-DNS server generates a data file for the data received within the 2 minute period every 2 minutes, and sets a file identifier corresponding to the generated data file. For example, the file identifier corresponding to the generated data file is set in order from small to large. When the first data file is generated, the file identifier corresponding to the generated first data file is set to 1; when the second data file is generated. The file identifier corresponding to the generated second data file is set to 2; when the third data file is generated, the file identifier corresponding to the generated third data file is set to 3 or the like. It can be understood by those skilled in the art that the first preset duration can be flexibly set according to actual conditions, and the file identifier corresponding to each data file generated can also be set according to other rules, and is not limited herein. At the same time, each time the data file is generated, the file identifier corresponding to the generated data file is written into the first configuration file.

Step S20, the data file is written into the distributed cache system, and when the writing is successful, the file identifier corresponding to the data file successfully written is written into the second configuration file;

The ENUM-DNS server writes the generated data file to the distributed cache system each time the data file is generated and the file identifier corresponding to the data file is written into the first configuration file. And each time the data file is successfully saved, the file identifier corresponding to the successfully saved data file is written into the second configuration file.

Step S30, detecting, according to the second preset duration, the file identifiers different from the first configuration file and the second configuration file, and writing different data files corresponding to the file identifiers to the Distributed cache system.

The ENUM-DNS server periodically compares the file identifiers written in the first configuration file and the second configuration file, and detects whether different file identifiers exist in the first configuration file and the second configuration file. In this embodiment, the second preset duration is also set in advance. The ENUM-DNS server detects whether there is a different file identifier in the first configuration file and the second configuration file when the second preset duration is separated, when the first configuration file is detected and the When different file identifiers exist in the second configuration file, the data files corresponding to the different file identifiers are written into the distributed cache system. For example, the file identifier in the first configuration file is file identifier 1 to file identifier 10, and the file identifier in the second configuration file is file identifier 1 to file identifier 5, and the first configuration file is The different file identifiers in the second configuration file are file identifier 6 to file identifier 10, that is, the data files corresponding to the file identifier 6 to the file identifier 10 are not successfully saved to the distributed cache system, and the ENUM-DNS server will The data files corresponding to the file identifier 6 to the file identifier 10 are written into the distributed cache system.

In the solution provided by the embodiment, the ENUM-DNS server generates a data file by periodically receiving the data, and each time the data file is generated, setting a file identifier corresponding to the data file, and writing the file identifier to the first configuration file. And then, the generated data file is written into the distributed cache system, and when the writing is successful, the file identifier corresponding to the data file successfully written is written into the second configuration file, ENUM- The DNS server periodically detects whether there is a different file identifier in the first configuration file and the second configuration file. If there is a different file identifier in the first configuration file and the second configuration file, the ENUM-DNS server is configured. Inconsistent with the data in the distributed cache system. At this time, the ENUM-DNS server saves the files corresponding to different file identifiers to the distributed cache system, because the data in the ENUM-DNS server and the distributed cache system are not required. Comparison of files greatly reduces the detection of data in the ENUM-DNS server and the distributed cache system. Time, which improves the efficiency of maintaining data consistency between the ENUM-DNS server and the distributed cache system, and improves the reliability of the ENUM-DNS server.

The second embodiment of the data management method of the present invention is proposed based on the first embodiment. In this embodiment, the step S10 includes: writing, by using the file identifier corresponding to the data file, the first configuration file to update the The file identifier saved in the first configuration file;

The step S20 includes: writing a file identifier corresponding to the saved data file to the second configuration file to update the file identifier saved in the second configuration file; wherein when a data file is not successfully saved , repeatedly write the data file until the write is successful, or report the alarm when the write fails;

The step S30 includes: detecting whether the file identifier saved in the first configuration file is consistent with the file identifier saved in the second configuration file; and the file identifier saved in the first configuration file and the second When the file identifiers saved in the configuration file are inconsistent, the data file corresponding to the file identifier of the difference between the file identifier saved in the first configuration file and the file identifier saved in the second configuration file is written to the distribution. In the cache system.

In this embodiment, the file identifier is a file number added to the data file according to the receiving order, and the ENUM-DNS server writes the file identifier saved in the first configuration file by using the file identifier corresponding to the generated data file. To update the file identifier saved in the first configuration file. For example, when the ENUM-DNS server generates the data file for the first time, the file identifier corresponding to the first data file generated is set to 1, and the file identifier 1 corresponding to the generated data file is written into the first configuration file and saved. When the ENUM-DNS server generates the data file for the second time, set the file identifier corresponding to the generated second data file to 2, write the file identifier 2 into the first configuration file, and replace the saved file identifier 1 with Update the file identifier saved in the first configuration file. When the ENUM-DNS server generates the data file for the third time, the file identifier corresponding to the third data file generated is set to 3, and the file identifier 3 is written into the first configuration file, and the saved file identifier 2 is replaced. In response to this, each time the ENUM-DNS server generates a file, the file identifier saved in the first configuration file is updated.

When the ENUM-DNS server successfully writes the first data file generated to the distributed cache system, the file identifier 1 corresponding to the first data file successfully written is written into the second configuration file and saved. When the ENUM-DNS server successfully saves the generated second data file to the distributed cache system, writes the file identifier 2 corresponding to the successfully saved second data file to the second configuration file, and replaces the saved file. File ID 1 to update the file ID saved in the second configuration file. The ENUM-DNS server successfully writes the generated third data file to the distributed cache system. When the file identifier 3 corresponding to the third data file successfully written is written into the second configuration file, the saved file identifier 2 is replaced. According to this, when the ENUM-DNS server successfully saves the data file to the distributed cache system, the file identifier saved in the second configuration file is updated. When a data file is not saved successfully, other generated data files that have not been written are not written, and the data file is continuously written repeatedly until the writing is successful, or an alarm is reported when the writing fails.

The ENUM-DNS server periodically detects whether the file identifier saved in the first configuration file is consistent with the file identifier saved in the second configuration file. And deleting, when the file identifier saved in the first configuration file is different from the file identifier saved in the second configuration file, the file identifier saved in the second configuration file and the file saved in the first configuration file A data file corresponding to the file identifier of the difference between the identifiers is written into the distributed cache system. For example, if the file identifier saved in the first configuration file is 10 and the file identifier saved in the second configuration file is 5, the file identifier saved in the first configuration file and the second configuration file are The file identifiers saved in the file are inconsistent, and the ENUM-DNS server compares the file identifiers corresponding to the file identifiers of the file identifiers 5 saved in the second configuration file with the file identifiers 10 stored in the first configuration file, that is, The five data files corresponding to the file identifier 6 to the file identifier 10 are saved in the distributed cache system.

In the solution provided by the embodiment, the ENUM-DNS server updates the file identifier saved in the first configuration file according to the file identifier corresponding to the data file each time the data file is generated, and successfully writes the generated data file. When the data is added to the distributed cache system, the file identifier saved in the second configuration file is updated according to the file identifier corresponding to the data file, and the ENUM-DNS server periodically detects the first configuration file and the second configuration file. If the file identifiers are consistent, if the first configuration file is inconsistent with the file identifier saved in the second configuration file, the data in the ENUM-DNS server is different from the distributed cache system. In this case, the ENUM-DNS server Saving a data file corresponding to the file identifier of the difference between the file identifier saved in the second configuration file and the file identifier saved in the first configuration file to the distributed cache system, so The first configuration file and the file identifier saved in the second configuration file reduce the detection of the ENUM-DNS server and the distributed Whether the data in the system is consistent or not, thereby improving the reliability of the ENUM-DNS server.

As shown in FIG. 2, a third embodiment of the data management method of the present invention is proposed based on the first embodiment or the second embodiment. In this embodiment, the data management method further includes the following steps:

Step S40: Send a detection message to a node corresponding to each preset space in the distributed cache system every third preset duration, and detect whether a response of the node corresponding to each preset space is received. Message

In step S50, when the response message fed back by the node corresponding to the preset space is not received, it is determined that the node corresponding to the preset space is an abnormal node;

Step S60: The number of abnormal nodes in the distributed cache system is counted. If the number of the abnormal nodes is greater than a preset number threshold, the alarm information is sent to the operation and maintenance system, and the function of receiving data is closed. A new data file is no longer generated for a preset duration until the system returns to normal.

In this embodiment, a preset space corresponding to each node in the distributed cache system is preset, and a special space is set, for example, the special space is set to space 0, and the space 0 is used for After the ENUM-DNS server generates the file of the data of the overhead user, the file generated by the ENUM-DNS server is saved, and the saved file is evenly loaded to the nodes corresponding to each preset space, thereby ensuring balanced storage of each node and Query data. The preset space corresponds to the nodes one by one, that is, each node is completely loaded with one query space.

When the ENUM-DNS server is initialized, a fixed record is written to each preset space, and the fixed record is a fixed record corresponding to the ENUM-DNS server initialization, and the form is not limited herein. When the write record is repeatedly written, as long as the key value of the record repeatedly written is the same as the key value of the record previously written, the record repeatedly written overwrites the previously written record. When the write fails, the ENUM-DNS server is not allowed to start, and the ENUM-DNS server fails to initialize; when the write is successful, the ENUM-DNS server is successfully initialized. After the ENUM-DNS server is successfully initialized, the ENUM-DNS server periodically sends a detection message to each node corresponding to the preset space for each third preset duration. For example, the ENUM-DNS server provides each preset space every 5 minutes. The corresponding node sends a detection message. If the node corresponding to the preset space is a normal node, the node corresponding to the preset space feeds back the response message to the ENUM-DNS server when receiving the detection message; if the node corresponding to the preset space is an abnormal node, the node cannot be fed back. Response message to the ENUM-DNS server.

After sending the detection message to the node corresponding to each preset space, the ENUM-DNS server detects whether the response message fed back by the node corresponding to each preset space is received, and if the response of the node corresponding to the preset space is not received, The message is determined to be an abnormal node; if the response message fed back by the node corresponding to the preset space is received, the node corresponding to the preset space is determined to be a normal node.

After that, the ENUM-DNS server counts the number of abnormal nodes in the distributed cache system. When the number of the abnormal nodes is greater than the preset threshold A, the data and the distributed data in the ENUM-DNS server cannot be guaranteed. ENUM-DNS server to operation and maintenance when the data in the cache system is consistent The system sends an alarm message. When the number of the abnormal nodes is less than or equal to the preset number threshold A, although there is theoretical possibility that the data in the ENUM-DNS server is inconsistent with the data in the distributed cache system, in order to balance the availability of data It can be considered that the data in the ENUM-DNS server can still be guaranteed to be consistent with the data in the distributed cache system. The value of the preset number threshold A is set according to the NRW copy policy adopted by the distributed cache system, for example, if the distributed cache system adopts a 3-2-2 copy policy, the preset number threshold The value of A is set to 3. When there are more than three nodes in the distributed cache system as abnormal nodes, the consistency between the data in the ENUM-DNS server and the data in the distributed cache system cannot be guaranteed. At this time, the ENUM-DNS server sends an alarm to the operation and maintenance system, and does not allow the operation and maintenance system or the SPN to continue to send the data of the overhead household to the ENUM-DNS server to ensure the data in the ENUM-DNS server and the distributed cache system. Data consistency. That is, the function of receiving data sent by the operation and maintenance system or the SPN is closed, and a new data file is no longer generated every interval of the first preset duration until the system returns to normal.

In this embodiment, the step S50 includes:

In step a, when the response message fed back by the node corresponding to the preset space is not received, the number of times the response message is not received is updated;

In step b, it is determined whether the number of times is greater than or equal to a preset number of times, wherein the node corresponding to the preset space is an abnormal node when the number of times the response message is not received is greater than or equal to a preset number of times.

In order to improve the accuracy of determining whether the node corresponding to each preset space in the distributed cache system is an abnormal node, in this embodiment, the ENUM-DNS server sends a detection message to each node corresponding to the preset space each time. And if the response message fed back by the node corresponding to the preset space is not received, updating the number of times the response message fed back by the node corresponding to the preset space is not received. The ENUM-DNS server determines whether the number of times is greater than or equal to a preset number of times. When the number of response messages fed back by the node corresponding to the preset space is not greater than or equal to a preset number of times, determining that the node corresponding to the preset space is Anomalous node. For example, if the preset number of times is set to 3 times, when the ENUM-DNS server sends the detection message 3 times to the node corresponding to each preset space, if the ENUM-DNS server does not receive a preset space 3 times. The response message fed back by the corresponding node determines that the node corresponding to the preset space is an abnormal node.

In the solution provided by the embodiment, the ENUM-DNS server periodically sends a detection message to the node corresponding to each preset space in the distributed cache system, and detects whether a response message fed back by the node corresponding to each preset space is received. When the response message fed back by the node corresponding to the preset space is not received, it is determined that the node corresponding to the preset space is an abnormal node, and the ENUM-DNS server collects the distributed information. The number of abnormal nodes in the cache system. When the number of abnormal nodes is greater than the preset threshold, that is, when the data in the ENUM-DNS server cannot be guaranteed to be consistent with the data in the distributed cache system, the operation and maintenance system is Sending alarm information, therefore, achieves consistency and availability of data, and improves the reliability of the ENUM-DNS server.

A fourth embodiment of the data management method of the present invention is provided based on any of the foregoing embodiments. In this embodiment, the data management method further includes:

In step c, when the data file of the node in the distributed cache system is faulty, the data file corresponding to the file directory is sequentially copied from the standby node to the node according to the file directory of the node.

Due to the complexity of the IMS network environment, the distributed cache system may have catastrophic failures, resulting in the loss of data of nodes corresponding to some preset spaces or unreliable data. At this point, you need to recover the data of these abnormal nodes with data failures. In this embodiment, each of the distributed cache systems has a backup system. Normally, the data stored in the standby node of the standby system is consistent with the data held by the distributed cache system. When the data of the abnormal node in the distributed cache system is restored, the data file corresponding to the file directory is sequentially copied to the abnormal node in the standby node of the standby system according to the file directory of the abnormal node, thereby The recovery of the data of the abnormal node is completed. If the file directory of the abnormal node is also damaged, that is, the data file cannot be copied from the standby node through the file directory, the preset space corresponding to each node in the distributed cache system is reset, and the ENUM-DNS server is re-initialized. After the initialization is completed, the data saved in the standby node of the backup system is sequentially copied to the node corresponding to each preset space in the distributed cache system.

In this embodiment, when the data of the abnormal node is restored, the data file corresponding to the file directory is sequentially copied from the standby node to the abnormal node according to the file directory of the abnormal node, and the The recovery of the data of the abnormal node without copying all the data files, thus realizing the rapid recovery of the data, thereby improving the reliability of the ENUM-DNS server.

The embodiment of the present invention further provides a data management apparatus. As shown in FIG. 3, FIG. 3 is a schematic diagram of functional modules of a first embodiment of a data management apparatus according to the present invention.

In this embodiment, the data management device includes:

The generating module 10 is configured to generate a data file for the received data every other first preset duration, set a file identifier corresponding to the data file, save the data file, and write the file identifier corresponding to the data file. Into the first configuration file;

The operation and maintenance system or the SPN (Service Provisioning Node) delivers the data of the overhead household to the ENUM-DNS (E.164 Number URI Mapping-Domain Name System) server, and the generation module 10 receives the data. After the data is received, the received data is periodically generated to generate a data file. In this embodiment, the first preset duration is set in advance, and the generating module 10 generates a data file for the data received within the first preset duration every time the first preset duration is separated, and Set the file identifier corresponding to the generated data file. For example, if the first preset duration is 2 minutes, the generating module 10 generates a data file for the data received within the 2 minute period every 2 minutes, and sets a file identifier corresponding to the generated data file, for example, The file identifier corresponding to the generated data file is set in order from small to large. When the first data file is generated, the file identifier corresponding to the first data file generated is set to 1; when the second data file is generated, The file identifier corresponding to the generated second data file is set to 2; when the third data file is generated, the file identifier corresponding to the generated third data file is set to 3 or the like. It can be understood by those skilled in the art that the first preset duration can be flexibly set according to actual conditions, and the file identifier corresponding to each data file generated can also be set according to other rules, and is not limited herein. At the same time, each time the data file is generated, the file identifier corresponding to the generated data file is written into the first configuration file.

The processing module 20 is configured to write the data file into the distributed cache system, and write the file identifier corresponding to the data file successfully written into the second configuration file when the writing is successful;

After the generating module 10 generates a data file each time and writes the file identifier corresponding to the data file into the foreground first configuration file, the processing module 20 writes the generated data file into the distributed cache system. And each time the data file is successfully written, the file identifier corresponding to the successfully written data file is written into the second configuration file.

The storage module 30 is configured to detect the file identifiers different from the first configuration file and the second configuration file every second preset duration, and identify different data files corresponding to the different files. Saved to the distributed cache system.

The storage module 30 periodically compares the file identifiers written in the first configuration file and the second configuration file, and detects whether different file identifiers exist in the first configuration file and the second configuration file. In this embodiment, the second preset duration is also set in advance. The storage module 30 detects whether there is a different file identifier in the first configuration file and the second configuration file when the second preset duration is separated, when the first configuration file and the first When different file identifiers exist in the second configuration file, the data files corresponding to the different file identifiers are written into the distributed cache system. For example, the file identifier in the first configuration file is file identifier 1 to file identifier 10, and the second configuration file The file identifiers in the file are file identifier 1 to file identifier 5, and the different file identifiers in the first configuration file and the second configuration file are file identifier 6 to file identifier 10, that is, the file identifier 6 to the file. If the data file corresponding to the identifier 10 is not successfully written into the distributed cache system, the storage module 30 writes the data file corresponding to the file identifier 6 to the file identifier 10 into the distributed cache system.

In the solution provided by the embodiment, the generating module 10 periodically generates a data file by receiving the data, and each time the data file is generated, setting a file identifier corresponding to the data file, and writing the file identifier into the first configuration file. The processing module 20 writes the generated data file into the distributed cache system, and writes the file identifier corresponding to the successfully written data file into the second configuration file when the write is successful, and the storage module The timer detects whether there is a different file identifier in the first configuration file and the second configuration file. If there is a different file identifier in the first configuration file and the second configuration file, the ENUM-DNS server is The data in the distributed cache system is inconsistent. At this time, the storage module 30 saves the data files corresponding to the different file identifiers to the distributed cache system, because the data files in the ENUM-DNS server and the distributed cache system are not required. Contrast, greatly reducing the time to detect whether the ENUM-DNS server is consistent with the data in the distributed cache system, thus improving the Data protection ENUM-DNS server with distributed cache coherence system efficiency, improve the reliability of ENUM-DNS server.

A second embodiment of the data management apparatus of the present invention is proposed based on the first embodiment. In this embodiment, the generating module 10 is further configured to write the file identifier corresponding to the generated data file into the first configuration file. To update the file identifier saved in the first configuration file;

The processing module 20 is further configured to write the file identifier corresponding to the saved data file to the second configuration file to update the file identifier saved in the second configuration file; wherein when a certain data file is not saved On success, the data file is continuously written repeatedly until the write is successful, or the alarm is reported when the write fails;

The storage module 30 is further configured to detect whether the file identifier saved in the first configuration file is consistent with the file identifier saved in the second configuration file; and the file identifier saved in the first configuration file When the file identifiers saved in the second configuration file are inconsistent, the data files corresponding to the file identifiers of the difference between the file identifiers saved in the first configuration file and the file identifiers saved in the second configuration file are written. To the distributed cache system.

In this embodiment, the file identifier is a file number added to the data file according to the receiving order, and the generating module 10 writes the file identifier saved in the first configuration file by using the file identifier corresponding to the generated data file. To update the file identifier saved in the first configuration file. For example, when generating When the module 10 generates the data file for the first time, the file identifier corresponding to the first data file generated is set to 1, and the file identifier 1 corresponding to the generated data file is written into the first configuration file and saved. When the generating module 10 generates the data file for the second time, setting the file identifier corresponding to the generated second data file to 2, writing the file identifier 2 into the first configuration file, and replacing the saved file identifier 1 to update The file identifier saved in the first configuration file. When the generating module 10 generates the data file for the third time, the file identifier corresponding to the generated third data file is set to 3, and the file identifier 3 is written into the first configuration file, and the saved file identifier 2 is replaced. In response to this, each time the generation module 10 generates a data file, the file identifier saved in the first configuration file is updated.

When the processing module 20 successfully writes the generated first data file into the distributed cache system, the file identifier 1 corresponding to the successfully written first data file is written into the second configuration file and saved. When the processing module 20 successfully writes the generated second data file to the distributed cache system, the file identifier 2 corresponding to the successfully written second data file is written into the second configuration file, and is replaced and saved. The file identifier 1 is to update the file identifier saved in the second configuration file. When the processing module 20 successfully saves the generated third data file to the distributed cache system, the file identifier 3 corresponding to the successfully saved third data file is written into the second configuration file, and the saved file is replaced. Logo 2. Accordingly, each time the processing module 20 successfully writes the data file to the distributed cache system, the file identifier saved in the second configuration file is updated. When a certain data file is not successfully written, the processing module 20 does not write other unwritten data files, continuously writes the data file repeatedly until the writing is successful, or reports an alarm when the writing fails.

The storage module 30 periodically detects whether the file identifier saved in the first configuration file is consistent with the file identifier saved in the second configuration file, for example, detecting the saving in the first configuration file every second preset duration Whether the file identifier is consistent with the file identifier saved in the second configuration file. And deleting, when the file identifier saved in the first configuration file is different from the file identifier saved in the second configuration file, the file identifier saved in the second configuration file and the file saved in the first configuration file A data file corresponding to the file identifier of the difference between the identifiers is written into the distributed cache system. For example, if the file identifier saved in the first configuration file is 10 and the file identifier saved in the second configuration file is 5, the file identifier saved in the first configuration file and the second configuration file are The file identifiers saved in the file are inconsistent, and the storage module 30 identifies the file identifiers stored in the second configuration file to a data file corresponding to the file identifier of the difference between the file identifiers 10 stored in the first configuration file, that is, the file. The five data files corresponding to the identifier 6 to the file identifier 10 are written into the distributed cache system.

In the solution provided by the embodiment, the generating module 10 updates the file identifier saved in the first configuration file according to the file identifier corresponding to the data file each time the data file is generated, and the processing module 20 succeeds in generating the data file. When the file is written to the distributed cache system, the file identifier saved in the second configuration file is updated according to the file identifier corresponding to the data file, and the storage module 30 periodically detects the first configuration file and the second configuration file. If the file identifiers are consistent, if the first configuration file is inconsistent with the file identifier saved in the second configuration file, the data in the ENUM-DNS server is different from that in the distributed cache system. In this case, the storage module 30 The data file corresponding to the file identifier of the difference between the file identifier saved in the second configuration file and the file identifier saved in the first configuration file is written into the distributed cache system, so only the comparison The first configuration file and the file identifier saved in the second configuration file reduce detection of ENUM-DNS server and distributed The data storage system is consistent in time, thereby improving the reliability of ENUM-DNS server.

As shown in FIG. 4, a third embodiment of the data management apparatus of the present invention is proposed based on the first embodiment or the second embodiment. In this embodiment, the data management apparatus further includes:

The sending module 40 is configured to send a detection message to a node corresponding to each preset space in the distributed cache system every third preset duration, and detect whether the node corresponding to each preset space is received. Response message

The determining module 50 is configured to determine that the node corresponding to the preset space is an abnormal node when the response message fed back by the node corresponding to the preset space is not received;

The alarm module 60 is configured to count the number of abnormal nodes in the distributed cache system. If the number of the abnormal nodes is greater than a preset threshold, the alarm information is sent to the operation and maintenance system, and the function of receiving data is closed. The new data file is no longer generated every first preset interval until the system returns to normal.

In this embodiment, a preset space corresponding to each node in the distributed cache system is preset, and a special space is set, for example, the special space is set to space 0, and the space 0 is used for After the generating module 10 generates the file of the data of the overhead user, the file generated by the generating module 10 is saved, and the saved file is evenly loaded to the nodes corresponding to each of the preset spaces, thereby ensuring balanced storage and query data of each node. . The preset space corresponds to the nodes one by one, that is, each node is completely loaded with one query space.

When the ENUM-DNS server is initialized, a fixed record is written to each preset space, and the fixed record is a fixed record corresponding to the initialization of the ENUM-DNS server, and the form is not limited herein. system. When the write record is repeatedly written, as long as the key value of the record repeatedly written is the same as the key value of the record previously written, the record repeatedly written overwrites the previously written record. When the write fails, the ENUM-DNS server is not allowed to start, and the ENUM-DNS server fails to initialize; when the write is successful, the ENUM-DNS server is successfully initialized. After the ENUM-DNS server is successfully initialized, the sending module 40 sends a detection message to each node corresponding to the preset space every third preset duration. For example, the sending module 40 corresponds to each preset space every 5 minutes. The node sends a detection message. If the node corresponding to the preset space is a normal node, the node corresponding to the preset space feeds back the response message to the sending module 40 when receiving the detection message; if the node corresponding to the preset space is an abnormal node, the response cannot be fed back. The message is sent to the sending module 40.

After sending the detection message to the node corresponding to each preset space, the sending module 40 detects whether the response message fed back by the node corresponding to each preset space is received, and if the response message of the node corresponding to the preset space is not received, The determining module 50 determines that the node corresponding to the preset space is an abnormal node. If the response message fed back by the node corresponding to the preset space is received, the determining module 50 determines that the node corresponding to the preset space is a normal node.

After that, the number of abnormal nodes in the distributed cache system is counted. When the number of the abnormal nodes is greater than the preset number threshold A, the data in the ENUM-DNS server and the data in the distributed cache system cannot be guaranteed. When the consistency is reached, the alarm module 60 sends an alarm message to the operation and maintenance system. When the number of the abnormal nodes is less than or equal to the preset number threshold A, although there is theoretical possibility that the data in the ENUM-DNS server is inconsistent with the data in the distributed cache system, in order to balance the availability of data It can be considered that the data in the ENUM-DNS server can still be guaranteed to be consistent with the data in the distributed cache system. The value of the preset number threshold A is set according to the NRW copy policy adopted by the distributed cache system, for example, if the distributed cache system adopts a 3-2-2 copy policy, the preset number threshold The value of A is set to 3. When there are more than three nodes in the distributed cache system as abnormal nodes, the consistency between the data in the ENUM-DNS server and the data in the distributed cache system cannot be guaranteed. At this time, the alarm module 60 sends an alarm to the operation and maintenance system, and does not allow the operation and maintenance system or the SPN to continue to send the data of the overhead household to the ENUM-DNS server to ensure the data in the ENUM-DNS server and the data in the distributed cache system. Consistency. That is, the function of receiving data sent by the operation and maintenance system or the SPN is closed, and a new data file is no longer generated every interval of the first preset duration until the system returns to normal.

In this embodiment, the determining module 50 is further configured to:

In order to improve the accuracy of determining whether the node corresponding to each preset space in the distributed cache system is an abnormal node, in the embodiment, after the sending module 40 sends the detection message to the node corresponding to each preset space, If the response message fed back by the node corresponding to the preset space is not received, the determining module 50 updates the number of times that the response message fed back by the node corresponding to the preset space is not received, and determines whether the number of times is greater than or equal to the preset number of times. And determining that the node corresponding to the preset space is an abnormal node, when the number of the response messages fed back by the node corresponding to the preset space is not greater than or equal to the preset number of times. For example, if the preset number of times is set to 3 times, when the sending module 40 sends the detection message 3 times to the node corresponding to each preset space, if the node corresponding to a preset space is not received 3 times, the feedback is not received. In response to the message, the determining module 50 determines that the node corresponding to the preset space is an abnormal node.

In the solution provided by the embodiment, the sending module 40 periodically sends a detection message to the node corresponding to each preset space in the distributed cache system, and detects whether a response message fed back by the node corresponding to each preset space is received. When the response message of the node corresponding to the preset space is not received, the determining module 50 determines that the node corresponding to the preset space is an abnormal node, and counts the number of abnormal nodes in the distributed cache system, and when the abnormal node When the number of the number is greater than the preset number threshold, that is, when the data in the ENUM-DNS server cannot be guaranteed to be consistent with the data in the distributed cache system, the alarm module 60 sends an alarm message to the operation and maintenance system, thereby achieving both Data consistency and availability improve the reliability of ENUM-DNS servers.

A fourth embodiment of the data management apparatus of the present invention is provided based on any of the foregoing embodiments. In this embodiment, the data management apparatus further includes:

Due to the complexity of the IMS network environment, the distributed cache system may have catastrophic failures, resulting in the loss of data of nodes corresponding to some preset spaces or unreliable data. At this point, you need to recover the data of these abnormal nodes with data failures. In this embodiment, each of the distributed cache systems has a backup system. Normally, the data stored in the standby node of the standby system is consistent with the data held by the distributed cache system. When recovering data of the abnormal node in the distributed cache system, the data recovery module is based on the file directory of the abnormal node. The data node corresponding to the file directory is sequentially copied to the abnormal node in the standby node of the standby system, thereby completing the recovery of the data of the abnormal node. If the file directory of the abnormal node is also damaged, that is, the data file cannot be copied from the standby node through the file directory, the preset space corresponding to each node in the distributed cache system is reset, and the ENUM-DNS server is re-initialized. After the initialization is completed, the data recovery module sequentially copies all the data saved in the standby node of the standby system to the node corresponding to each preset space in the distributed cache system.

In this embodiment, when the data of the abnormal node is restored, the data recovery module sequentially copies the data file corresponding to the file directory from the standby node to the abnormal node according to the file directory of the abnormal node. The recovery of the data of the abnormal node is completed without copying all the data files, thereby realizing the rapid recovery of the data, thereby improving the reliability of the ENUM-DNS server.

The embodiment of the present invention further provides a non-transitory computer readable storage medium storing computer executable instructions, the computer executable instructions being configured to perform the method in any of the above embodiments.

The embodiment of the invention further provides a schematic structural diagram of an electronic device. Referring to FIG. 5, the electronic device includes:

At least one processor 50, which is exemplified by a processor 50 in FIG. 5; and a memory 51, may further include a communication interface 52 and a bus 53. The processor 50, the communication interface 52, and the memory 51 can complete communication with each other through the bus 53. Communication interface 52 can be used for information transmission. Processor 50 can invoke logic instructions in memory 51 to perform the methods of the above-described embodiments.

Furthermore, the logic instructions in the memory 51 described above may be implemented in the form of software functional units and sold or used as separate products, and may be stored in a computer readable storage medium.

The memory 51 is a computer readable storage medium, and can be used to store a software program, a computer executable program, and a program instruction/module corresponding to the method in the embodiment of the present invention. The processor 50 executes the function application and the data processing by executing the software programs, the instructions, and the modules stored in the memory 51, that is, the data management method in the above method embodiments.

The memory 51 may include a storage program area and an storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created according to use of the terminal device, and the like. Further, the memory 51 may include a high speed random access memory, and may also include a nonvolatile memory.

The technical solution of the embodiment of the present invention may be embodied in the form of a software product, the computer software The product is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present invention. The foregoing storage medium may be a non-transitory storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and the like. A medium that can store program code, or a transitory storage medium.

The above are only the embodiments of the present invention, and are not intended to limit the scope of the invention, and the equivalent structure or equivalent process transformations made by the description of the present invention and the drawings are directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of the present invention.

Industrial applicability

The present invention improves the efficiency of maintaining data consistency in an ENUM-DNS server and a distributed cache system.

Claims

A data management method that includes:

Generating a data file for the received data every first preset duration, setting a file identifier corresponding to the data file, saving the data file, and writing the file identifier corresponding to the data file into the first configuration file;

Writing the data file to the distributed cache system, and writing the file identifier corresponding to the successfully written data file to the second configuration file when the writing is successful;

Detecting, differently, the file identifiers in the first configuration file and the second configuration file every second preset duration, and writing different data files corresponding to the file identifiers to the distributed cache In the system.
The method of claim 1, wherein the file identifier corresponding to the data file is a file number added to the data file in a receiving order, and the file identifier corresponding to the data file is written in the first configuration file. The steps include:

Writing, by using the file identifier corresponding to the data file, the first configuration file to update the file identifier saved in the first configuration file;

The steps of writing the file identifier corresponding to the data file successfully written into the distributed cache system into the second configuration file include:

Writing, by using a file identifier corresponding to the data file successfully written in the distributed cache system, to the second configuration file, to update the file identifier saved in the second configuration file; wherein when a data file is not successfully written , repeatedly write the data file until the write is successful, or report the alarm when the write fails;

The step of detecting the file identifiers in the first configuration file and the second configuration file, and writing the data files corresponding to the different file identifiers to the distributed cache system includes:

Detecting whether the file identifier saved in the first configuration file is consistent with the file identifier saved in the second configuration file;

And deleting, when the file identifier saved in the first configuration file is different from the file identifier saved in the second configuration file, the file identifier saved in the first configuration file and the file saved in the second configuration file A data file corresponding to the file identifier of the difference between the identifiers is written into the distributed cache system.
The method of claim 1 wherein said every other first predetermined duration is performed The collected data generates a data file, and the steps of setting the file identifier corresponding to the data file are performed, and the following operations are performed:

Sending a detection message to a node corresponding to each preset space in the distributed cache system every third preset duration, and detecting whether a response message fed back by the node corresponding to each preset space is received;

When the response message fed back by the node corresponding to the preset space is not received, determining that the node corresponding to the preset space is an abnormal node;

The number of the abnormal nodes in the distributed cache system is counted. If the number of the abnormal nodes is greater than the preset number threshold, the alarm information is sent to the operation and maintenance system, and the function of receiving data is closed, and the first preset is separated. The duration no longer generates new data files until the system returns to normal.
The method of claim 3, wherein, when the response message fed back by the node corresponding to the preset space is not received, determining that the node corresponding to the preset space is an abnormal node comprises:

Updating the number of times the response message is not received when the response message fed back by the node corresponding to the preset space is not received;

Determining whether the number of times is greater than or equal to a preset number of times, wherein the node corresponding to the preset space is an abnormal node when the number of times the response message is not received is greater than or equal to a preset number of times.
The method of any of claims 1-4, further comprising:

When the data file of the node in the distributed cache system is faulty, the data file corresponding to the file directory is sequentially copied from the standby node to the node according to the file directory of the node.
A data management device comprising:

Generating a module, configured to generate a data file for the received data every other first preset duration, set a file identifier corresponding to the data file, save the data file, and write the file identifier corresponding to the data file In the first configuration file;

The processing module is configured to write the data file into the distributed cache system, and write the file identifier corresponding to the data file successfully written into the second configuration file when the writing is successful;

The storage module is configured to detect the file identifiers different from the first configuration file and the second configuration file every second preset duration, and write different data files corresponding to the file identifiers Into the distributed cache system.
The apparatus of claim 6, wherein the file identifier is a file number added to the data file in a receiving order; the generating module is further configured to write a file with a file identifier corresponding to the generated data file In the first configuration file, to update the file identifier saved in the first configuration file;

The processing module is further configured to be written in the second configuration file by using a file identifier corresponding to the data file successfully written to the distributed cache system, to update the file identifier saved in the second configuration file; When the data file is not saved successfully, the data file is continuously written repeatedly until the writing is successful, or the alarm is reported when the writing fails;

The storage module is further configured to: detect whether the file identifier saved in the first configuration file is consistent with the file identifier saved in the second configuration file; and the file identifier and the file saved in the first configuration file When the file identifiers saved in the second configuration file are inconsistent, the data files corresponding to the file identifiers of the differences between the file identifiers saved in the first configuration file and the file identifiers saved in the second configuration file are written to In the distributed cache system.
The apparatus of claim 6 further comprising:

The sending module is configured to send a detection message to a node corresponding to each preset space in the distributed cache system every third preset duration, and detect whether a response of the node corresponding to each preset space is received. Message

The determining module is configured to determine that the node corresponding to the preset space is an abnormal node when the response message fed back by the node corresponding to the preset space is not received;

The alarm module is configured to count the number of abnormal nodes in the distributed cache system. If the number of the abnormal nodes is greater than a preset threshold, the alarm information is sent to the operation and maintenance system, and the function of receiving data is closed. The new data file is no longer generated every first preset interval until the system returns to normal.
The apparatus of claim 8 wherein said determining module is configured to:

Updating the number of times the response message is not received when the response message fed back by the node corresponding to the preset space is not received;

Determining whether the number of times is greater than or equal to a preset number of times, wherein the node corresponding to the preset space is an abnormal node when the number of times the response message is not received is greater than or equal to a preset number of times.
The apparatus of any of claims 6-9, further comprising:

The data recovery module is configured to, when the data file of the node in the distributed cache system is faulty, sequentially copy the data file corresponding to the file directory from the standby node to the node according to the file directory of the node.
A non-transitory computer readable storage medium storing computer executable instructions arranged to perform the method of any of claims 1-5.