WO2017080362A1 - Procédé et dispositif de gestion de données - Google Patents

Procédé et dispositif de gestion de données Download PDF

Info

Publication number
WO2017080362A1
WO2017080362A1 PCT/CN2016/103529 CN2016103529W WO2017080362A1 WO 2017080362 A1 WO2017080362 A1 WO 2017080362A1 CN 2016103529 W CN2016103529 W CN 2016103529W WO 2017080362 A1 WO2017080362 A1 WO 2017080362A1
Authority
WO
WIPO (PCT)
Prior art keywords
file
data
node
configuration file
saved
Prior art date
Application number
PCT/CN2016/103529
Other languages
English (en)
Chinese (zh)
Inventor
张朋飞
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2017080362A1 publication Critical patent/WO2017080362A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/172—Caching, prefetching or hoarding of files
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/10—Architectures or entities
    • H04L65/1016—IP multimedia subsystem [IMS]

Definitions

  • the present application relates to the field of data management technologies in an IMS (IP Multimedia Subsystem) network, for example, to a data management method and apparatus.
  • IMS IP Multimedia Subsystem
  • IMS is a new form of multimedia service that meets the needs of users for more innovative and diverse multimedia services.
  • the ENUM-DNS (E.164 Number URI Mapping-Domain Name System) server provides the IMS with a phone number and domain name and a host resource correspondence query service.
  • ENUM-DNS servers With the development of technology, IMS has higher and higher performance requirements for ENUM-DNS servers, such as ensuring data consistency between ENUM-DNS servers and distributed cache systems.
  • the ENUM-DNS server is usually compared with the data in the distributed cache system, and the data is compared when the comparison result is different. Consistency maintenance. Because of the long time taken to compare data, it is less efficient to maintain data consistency between the ENUM-DNS server and the distributed cache system.
  • the embodiment of the invention provides a data management method and device, which aims to solve the technical problem of low maintenance efficiency of data consistency in the ENUM-DNS server and the distributed cache system.
  • the data management method is implemented by software, and the method includes the following steps:
  • the file identifier may be a file number added to the data file in the order of receiving, and the step of writing the file identifier corresponding to the data file into the first configuration file may include:
  • the steps of writing the file identifier corresponding to the data file successfully written into the distributed cache system into the second configuration file include:
  • the step of detecting the file identifiers in the first configuration file and the second configuration file, and writing the data files corresponding to the different file identifiers to the distributed cache system includes:
  • the number of the abnormal nodes in the distributed cache system is counted. If the number of the abnormal nodes is greater than the preset number threshold, the alarm information is sent to the operation and maintenance system, and the function of receiving data is closed, and the first preset is separated. The duration no longer generates new data files until the system returns to normal.
  • the step of determining that the node corresponding to the preset space is an abnormal node may include:
  • the update does not receive the received message. The number of times the response message is reported;
  • the data management method may further include:
  • the data file corresponding to the file directory is sequentially copied from the standby node to the node according to the file directory of the node.
  • an embodiment of the present invention further provides a data management apparatus, where the data management apparatus includes:
  • Generating a module configured to generate a data file for the received data every other first preset duration, set a file identifier corresponding to the data file, save the data file, and write the file identifier corresponding to the data file In the first configuration file;
  • a processing module configured to write the data file into the distributed cache system, and write the file identifier corresponding to the saved data file to the second configuration file when the writing is successful;
  • the storage module is configured to detect that the first configuration file and the data file are saved to the distributed cache system every second preset duration.
  • the file identifier may be a file number added to the data file in the order of receiving, and the generating module may be further configured to write the file identifier corresponding to the generated data file into the first configuration file to update a file identifier saved in the first configuration file;
  • the processing module is further configured to write the file identifier corresponding to the successfully written data file into the second configuration file to update the file identifier saved in the second configuration file; wherein when a certain data file is not saved On success, the data file is continuously written repeatedly until the write is successful, or the alarm is reported when the write fails;
  • the storage module is further configured to: detect whether the file identifier saved in the first configuration file is consistent with the file identifier saved in the second configuration file; and the file identifier and the file saved in the first configuration file When the file identifiers saved in the second configuration file are inconsistent, the data files corresponding to the file identifiers of the differences between the file identifiers saved in the first configuration file and the file identifiers saved in the second configuration file are written to In the distributed cache system.
  • the data management device may further include:
  • the sending module is configured to send a detection message to a node corresponding to each preset space in the distributed cache system every third preset duration, and detect whether a response of the node corresponding to each preset space is received.
  • the determining module is configured to receive a response message of the node corresponding to a certain preset space Determining, that the node corresponding to the preset space is an abnormal node;
  • the alarm module is configured to count the number of abnormal nodes in the distributed cache system. If the number of the abnormal nodes is greater than a preset threshold, the alarm information is sent to the operation and maintenance system, and the function of receiving data is closed. The new data file is no longer generated every first preset interval until the system returns to normal.
  • the determining module can be configured to:
  • the data management device may further include:
  • the data recovery module is configured to, when the data file of the node in the distributed cache system is faulty, sequentially copy the data file corresponding to the file directory from the standby node to the node according to the file directory of the node.
  • the embodiment of the invention further provides a non-transitory computer readable storage medium storing computer executable instructions, the computer executable instructions being arranged to perform the above method.
  • An embodiment of the present invention further provides an electronic device, including:
  • At least one processor At least one processor
  • the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform the method described above.
  • a data management method and apparatus in the data management method, periodically generating a data file from the received data, and writing a file identifier corresponding to the data file into the first configuration file, and successfully writing the data file
  • the file identifier corresponding to the successfully written data file is written into the second configuration file, and the file identifiers of the first configuration file and the second configuration file are compared, and
  • FIG. 1 is a schematic flowchart of a first embodiment of a data management method according to the present invention
  • FIG. 2 is a schematic flowchart of a third embodiment of a data management method according to the present invention.
  • FIG. 3 is a schematic diagram of functional modules of a first embodiment of a data management apparatus according to the present invention.
  • FIG. 4 is a schematic diagram of functional modules of a third embodiment of a data management apparatus according to the present invention.
  • FIG. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
  • FIG. 1 is a schematic flowchart of a first embodiment of a data management method according to the present invention.
  • the data management method includes the following steps:
  • Step S10 Generate a data file for the received data every first preset duration, set a file identifier corresponding to the data file, save the data file, and write the file identifier corresponding to the data file into the first configuration file. in;
  • the O&M system or the SPN delivers the data of the overhead household to the ENUM-DNS (E.164Number URI Mapping-Domain Name System) server.
  • the ENUM-DNS server is timed.
  • the data to be received is generated into a data file.
  • the first preset duration is set in advance, and the ENUM-DNS server generates a data file for the data received within the first preset duration every time the first preset duration is separated. And set the file identifier corresponding to the generated file. For example, if the first preset duration is 2 minutes, the ENUM-DNS server generates a data file for the data received within the 2 minute period every 2 minutes, and sets a file identifier corresponding to the generated data file.
  • the file identifier corresponding to the generated data file is set in order from small to large.
  • the file identifier corresponding to the generated first data file is set to 1; when the second data file is generated.
  • the file identifier corresponding to the generated second data file is set to 2; when the third data file is generated, the file identifier corresponding to the generated third data file is set to 3 or the like.
  • the first preset duration can be flexibly set according to actual conditions, and the file identifier corresponding to each data file generated can also be set according to other rules, and is not limited herein.
  • the file identifier corresponding to the generated data file is written into the first configuration file.
  • Step S20 the data file is written into the distributed cache system, and when the writing is successful, the file identifier corresponding to the data file successfully written is written into the second configuration file;
  • the ENUM-DNS server writes the generated data file to the distributed cache system each time the data file is generated and the file identifier corresponding to the data file is written into the first configuration file. And each time the data file is successfully saved, the file identifier corresponding to the successfully saved data file is written into the second configuration file.
  • Step S30 detecting, according to the second preset duration, the file identifiers different from the first configuration file and the second configuration file, and writing different data files corresponding to the file identifiers to the Distributed cache system.
  • the ENUM-DNS server periodically compares the file identifiers written in the first configuration file and the second configuration file, and detects whether different file identifiers exist in the first configuration file and the second configuration file.
  • the second preset duration is also set in advance.
  • the ENUM-DNS server detects whether there is a different file identifier in the first configuration file and the second configuration file when the second preset duration is separated, when the first configuration file is detected and the When different file identifiers exist in the second configuration file, the data files corresponding to the different file identifiers are written into the distributed cache system.
  • the file identifier in the first configuration file is file identifier 1 to file identifier 10
  • the file identifier in the second configuration file is file identifier 1 to file identifier 5
  • the first configuration file is The different file identifiers in the second configuration file are file identifier 6 to file identifier 10, that is, the data files corresponding to the file identifier 6 to the file identifier 10 are not successfully saved to the distributed cache system, and the ENUM-DNS server will The data files corresponding to the file identifier 6 to the file identifier 10 are written into the distributed cache system.
  • the ENUM-DNS server generates a data file by periodically receiving the data, and each time the data file is generated, setting a file identifier corresponding to the data file, and writing the file identifier to the first configuration file. And then, the generated data file is written into the distributed cache system, and when the writing is successful, the file identifier corresponding to the data file successfully written is written into the second configuration file, ENUM- The DNS server periodically detects whether there is a different file identifier in the first configuration file and the second configuration file. If there is a different file identifier in the first configuration file and the second configuration file, the ENUM-DNS server is configured. Inconsistent with the data in the distributed cache system.
  • the ENUM-DNS server saves the files corresponding to different file identifiers to the distributed cache system, because the data in the ENUM-DNS server and the distributed cache system are not required. Comparison of files greatly reduces the detection of data in the ENUM-DNS server and the distributed cache system. Time, which improves the efficiency of maintaining data consistency between the ENUM-DNS server and the distributed cache system, and improves the reliability of the ENUM-DNS server.
  • the step S10 includes: writing, by using the file identifier corresponding to the data file, the first configuration file to update the The file identifier saved in the first configuration file;
  • the step S20 includes: writing a file identifier corresponding to the saved data file to the second configuration file to update the file identifier saved in the second configuration file; wherein when a data file is not successfully saved , repeatedly write the data file until the write is successful, or report the alarm when the write fails;
  • the step S30 includes: detecting whether the file identifier saved in the first configuration file is consistent with the file identifier saved in the second configuration file; and the file identifier saved in the first configuration file and the second When the file identifiers saved in the configuration file are inconsistent, the data file corresponding to the file identifier of the difference between the file identifier saved in the first configuration file and the file identifier saved in the second configuration file is written to the distribution. In the cache system.
  • the file identifier is a file number added to the data file according to the receiving order
  • the ENUM-DNS server writes the file identifier saved in the first configuration file by using the file identifier corresponding to the generated data file.
  • the file identifier saved in the first configuration file For example, when the ENUM-DNS server generates the data file for the first time, the file identifier corresponding to the first data file generated is set to 1, and the file identifier 1 corresponding to the generated data file is written into the first configuration file and saved.
  • the ENUM-DNS server When the ENUM-DNS server generates the data file for the second time, set the file identifier corresponding to the generated second data file to 2, write the file identifier 2 into the first configuration file, and replace the saved file identifier 1 with Update the file identifier saved in the first configuration file.
  • the file identifier corresponding to the third data file generated is set to 3
  • the file identifier 3 is written into the first configuration file, and the saved file identifier 2 is replaced.
  • the file identifier saved in the first configuration file is updated.
  • the file identifier 1 corresponding to the first data file successfully written is written into the second configuration file and saved.
  • the ENUM-DNS server successfully saves the generated second data file to the distributed cache system writes the file identifier 2 corresponding to the successfully saved second data file to the second configuration file, and replaces the saved file.
  • File ID 1 to update the file ID saved in the second configuration file.
  • the ENUM-DNS server successfully writes the generated third data file to the distributed cache system.
  • the file identifier 3 corresponding to the third data file successfully written is written into the second configuration file, the saved file identifier 2 is replaced.
  • the file identifier saved in the second configuration file is updated.
  • other generated data files that have not been written are not written, and the data file is continuously written repeatedly until the writing is successful, or an alarm is reported when the writing fails.
  • the ENUM-DNS server periodically detects whether the file identifier saved in the first configuration file is consistent with the file identifier saved in the second configuration file. And deleting, when the file identifier saved in the first configuration file is different from the file identifier saved in the second configuration file, the file identifier saved in the second configuration file and the file saved in the first configuration file A data file corresponding to the file identifier of the difference between the identifiers is written into the distributed cache system.
  • the file identifier saved in the first configuration file is 10 and the file identifier saved in the second configuration file is 5, the file identifier saved in the first configuration file and the second configuration file are The file identifiers saved in the file are inconsistent, and the ENUM-DNS server compares the file identifiers corresponding to the file identifiers of the file identifiers 5 saved in the second configuration file with the file identifiers 10 stored in the first configuration file, that is, The five data files corresponding to the file identifier 6 to the file identifier 10 are saved in the distributed cache system.
  • the ENUM-DNS server updates the file identifier saved in the first configuration file according to the file identifier corresponding to the data file each time the data file is generated, and successfully writes the generated data file.
  • the file identifier saved in the second configuration file is updated according to the file identifier corresponding to the data file, and the ENUM-DNS server periodically detects the first configuration file and the second configuration file. If the file identifiers are consistent, if the first configuration file is inconsistent with the file identifier saved in the second configuration file, the data in the ENUM-DNS server is different from the distributed cache system.
  • the ENUM-DNS server Saving a data file corresponding to the file identifier of the difference between the file identifier saved in the second configuration file and the file identifier saved in the first configuration file to the distributed cache system, so The first configuration file and the file identifier saved in the second configuration file reduce the detection of the ENUM-DNS server and the distributed Whether the data in the system is consistent or not, thereby improving the reliability of the ENUM-DNS server.
  • a third embodiment of the data management method of the present invention is proposed based on the first embodiment or the second embodiment.
  • the data management method further includes the following steps:
  • Step S40 Send a detection message to a node corresponding to each preset space in the distributed cache system every third preset duration, and detect whether a response of the node corresponding to each preset space is received.
  • step S50 when the response message fed back by the node corresponding to the preset space is not received, it is determined that the node corresponding to the preset space is an abnormal node;
  • Step S60 The number of abnormal nodes in the distributed cache system is counted. If the number of the abnormal nodes is greater than a preset number threshold, the alarm information is sent to the operation and maintenance system, and the function of receiving data is closed. A new data file is no longer generated for a preset duration until the system returns to normal.
  • a preset space corresponding to each node in the distributed cache system is preset, and a special space is set, for example, the special space is set to space 0, and the space 0 is used for
  • the ENUM-DNS server After the ENUM-DNS server generates the file of the data of the overhead user, the file generated by the ENUM-DNS server is saved, and the saved file is evenly loaded to the nodes corresponding to each preset space, thereby ensuring balanced storage of each node and Query data.
  • the preset space corresponds to the nodes one by one, that is, each node is completely loaded with one query space.
  • the ENUM-DNS server When the ENUM-DNS server is initialized, a fixed record is written to each preset space, and the fixed record is a fixed record corresponding to the ENUM-DNS server initialization, and the form is not limited herein.
  • the write record When the write record is repeatedly written, as long as the key value of the record repeatedly written is the same as the key value of the record previously written, the record repeatedly written overwrites the previously written record.
  • the write fails the ENUM-DNS server is not allowed to start, and the ENUM-DNS server fails to initialize; when the write is successful, the ENUM-DNS server is successfully initialized.
  • the ENUM-DNS server After the ENUM-DNS server is successfully initialized, the ENUM-DNS server periodically sends a detection message to each node corresponding to the preset space for each third preset duration. For example, the ENUM-DNS server provides each preset space every 5 minutes. The corresponding node sends a detection message. If the node corresponding to the preset space is a normal node, the node corresponding to the preset space feeds back the response message to the ENUM-DNS server when receiving the detection message; if the node corresponding to the preset space is an abnormal node, the node cannot be fed back. Response message to the ENUM-DNS server.
  • the ENUM-DNS server After sending the detection message to the node corresponding to each preset space, the ENUM-DNS server detects whether the response message fed back by the node corresponding to each preset space is received, and if the response of the node corresponding to the preset space is not received, The message is determined to be an abnormal node; if the response message fed back by the node corresponding to the preset space is received, the node corresponding to the preset space is determined to be a normal node.
  • the ENUM-DNS server counts the number of abnormal nodes in the distributed cache system. When the number of the abnormal nodes is greater than the preset threshold A, the data and the distributed data in the ENUM-DNS server cannot be guaranteed. ENUM-DNS server to operation and maintenance when the data in the cache system is consistent The system sends an alarm message. When the number of the abnormal nodes is less than or equal to the preset number threshold A, although there is theoretical possibility that the data in the ENUM-DNS server is inconsistent with the data in the distributed cache system, in order to balance the availability of data It can be considered that the data in the ENUM-DNS server can still be guaranteed to be consistent with the data in the distributed cache system.
  • the value of the preset number threshold A is set according to the NRW copy policy adopted by the distributed cache system, for example, if the distributed cache system adopts a 3-2-2 copy policy, the preset number threshold The value of A is set to 3.
  • the ENUM-DNS server sends an alarm to the operation and maintenance system, and does not allow the operation and maintenance system or the SPN to continue to send the data of the overhead household to the ENUM-DNS server to ensure the data in the ENUM-DNS server and the distributed cache system.
  • Data consistency That is, the function of receiving data sent by the operation and maintenance system or the SPN is closed, and a new data file is no longer generated every interval of the first preset duration until the system returns to normal.
  • the step S50 includes:
  • step a when the response message fed back by the node corresponding to the preset space is not received, the number of times the response message is not received is updated;
  • step b it is determined whether the number of times is greater than or equal to a preset number of times, wherein the node corresponding to the preset space is an abnormal node when the number of times the response message is not received is greater than or equal to a preset number of times.
  • the ENUM-DNS server sends a detection message to each node corresponding to the preset space each time. And if the response message fed back by the node corresponding to the preset space is not received, updating the number of times the response message fed back by the node corresponding to the preset space is not received.
  • the ENUM-DNS server determines whether the number of times is greater than or equal to a preset number of times.
  • the node corresponding to the preset space determines that the node corresponding to the preset space is Anomalous node. For example, if the preset number of times is set to 3 times, when the ENUM-DNS server sends the detection message 3 times to the node corresponding to each preset space, if the ENUM-DNS server does not receive a preset space 3 times.
  • the response message fed back by the corresponding node determines that the node corresponding to the preset space is an abnormal node.
  • the ENUM-DNS server periodically sends a detection message to the node corresponding to each preset space in the distributed cache system, and detects whether a response message fed back by the node corresponding to each preset space is received. When the response message fed back by the node corresponding to the preset space is not received, it is determined that the node corresponding to the preset space is an abnormal node, and the ENUM-DNS server collects the distributed information. The number of abnormal nodes in the cache system.
  • the operation and maintenance system is Sending alarm information, therefore, achieves consistency and availability of data, and improves the reliability of the ENUM-DNS server.
  • a fourth embodiment of the data management method of the present invention is provided based on any of the foregoing embodiments.
  • the data management method further includes:
  • step c when the data file of the node in the distributed cache system is faulty, the data file corresponding to the file directory is sequentially copied from the standby node to the node according to the file directory of the node.
  • each of the distributed cache systems has a backup system. Normally, the data stored in the standby node of the standby system is consistent with the data held by the distributed cache system. When the data of the abnormal node in the distributed cache system is restored, the data file corresponding to the file directory is sequentially copied to the abnormal node in the standby node of the standby system according to the file directory of the abnormal node, thereby The recovery of the data of the abnormal node is completed.
  • the preset space corresponding to each node in the distributed cache system is reset, and the ENUM-DNS server is re-initialized. After the initialization is completed, the data saved in the standby node of the backup system is sequentially copied to the node corresponding to each preset space in the distributed cache system.
  • the data file corresponding to the file directory is sequentially copied from the standby node to the abnormal node according to the file directory of the abnormal node, and the The recovery of the data of the abnormal node without copying all the data files, thus realizing the rapid recovery of the data, thereby improving the reliability of the ENUM-DNS server.
  • FIG. 3 is a schematic diagram of functional modules of a first embodiment of a data management apparatus according to the present invention.
  • the data management device includes:
  • the generating module 10 is configured to generate a data file for the received data every other first preset duration, set a file identifier corresponding to the data file, save the data file, and write the file identifier corresponding to the data file.
  • the first configuration file is configured to generate a data file for the received data every other first preset duration, set a file identifier corresponding to the data file, save the data file, and write the file identifier corresponding to the data file.
  • the operation and maintenance system or the SPN delivers the data of the overhead household to the ENUM-DNS (E.164 Number URI Mapping-Domain Name System) server, and the generation module 10 receives the data. After the data is received, the received data is periodically generated to generate a data file.
  • the first preset duration is set in advance, and the generating module 10 generates a data file for the data received within the first preset duration every time the first preset duration is separated, and Set the file identifier corresponding to the generated data file.
  • the generating module 10 generates a data file for the data received within the 2 minute period every 2 minutes, and sets a file identifier corresponding to the generated data file, for example, The file identifier corresponding to the generated data file is set in order from small to large.
  • the file identifier corresponding to the first data file generated is set to 1; when the second data file is generated, The file identifier corresponding to the generated second data file is set to 2; when the third data file is generated, the file identifier corresponding to the generated third data file is set to 3 or the like.
  • the first preset duration can be flexibly set according to actual conditions, and the file identifier corresponding to each data file generated can also be set according to other rules, and is not limited herein.
  • the file identifier corresponding to the generated data file is written into the first configuration file.
  • the processing module 20 is configured to write the data file into the distributed cache system, and write the file identifier corresponding to the data file successfully written into the second configuration file when the writing is successful;
  • the processing module 20 After the generating module 10 generates a data file each time and writes the file identifier corresponding to the data file into the foreground first configuration file, the processing module 20 writes the generated data file into the distributed cache system. And each time the data file is successfully written, the file identifier corresponding to the successfully written data file is written into the second configuration file.
  • the storage module 30 is configured to detect the file identifiers different from the first configuration file and the second configuration file every second preset duration, and identify different data files corresponding to the different files. Saved to the distributed cache system.
  • the storage module 30 periodically compares the file identifiers written in the first configuration file and the second configuration file, and detects whether different file identifiers exist in the first configuration file and the second configuration file.
  • the second preset duration is also set in advance. The storage module 30 detects whether there is a different file identifier in the first configuration file and the second configuration file when the second preset duration is separated, when the first configuration file and the first When different file identifiers exist in the second configuration file, the data files corresponding to the different file identifiers are written into the distributed cache system.
  • the file identifier in the first configuration file is file identifier 1 to file identifier 10
  • the second configuration file The file identifiers in the file are file identifier 1 to file identifier 5, and the different file identifiers in the first configuration file and the second configuration file are file identifier 6 to file identifier 10, that is, the file identifier 6 to the file. If the data file corresponding to the identifier 10 is not successfully written into the distributed cache system, the storage module 30 writes the data file corresponding to the file identifier 6 to the file identifier 10 into the distributed cache system.
  • the generating module 10 periodically generates a data file by receiving the data, and each time the data file is generated, setting a file identifier corresponding to the data file, and writing the file identifier into the first configuration file.
  • the processing module 20 writes the generated data file into the distributed cache system, and writes the file identifier corresponding to the successfully written data file into the second configuration file when the write is successful, and the storage module
  • the timer detects whether there is a different file identifier in the first configuration file and the second configuration file. If there is a different file identifier in the first configuration file and the second configuration file, the ENUM-DNS server is The data in the distributed cache system is inconsistent.
  • the storage module 30 saves the data files corresponding to the different file identifiers to the distributed cache system, because the data files in the ENUM-DNS server and the distributed cache system are not required. Contrast, greatly reducing the time to detect whether the ENUM-DNS server is consistent with the data in the distributed cache system, thus improving the Data protection ENUM-DNS server with distributed cache coherence system efficiency, improve the reliability of ENUM-DNS server.
  • a second embodiment of the data management apparatus of the present invention is proposed based on the first embodiment.
  • the generating module 10 is further configured to write the file identifier corresponding to the generated data file into the first configuration file. To update the file identifier saved in the first configuration file;
  • the processing module 20 is further configured to write the file identifier corresponding to the saved data file to the second configuration file to update the file identifier saved in the second configuration file; wherein when a certain data file is not saved On success, the data file is continuously written repeatedly until the write is successful, or the alarm is reported when the write fails;
  • the storage module 30 is further configured to detect whether the file identifier saved in the first configuration file is consistent with the file identifier saved in the second configuration file; and the file identifier saved in the first configuration file When the file identifiers saved in the second configuration file are inconsistent, the data files corresponding to the file identifiers of the difference between the file identifiers saved in the first configuration file and the file identifiers saved in the second configuration file are written. To the distributed cache system.
  • the file identifier is a file number added to the data file according to the receiving order
  • the generating module 10 writes the file identifier saved in the first configuration file by using the file identifier corresponding to the generated data file.
  • To update the file identifier saved in the first configuration file For example, when generating When the module 10 generates the data file for the first time, the file identifier corresponding to the first data file generated is set to 1, and the file identifier 1 corresponding to the generated data file is written into the first configuration file and saved.
  • the generating module 10 When the generating module 10 generates the data file for the second time, setting the file identifier corresponding to the generated second data file to 2, writing the file identifier 2 into the first configuration file, and replacing the saved file identifier 1 to update The file identifier saved in the first configuration file.
  • the file identifier corresponding to the generated third data file is set to 3, and the file identifier 3 is written into the first configuration file, and the saved file identifier 2 is replaced.
  • the file identifier saved in the first configuration file is updated.
  • the file identifier 1 corresponding to the successfully written first data file is written into the second configuration file and saved.
  • the file identifier 2 corresponding to the successfully written second data file is written into the second configuration file, and is replaced and saved.
  • the file identifier 1 is to update the file identifier saved in the second configuration file.
  • the file identifier 3 corresponding to the successfully saved third data file is written into the second configuration file, and the saved file is replaced.
  • the file identifier saved in the second configuration file is updated.
  • the processing module 20 does not write other unwritten data files, continuously writes the data file repeatedly until the writing is successful, or reports an alarm when the writing fails.
  • the storage module 30 periodically detects whether the file identifier saved in the first configuration file is consistent with the file identifier saved in the second configuration file, for example, detecting the saving in the first configuration file every second preset duration Whether the file identifier is consistent with the file identifier saved in the second configuration file. And deleting, when the file identifier saved in the first configuration file is different from the file identifier saved in the second configuration file, the file identifier saved in the second configuration file and the file saved in the first configuration file A data file corresponding to the file identifier of the difference between the identifiers is written into the distributed cache system.
  • the file identifier saved in the first configuration file is 10 and the file identifier saved in the second configuration file is 5, the file identifier saved in the first configuration file and the second configuration file are The file identifiers saved in the file are inconsistent, and the storage module 30 identifies the file identifiers stored in the second configuration file to a data file corresponding to the file identifier of the difference between the file identifiers 10 stored in the first configuration file, that is, the file.
  • the five data files corresponding to the identifier 6 to the file identifier 10 are written into the distributed cache system.
  • the generating module 10 updates the file identifier saved in the first configuration file according to the file identifier corresponding to the data file each time the data file is generated, and the processing module 20 succeeds in generating the data file.
  • the file identifier saved in the second configuration file is updated according to the file identifier corresponding to the data file, and the storage module 30 periodically detects the first configuration file and the second configuration file. If the file identifiers are consistent, if the first configuration file is inconsistent with the file identifier saved in the second configuration file, the data in the ENUM-DNS server is different from that in the distributed cache system.
  • the storage module 30 The data file corresponding to the file identifier of the difference between the file identifier saved in the second configuration file and the file identifier saved in the first configuration file is written into the distributed cache system, so only the comparison The first configuration file and the file identifier saved in the second configuration file reduce detection of ENUM-DNS server and distributed The data storage system is consistent in time, thereby improving the reliability of ENUM-DNS server.
  • the data management apparatus further includes:
  • the sending module 40 is configured to send a detection message to a node corresponding to each preset space in the distributed cache system every third preset duration, and detect whether the node corresponding to each preset space is received. Response message
  • the determining module 50 is configured to determine that the node corresponding to the preset space is an abnormal node when the response message fed back by the node corresponding to the preset space is not received;
  • the alarm module 60 is configured to count the number of abnormal nodes in the distributed cache system. If the number of the abnormal nodes is greater than a preset threshold, the alarm information is sent to the operation and maintenance system, and the function of receiving data is closed. The new data file is no longer generated every first preset interval until the system returns to normal.
  • a preset space corresponding to each node in the distributed cache system is preset, and a special space is set, for example, the special space is set to space 0, and the space 0 is used for
  • the file generated by the generating module 10 is saved, and the saved file is evenly loaded to the nodes corresponding to each of the preset spaces, thereby ensuring balanced storage and query data of each node.
  • the preset space corresponds to the nodes one by one, that is, each node is completely loaded with one query space.
  • the ENUM-DNS server When the ENUM-DNS server is initialized, a fixed record is written to each preset space, and the fixed record is a fixed record corresponding to the initialization of the ENUM-DNS server, and the form is not limited herein. system.
  • the write record When the write record is repeatedly written, as long as the key value of the record repeatedly written is the same as the key value of the record previously written, the record repeatedly written overwrites the previously written record.
  • the write fails the ENUM-DNS server is not allowed to start, and the ENUM-DNS server fails to initialize; when the write is successful, the ENUM-DNS server is successfully initialized.
  • the sending module 40 After the ENUM-DNS server is successfully initialized, the sending module 40 sends a detection message to each node corresponding to the preset space every third preset duration.
  • the sending module 40 corresponds to each preset space every 5 minutes.
  • the node sends a detection message. If the node corresponding to the preset space is a normal node, the node corresponding to the preset space feeds back the response message to the sending module 40 when receiving the detection message; if the node corresponding to the preset space is an abnormal node, the response cannot be fed back. The message is sent to the sending module 40.
  • the sending module 40 After sending the detection message to the node corresponding to each preset space, the sending module 40 detects whether the response message fed back by the node corresponding to each preset space is received, and if the response message of the node corresponding to the preset space is not received, The determining module 50 determines that the node corresponding to the preset space is an abnormal node. If the response message fed back by the node corresponding to the preset space is received, the determining module 50 determines that the node corresponding to the preset space is a normal node.
  • the number of abnormal nodes in the distributed cache system is counted.
  • the number of the abnormal nodes is greater than the preset number threshold A, the data in the ENUM-DNS server and the data in the distributed cache system cannot be guaranteed.
  • the alarm module 60 sends an alarm message to the operation and maintenance system.
  • the number of the abnormal nodes is less than or equal to the preset number threshold A, although there is theoretical possibility that the data in the ENUM-DNS server is inconsistent with the data in the distributed cache system, in order to balance the availability of data It can be considered that the data in the ENUM-DNS server can still be guaranteed to be consistent with the data in the distributed cache system.
  • the value of the preset number threshold A is set according to the NRW copy policy adopted by the distributed cache system, for example, if the distributed cache system adopts a 3-2-2 copy policy, the preset number threshold The value of A is set to 3.
  • the alarm module 60 sends an alarm to the operation and maintenance system, and does not allow the operation and maintenance system or the SPN to continue to send the data of the overhead household to the ENUM-DNS server to ensure the data in the ENUM-DNS server and the data in the distributed cache system. Consistency. That is, the function of receiving data sent by the operation and maintenance system or the SPN is closed, and a new data file is no longer generated every interval of the first preset duration until the system returns to normal.
  • the determining module 50 is further configured to:
  • the update does not receive the received message. The number of times the response message is reported;
  • the determining module 50 updates the number of times that the response message fed back by the node corresponding to the preset space is not received, and determines whether the number of times is greater than or equal to the preset number of times. And determining that the node corresponding to the preset space is an abnormal node, when the number of the response messages fed back by the node corresponding to the preset space is not greater than or equal to the preset number of times.
  • the sending module 40 sends the detection message 3 times to the node corresponding to each preset space
  • the node corresponding to a preset space is not received 3 times
  • the feedback is not received.
  • the determining module 50 determines that the node corresponding to the preset space is an abnormal node.
  • the sending module 40 periodically sends a detection message to the node corresponding to each preset space in the distributed cache system, and detects whether a response message fed back by the node corresponding to each preset space is received.
  • the determining module 50 determines that the node corresponding to the preset space is an abnormal node, and counts the number of abnormal nodes in the distributed cache system, and when the abnormal node
  • the alarm module 60 sends an alarm message to the operation and maintenance system, thereby achieving both Data consistency and availability improve the reliability of ENUM-DNS servers.
  • a fourth embodiment of the data management apparatus of the present invention is provided based on any of the foregoing embodiments.
  • the data management apparatus further includes:
  • the data recovery module is configured to, when the data file of the node in the distributed cache system is faulty, sequentially copy the data file corresponding to the file directory from the standby node to the node according to the file directory of the node.
  • each of the distributed cache systems has a backup system. Normally, the data stored in the standby node of the standby system is consistent with the data held by the distributed cache system.
  • the data recovery module is based on the file directory of the abnormal node. The data node corresponding to the file directory is sequentially copied to the abnormal node in the standby node of the standby system, thereby completing the recovery of the data of the abnormal node.
  • the data recovery module sequentially copies all the data saved in the standby node of the standby system to the node corresponding to each preset space in the distributed cache system.
  • the data recovery module sequentially copies the data file corresponding to the file directory from the standby node to the abnormal node according to the file directory of the abnormal node.
  • the recovery of the data of the abnormal node is completed without copying all the data files, thereby realizing the rapid recovery of the data, thereby improving the reliability of the ENUM-DNS server.
  • the embodiment of the present invention further provides a non-transitory computer readable storage medium storing computer executable instructions, the computer executable instructions being configured to perform the method in any of the above embodiments.
  • the embodiment of the invention further provides a schematic structural diagram of an electronic device.
  • the electronic device includes:
  • At least one processor 50 which is exemplified by a processor 50 in FIG. 5; and a memory 51, may further include a communication interface 52 and a bus 53.
  • the processor 50, the communication interface 52, and the memory 51 can complete communication with each other through the bus 53.
  • Communication interface 52 can be used for information transmission.
  • Processor 50 can invoke logic instructions in memory 51 to perform the methods of the above-described embodiments.
  • logic instructions in the memory 51 described above may be implemented in the form of software functional units and sold or used as separate products, and may be stored in a computer readable storage medium.
  • the memory 51 is a computer readable storage medium, and can be used to store a software program, a computer executable program, and a program instruction/module corresponding to the method in the embodiment of the present invention.
  • the processor 50 executes the function application and the data processing by executing the software programs, the instructions, and the modules stored in the memory 51, that is, the data management method in the above method embodiments.
  • the memory 51 may include a storage program area and an storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created according to use of the terminal device, and the like. Further, the memory 51 may include a high speed random access memory, and may also include a nonvolatile memory.
  • the technical solution of the embodiment of the present invention may be embodied in the form of a software product, the computer software
  • the product is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present invention.
  • the foregoing storage medium may be a non-transitory storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and the like.
  • the present invention improves the efficiency of maintaining data consistency in an ENUM-DNS server and a distributed cache system.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un procédé de traitement de données volumineuses ENUM-DNS. Le procédé gère des données d'arrière-plan ENUM-DNS par l'intermédiaire d'un système de mémoire cache distribué. Le procédé comprend principalement : un procédé pour garantir la cohérence des données de premier plan-arrière-plan ENUM-DNS. Le procédé garantit la cohérence des données de premier plan-arrière-plan d'un système ENUM-DNS en détectant régulièrement des identificateurs de fichier dans un premier fichier de configuration et dans un second fichier de configuration, améliorant l'efficacité de maintenance pour la cohérence des données dans un serveur ENUM-DNS et le système de mémoire cache distribué.
PCT/CN2016/103529 2015-11-11 2016-10-27 Procédé et dispositif de gestion de données WO2017080362A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510765678.3A CN106682040A (zh) 2015-11-11 2015-11-11 数据管理方法及装置
CN201510765678.3 2015-11-11

Publications (1)

Publication Number Publication Date
WO2017080362A1 true WO2017080362A1 (fr) 2017-05-18

Family

ID=58695917

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/103529 WO2017080362A1 (fr) 2015-11-11 2016-10-27 Procédé et dispositif de gestion de données

Country Status (2)

Country Link
CN (1) CN106682040A (fr)
WO (1) WO2017080362A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112199332A (zh) * 2020-10-16 2021-01-08 杭州安森智能信息技术有限公司 分布式防爆巡检机器人集群智能化文件管理方法及系统
CN113568891A (zh) * 2021-08-05 2021-10-29 深圳墨世科技有限公司 分布式id生成方法、装置、服务器和可读存储介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109542338B (zh) * 2018-10-19 2022-02-18 郑州云海信息技术有限公司 一种实现分布式存储系统中节点信息一致性方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102404338A (zh) * 2011-12-13 2012-04-04 华为技术有限公司 一种文件同步方法和装置
CN102667772A (zh) * 2010-03-01 2012-09-12 株式会社日立制作所 文件级分级存储管理系统、方法和设备
CN103946844A (zh) * 2011-11-29 2014-07-23 国际商业机器公司 跨集群文件系统同步更新
US9015180B1 (en) * 2007-05-09 2015-04-21 Vmware, Inc. Repository including file identification

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101483576B (zh) * 2008-01-11 2012-07-25 上海博达数据通信有限公司 一种分布式系统中的版本管理实现方法
CN104750738B (zh) * 2013-12-30 2018-06-26 中国移动通信集团公司 一种更新数据信息的方法、数据节点、管理节点及系统
CN104182182B (zh) * 2014-08-18 2017-04-12 四川航天系统工程研究所 智能终端及智能终端的数据备份方法
CN104219327B (zh) * 2014-09-27 2017-05-10 上海瀚之友信息技术服务有限公司 一种分布式缓存系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9015180B1 (en) * 2007-05-09 2015-04-21 Vmware, Inc. Repository including file identification
CN102667772A (zh) * 2010-03-01 2012-09-12 株式会社日立制作所 文件级分级存储管理系统、方法和设备
CN103946844A (zh) * 2011-11-29 2014-07-23 国际商业机器公司 跨集群文件系统同步更新
CN102404338A (zh) * 2011-12-13 2012-04-04 华为技术有限公司 一种文件同步方法和装置

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112199332A (zh) * 2020-10-16 2021-01-08 杭州安森智能信息技术有限公司 分布式防爆巡检机器人集群智能化文件管理方法及系统
CN113568891A (zh) * 2021-08-05 2021-10-29 深圳墨世科技有限公司 分布式id生成方法、装置、服务器和可读存储介质
CN113568891B (zh) * 2021-08-05 2024-02-20 深圳墨世科技有限公司 分布式id生成方法、装置、服务器和可读存储介质

Also Published As

Publication number Publication date
CN106682040A (zh) 2017-05-17

Similar Documents

Publication Publication Date Title
US10298436B2 (en) Arbitration processing method after cluster brain split, quorum storage apparatus, and system
WO2017177941A1 (fr) Procédé et appareil de commutation de base de données active/en attente
US20150213100A1 (en) Data synchronization method and system
US11892922B2 (en) State management methods, methods for switching between master application server and backup application server, and electronic devices
CN107656705B (zh) 一种计算机存储介质和一种数据迁移方法、装置及系统
CN107506266B (zh) 一种数据恢复方法及系统
US8775859B2 (en) Method, apparatus and system for data disaster tolerance
WO2022036901A1 (fr) Procédé et appareil de mise en œuvre pour ensemble de reproductions de redis
US8612799B2 (en) Method and apparatus of backing up subversion repository
CN113905005A (zh) 即时通讯的客户端状态更新方法和装置
WO2017080362A1 (fr) Procédé et dispositif de gestion de données
CN111901176B (zh) 故障确定方法、装置、设备及存储介质
US11500812B2 (en) Intermediate file processing method, client, server, and system
US20180121531A1 (en) Data Updating Method, Device, and Related System
CN111342986B (zh) 分布式节点管理方法及装置、分布式系统、存储介质
EP3896571B1 (fr) Procédé, appareil et système de sauvegarde de données
US20160050113A1 (en) Methods for managing storage virtual machine configuration changes in a distributed storage system and devices thereof
CN108509296B (zh) 一种处理设备故障的方法和系统
CN110858168B (zh) 集群节点故障处理方法、装置及集群节点
CN112015593B (zh) Kubernetes集群的容灾备份方法及相关组件
CN106599006B (zh) 一种数据恢复方法和装置
CN111625402A (zh) 数据恢复方法、装置、电子设备及计算机可读存储介质
CN109104314B (zh) 一种修改日志配置文件的方法及装置
CN115314361B (zh) 一种服务器集群管理方法及其相关组件
WO2019178839A1 (fr) Procédé et dispositif de création d'un instantané de cohérence pour une application distribuée et un système distribué

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16863539

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16863539

Country of ref document: EP

Kind code of ref document: A1