WO2020094063A1 - 存储数据的方法、装置、存储介质及电子装置 - Google Patents

存储数据的方法、装置、存储介质及电子装置 Download PDF

Info

Publication number
WO2020094063A1
WO2020094063A1 PCT/CN2019/116019 CN2019116019W WO2020094063A1 WO 2020094063 A1 WO2020094063 A1 WO 2020094063A1 CN 2019116019 W CN2019116019 W CN 2019116019W WO 2020094063 A1 WO2020094063 A1 WO 2020094063A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
data set
success flag
stored
storage
Prior art date
Application number
PCT/CN2019/116019
Other languages
English (en)
French (fr)
Inventor
魏涛
周明勇
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2020094063A1 publication Critical patent/WO2020094063A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/70—Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer
    • G06F21/78—Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure storage of data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers

Definitions

  • the present application relates to the communication field, for example, to a method, device, storage medium, and electronic device for storing data.
  • Figure 1 shows the data collection, data transmission, data processing, and data storage functions in a distributed environment.
  • the upstream data collection node (referred to as the collection node for short) first passes a certain protocol (such as a serial number stop response protocol, a general packet radio service tunnel protocol (General Packet Radio Service Tuningelling Protocol, GTP), a routing representative (Diameter) protocol, etc. ) Collect data from the data generator in real time, and save the collected data to disk (this part of the data is called the original data), and then forward the collected data to the downstream data processing node (referred to as the processing node).
  • a certain protocol such as a serial number stop response protocol, a general packet radio service tunnel protocol (General Packet Radio Service Tuningelling Protocol, GTP), a routing representative (Diameter) protocol, etc.
  • GTP General Packet Radio Service Tuningelling Protocol
  • Diameter Diameter
  • the collection node After the collection node confirms that the processing node receives the data (receives the response message returned by the processing node), it replies to the data generator with the received response, and the processing node reprocesses the received data, and then processes the processed data (this Part of the data is called final data) and saved to disk.
  • the above is a normal process of data collection, processing and data storage in a distributed environment, but because of the data source itself and the network, protocol and other reasons, the processing node often receives duplicate data.
  • the processing node In order to ensure the uniqueness of the final data, the processing node is Before saving the final data, it is often necessary to determine whether the data is duplicated with the previously saved data. If it is repeated, there is no need to save it. In fact, this is also a very common method of data rearrangement. Data rearrangement takes into account factors such as the final data record may be large and the rearrangement period is long.
  • the practice is to rearrange the weight based on several key fields.
  • the data composed of the rearrangement key fields is the rearrangement data.
  • the rearrangement data and the final data also need to be saved on the disk, so that the rearrangement information can be read from the rearrangement data file to perform normal rearrangement work when restarting.
  • the processing node can ensure the integrity and uniqueness of the final data through the above data rearrangement mechanism, but if the processing node has sudden abnormalities (such as power failure, etc.), it is difficult to ensure that the data is complete and unique ( When abnormal, the relevant final data and rearrangement data in memory cannot be saved to disk in time).
  • the collection node is powered off
  • the data generator will resend the data that was sent before the collection node was powered down but did not receive a response according to the corresponding protocol (the data that has received the response indicates that the collection node has successfully forwarded it to the processing node), After the collection node receives the retransmitted data, it forwards it to the processing node, and the processing node can rearrange the processed final data according to the rearrangement data. Therefore, there is no data duplication and data loss when the simple collection node is powered off.
  • the processing node is powered off
  • the collection node needs to send the cached history to send the original data (the collection node needs to cache the most recent transmission data when sending the original data to the processing node) to resend to the processing node for processing and saving, but Because the final data saved by the processing node before the power failure and the rearrangement data cannot be guaranteed to be consistent, data duplication problems (the rearrangement data saved before the power failure is less than the final data) or data loss problems (preserved before the power failure) The rearrangement data is more than the final data).
  • the collection node needs to read the original data saved before the power failure and send it to the processing node for processing and preservation, but the final data and the rearrangement data saved by the processing node before the power failure cannot be consistent. Therefore, there is a problem of data duplication (rearranged data saved before power failure is less than the final data) or data loss (rearranged data saved before power failure is more than the final data).
  • Embodiments of the present application provide a method, device, storage medium, and electronic device for storing data, so as to at least solve the problem of duplication and loss of stored data after a processing node in the system in the related art is powered off and restarted.
  • a method for storing data including: detecting that a first device is switched from a power-off state to a power-on state at a first moment; when the first device is in the power-on state Next, it is detected whether a first storage success flag is set, wherein the first storage success flag is used to indicate that the first device has successfully stored the first data set to be stored in the first time period, and the first time The segment is earlier than the first moment; when it is detected that the first storage success flag is not set or the first storage success flag is abnormal, delete the first data set stored in the first time period and merge Store the first data set again.
  • a device for storing data including: a first detection module configured to detect that the first device is switched from a power-off state to a power-on state at a first moment; a second detection module , Set to detect whether a first storage success flag is set when the first device is in the power-on state, wherein the first storage success flag is used to indicate that the first device has successfully stored The first data set to be stored in the first time period, the first time period is earlier than the first moment; the storage module is set to detect that the first storage success flag or the first storage success flag is not set In case of abnormality, delete the first data set stored in the first time period and re-store the first data set.
  • a storage medium in which a computer program is stored, wherein the computer program is set to execute the steps in any one of the above method embodiments during runtime.
  • an electronic device including a memory and a processor, the memory stores a computer program, the processor is configured to run the computer program to perform any of the above The steps in the method embodiment.
  • Figure 1 is a flowchart of data collection, transmission, processing, and preservation in a distributed environment in the related art
  • FIG. 2 is a block diagram of a hardware structure of a mobile terminal according to a method for storing data provided by an embodiment of the present application;
  • FIG. 3 is a flowchart of storing data provided by an embodiment of the present application.
  • FIG. 4 is a flow chart of data collection and processing message interaction provided by an embodiment of the present application.
  • FIG. 5 is a flowchart of processing and rearrangement of original data by a processing node provided by an embodiment of the present application
  • FIG. 9 is a flowchart of processing node data storage provided by an embodiment of the present invention.
  • FIG. 10 is a structural block diagram of an apparatus for storing data provided by an embodiment of the present invention.
  • FIG. 2 is a block diagram of a hardware structure of a mobile terminal according to a method for storing data provided by an embodiment of the present application.
  • the mobile terminal 20 may include one or more (only one is shown in FIG. 2) processor 202 (the processor 202 may include, but is not limited to, a microprocessor (Microcontroller Unit, MCU) or programmable logic device (Processing device such as Field Programmable Gate Array, FPGA) and the memory 204 provided to store data.
  • MCU Microcontroller Unit
  • FPGA Field Programmable Gate Array
  • the above mobile terminal may further include a transmission device 206 for communication functions and an input and output device 208.
  • a transmission device 206 for communication functions may be further included in the above mobile terminal.
  • FIG. 2 is merely an illustration, and FIG. 2 does not limit the structure of the mobile terminal described above.
  • the mobile terminal 20 may also include more or fewer components than those shown in FIG. 2 or have a different configuration from that shown in FIG. 2.
  • the memory 204 may be configured to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for storing data in the embodiments of the present application, and the processor 202 runs the computer program stored in the memory 204, thereby The implementation of multiple functional applications and data processing is the realization of the above method.
  • the memory 204 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.
  • the memory 204 may include memories remotely provided with respect to the processor 202, and these remote memories may be connected to the mobile terminal 20 through a network. Examples of the above network include but are not limited to the Internet, intranet, local area network, mobile communication network, and combinations thereof.
  • the transmission device 206 is configured to receive or transmit data via a network.
  • the above specific example of the network may include a wireless network provided by a communication provider of the mobile terminal 20.
  • the transmission device 206 includes a network adapter (Network Interface Controller, NIC).
  • the NIC can be connected to other network devices through the base station to communicate with the Internet.
  • the transmission device 206 may be a radio frequency (RF) module, and the RF module is configured to communicate with the Internet in a wireless manner.
  • RF radio frequency
  • FIG. 3 is a flowchart of storing data provided by an embodiment of the present application. As shown in FIG. 3, the process includes the following steps:
  • Step S302 At the first moment, it is detected that the first device is switched from the power-off state to the power-on state.
  • Step S304 when the first device is in the power-on state, detecting whether a first storage success flag is set, wherein the first storage success flag is used to indicate that the first device has successfully stored
  • the first data set to be stored in the first time period is earlier than the first time.
  • Step S306 when it is detected that the first storage success flag is not set or the first storage success flag is abnormal, delete the first data set stored in the first time period and re-store the first Data collection.
  • the first device when the first device is detected from the power-down state to the power-up state, after confirming that the normal first storage success flag is not set in the first device, the first data set is re-stored by the first device. Therefore, the problem of duplication and loss of stored data after the processing node in the system is powered off and restarted can be solved, and the effect of ensuring that the data is not duplicated and not lost is achieved.
  • the execution body of the above steps may be a terminal, etc., but it is not limited thereto.
  • the method further includes: when it is detected that the first storage success flag has been set and the first storage success flag is correct, the first The device stores a second data set to be stored in a second time period, wherein the second time period is later than the first time.
  • the processing node after receiving the original data, the processing node (corresponding to the first device) starts data processing and rearrangement work.
  • Step 1) the processing node records the final data in memory and rearrangement data to disk, and marks it as a temporary file;
  • Step 2) Create and save a control information file, the content of the control information file can be a fixed value (corresponding to the first storage success flag);
  • Step 3 rename the saved final data and rearrangement data temporary files as official files.
  • the processing node is powered down.
  • the processing node After the processing node restarts, if it is found that the data storage process operation before power down is not completed, the processing node needs to complete the power down Before the data storage operation, to ensure the consistency of the final data and the rearrangement data (the final data and the rearrangement data correspond to the first data set), after that, the processing node repeats the above steps to continue to complete the processing and scheduling of the subsequent original data Heavy work and data storage process.
  • the processing node After the processing node restarts, when detecting that the control information file exists and the file content is normal (corresponding to the first storage success flag has been set and the first storage success flag is correct), the processing node repeats the above steps and continues to complete Processing of subsequent raw data, rearrangement work and data storage process (corresponding to the first device storing the second data set to be stored in the second time period).
  • the method when it is detected that the first storage success flag has been set and the first storage success flag is correct, the method further includes: waiting for the second period of time for the first device to store Before the stored second data set, the first device deletes the first storage success flag; after the first device stores the second data set to be stored in the second time period, the method further includes: The first device sets the first storage success flag.
  • the data storage process further includes: after step 3) of the data storage process of the foregoing embodiment: step 4) deleting the control information file (corresponding to the first storage success flag). That is, the control information file is deleted in the last step of the data storage process for the final data and the rearrangement data within the first time period.
  • the processing node After completing the data storage process in the first time period, the processing node continues to process the remaining original data and starts the data storage process in the second time period.
  • the data storage process in the second time period is the same as the data storage process in the first time period.
  • the first storage success flag may not be deleted in the data storage process, but different storage success flags are set in the first time period and the second time period, respectively.
  • the processing node (corresponding to the first device) sets the first storage success flag in the data storage process of the final data and the rearrangement data in the first time period, and sets it in the second time period Is the second storage success flag.
  • the first storage success flag and the second storage success flag are two different types of flags.
  • the first control information file is created within the first time period
  • the content in the first control information file may be 123
  • the second control information file is created within the second time period
  • the content in the second control information file is 456 .
  • the content in the control information file is not unique here, and may be any content. 123 and 456 only represent that the first control information file and the second control information file are two different files.
  • the method before detecting that the first device switches from the power-off state to the power-on state at the first moment, the method further includes: the first device acquiring the first processing data set and the first keyword data set , Where the first data set includes a first processed data set and a first keyword data set, the first processed data in the first processed data set and the first keyword data in the first keyword data set One-to-one correspondence, the first keyword data is used to determine whether the first processing data is stored in the first device; the first device stores the first to be stored in the first time period A data set, wherein, during the process of storing the first data set to be stored in the first time period by the first device or the first device successfully stores the to-be-stored data in the first time period After the first data set, the first device switches from the power-on state to the power-down state.
  • the processing node obtains the original data from the collection node, and processes the original data to obtain the final data. Before saving the final data, the processing node often needs to determine whether the data and the previously saved data are duplicated. If the data is duplicated, there is no need to save it again.
  • the processing node determines whether to store the final data by checking whether the first device has the rearrangement data corresponding to the final data, and if there is, the final data is no longer stored , If not, store the final data. Therefore, the rearrangement data is to ensure that the final data stored by the processing node is unique.
  • the first data set stored by the processing node includes final data (corresponding to the first processing data set) and rearrangement data (corresponding to the first keyword data set).
  • the final data and rearrangement data are one-to-one correspondence.
  • the processing node is powered off, the stored final data may be duplicated or lost.
  • the rearrangement data is stored but the final data is not saved.
  • the processing node is powered off.
  • the processing node is restarted, If it is found that there is rearrangement data, then the processing node will no longer save the final data corresponding to the rearrangement data, and the final data will be lost.
  • Another situation is that the final data is stored but the rearrangement data is not stored.
  • the processing node is powered off. After the processing node is restarted, it is found that there is no rearrangement data corresponding to the final data. After receiving the final data, since there is no corresponding rearrangement data, the final data will be stored again, and the final data will be repeated at this time.
  • the first device acquiring the first processing data set and the first keyword data set includes: the first device receiving the original data sent by the second device; the first device The data is processed to obtain the first processed data set.
  • the collection node (corresponding to the second device) collects the original data in the data generator and sends it to the processing node (corresponding to the first device).
  • the processing node processes the original data to obtain the final data (corresponding to the first device) One processing data).
  • the method includes: the second device stores the original data sent to the first device.
  • the collection node (corresponding to the second device) records the raw data that has been recently sent.
  • Figure 4 is a flow chart of data collection and processing message interaction
  • Step 4010 The acquisition node (corresponding to the second device) collects the original data from the data generator in real time, writes the original data to the disk, and sends the original data to the processing node (corresponding to the first device). Record the original data that has been sent recently (if the processing node is powered down and restarted, the collection node needs to resend this part of the original data to the processing node).
  • Step 4020 After receiving the original data of the collection node, the processing node replies to the collection node with a response message, and then starts subsequent data processing and rearrangement work (see FIG. 5).
  • First perform step 5010 process the original data and form the final data (corresponding to the first processed data set), then perform step 5020: determine whether the rearranged data is based on the rearranged data of the final data (corresponding to the first keyword data set) Exist in the historical rearrangement data, if there is step 5030: do not save the final data record, if there is no step 5040: record the final data and rearrangement data to the memory.
  • Step 4030 The collection node returns a response message to the data generator after receiving the response message that the processing node has received the data.
  • Step 6010 Record the final data (corresponding to the first processed data) and rearrangement data (corresponding to the first key data) in the memory to the disk, and mark them as temporary files.
  • Step 6020 Create and save a control information file.
  • the content of the control information file may be a fixed value (corresponding to the first storage success flag).
  • Step 6030 Under the condition that the control information file exists and the first storage success flag is normal, the final data and the rearrangement data temporary file saved in step 6010 are renamed as official files.
  • Step 6040 Delete the control information file.
  • the first step in data recovery is to deal with abnormal node recovery. If the processing node finds that there was a power failure before startup and found that the data storage operation before power failure was not completed (a power failure occurred during the storage process), the processing node needs to complete the data storage operation before power failure to ensure the final Consistency of data files and rearranged data files.
  • the data recovery flowchart is shown in Figure 7. The process includes the following two branches:
  • step 7010 delete the temporary file in the data storage process to ensure the consistency of the final data and the rearrangement data. For the deleted final data and the rearrangement data It can be restored by subsequent retransmission of the original data.
  • step 7020 rename the temporary file
  • step 7030 delete the control information file, and continue to process the remaining original data
  • Store the final data and the rearrangement data corresponding to the remaining original data corresponding to storing the second data set to be stored in the second time period by the first device.
  • the second step of data recovery is to retransmit the original data.
  • the original data retransmission is the acquisition node (corresponding to the second device) to resend the original data to the processing node (corresponding to the first device), to prevent the processing node from losing data If the storage process is not completed and the final data is lost,
  • FIG. 8 is a flowchart of the original data retransmission provided by the embodiment of the present application.
  • the original data retransmission process includes: Step 8010: Only the processing node is powered off, then the collection node needs to send the saved history to send the original data to the processing node; Step 8020: If all nodes such as the collection node are powered off and restarted, the collection node After restarting, the original data file needs to be re-read and sent to the processing node.
  • FIG. 9 is a flowchart of data storage of a processing node (corresponding to a first device) provided by an embodiment of the present application.
  • the data storage process is as follows:
  • Step 9010 Use the Direct I / O (direct I / O) method (write the data in the memory directly to the disk and skip the file system cache of the operating system) to finalize the data in the memory (corresponding to the first processed data ) Stored to disk, the final temporary file of data.
  • Direct I / O direct I / O
  • Step 9020 Use the direct I / O method to store the rearrangement data in the memory (corresponding to the first keyword data) to the disk, that is, a temporary file for rearrangement data.
  • Step 9030 Create a control information file, fill in the file content with a fixed value of "infoisok” (corresponding to the first storage success flag), and store it to the disk in a direct I / O mode.
  • Step 9040 Rename the final temporary data file to the final data file.
  • Step 9050 Rename the temporary rearrangement data file to the rearrangement data file.
  • Step 9060 Delete the control information file.
  • data recovery has different recovery methods at different stages according to the data storage operation when the processing node is powered off. The following assumes that power loss has occurred at several different stages in FIG. 9 and gives four examples of data recovery.
  • the processing node is powered off before step 9010 or step 9020 in FIG. 9, so that the control information file (corresponding to the first storage success flag) does not exist after the restart (the data storage process has not yet reached step 9030 before power off), and the data is restored Proceed as follows:
  • FIG. 8 is a flowchart of the original data retransmission provided by the application.
  • the processing node processes the data normally after receiving the retransmitted original data of the collection node (including the data processing flow shown in FIG. 5 and the data storage flow shown in FIG. 9).
  • step 9020 or step 9030 of FIG. 9 the processing node is powered off, so that the control information file (corresponding to the first storage success flag) does not exist after restarting, and the data recovery steps are as follows:
  • FIG. 8 is a flowchart of the original data retransmission provided by the application.
  • the processing node processes the data normally after receiving the retransmitted original data of the collection node (including the data processing flow shown in FIG. 5 and the data storage flow shown in FIG. 9).
  • the processing node (corresponding to the first device) is powered off when the control information file is saved in step 9030 of FIG. 9, so that the control information file (corresponding to the first storage success flag) does not exist or the file content is incomplete after restarting, and the data recovery step as follows:
  • FIG. 8 is a flowchart of the original data retransmission provided by the application.
  • the processing node processes the data normally after receiving the retransmitted original data of the collection node (including the data processing flow shown in FIG. 5 and the data storage flow shown in FIG. 9).
  • step 9040 and step 9060 of FIG. 9 the processing node (corresponding to the first device) is powered off. After restarting, the content of the control information file is “infoisok” (corresponding to the first storage success flag) due to the existence of the control information file. ), Consistent with the values filled in the data storage process, the data recovery steps are as follows:
  • the method according to the above embodiments can be implemented by means of software plus a general hardware platform, and of course, can also be implemented by hardware.
  • the technical solution of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as Read-Only Memory (ROM) / Random Access Memory) , RAM), magnetic disks, and optical disks, including multiple instructions to enable a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to perform the method described in any embodiment of the present application.
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • magnetic disks magnetic disks
  • optical disks including multiple instructions to enable a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to perform the method described in any embodiment of the present application.
  • an apparatus for storing data is also provided.
  • the apparatus is used to implement the foregoing embodiments and implementation modes, and those that have already been described will not be repeated.
  • the term "module" may implement a combination of software and / or hardware that performs predetermined functions.
  • the devices described in the following embodiments may be implemented in software, implementation of hardware or a combination of software and hardware is also possible and conceived.
  • the device includes: a first detection module 1002 configured to detect that the first device is switched from a power-off state to Power-on state; the second detection module 1004 is configured to detect whether a first storage success flag is set when the first device is in the power-on state, wherein the first storage success flag is used to Indicating that the first device has successfully stored the first data set to be stored in the first time period, the first time period is earlier than the first moment; the deletion module 1006 is set to detect that the first data set is not set If a storage success flag or the first storage success flag is abnormal, delete the first data set stored in the first time period and re-store the first data set.
  • a first detection module 1002 configured to detect that the first device is switched from a power-off state to Power-on state
  • the second detection module 1004 is configured to detect whether a first storage success flag is set when the first device is in the power-on state, wherein the first storage success flag is used to Indicating that the first device has successfully stored the first data set to be stored in the first
  • the device further includes a storage module configured to detect that the first storage success flag has been set and the first storage success flag is correct after detecting whether the first storage success flag is set In the case of, the first device stores a second data set to be stored in a second time period, wherein the second time period is later than the first time.
  • the above-mentioned device is further configured to, when detecting that the first storage success flag has been set and the first storage success flag is correct, the first device stores the Before the second data set, the first device deletes the first storage success flag; after the first device stores the second data set to be stored in the second time period, the first device sets the first One store success sign.
  • the first device is configured to acquire the first processing data set and the first keyword data set before detecting that the first device switches from the power-down state to the power-up state at the first moment.
  • the first data set includes a first processed data set and a first keyword data set, and the first processed data in the first processed data set and the first keyword data in the first keyword data set are one by one
  • the first keyword data is used to determine whether the first processing data is stored in the first device; store the first data set to be stored in the first period of time, wherein During the first device storing the first data set to be stored in the first time period or after the first device successfully stores the first data set to be stored in the first time period , The first device switches from a power-on state to a power-off state.
  • the first device is configured to acquire the first processed data set and the first keyword data set by: receiving the original data sent by the second device; processing the original data to obtain The first processed data set.
  • the second device is configured to store the original data sent to the first device after the first device receives the original data sent by the second device.
  • the above-mentioned multiple modules can be implemented by software or hardware. For the latter, they can be implemented by the following methods, but not limited to this: the above-mentioned modules are all located in the same processor; or Any combination of forms is located in different processors.
  • An embodiment of the present application further provides a storage medium in which a computer program is stored, wherein the computer program is configured to execute any of the steps in the above method embodiments during runtime.
  • the above storage medium may be set to store a computer program for performing the following steps:
  • the above storage medium may include, but is not limited to, U disk, ROM, RAM, mobile hard disk, magnetic disk, or optical disk and other media that can store computer programs.
  • An embodiment of the present application further provides an electronic device, including a memory and a processor, where the computer program is stored in the memory, and the processor is configured to run the computer program to perform the steps in any one of the foregoing method embodiments.
  • the electronic device may further include a transmission device and an input-output device, wherein the transmission device is connected to the processor, and the input-output device is connected to the processor.
  • the above processor may be configured to perform the following steps through a computer program:
  • the above-mentioned multiple modules or multiple steps may be implemented by program code executable by the computing device, so that the above-mentioned multiple modules or multiple steps may be stored in the storage device and executed by the computing device, and In some cases, the steps shown or described may be performed in an order different from here, or the above-mentioned multiple modules or multiple steps are separately made into multiple integrated circuit modules, or the above-mentioned multiple modules or multiple Multiple modules or steps in each step are made into a single integrated circuit module to achieve. In this way, this application is not limited to any specific combination of hardware and software.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Retry When Errors Occur (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种存储数据的方法、装置、存储介质及电子装置，该方法包括：在第一时刻检测到第一装置从掉电状态切换到上电状态（S302）；在所述第一装置处于所述上电状态下的情况下，检测是否设置有第一存储成功标志，其中，所述第一存储成功标志用于指示所述第一装置已成功存储第一时间段内待存储的第一数据集合，所述第一时间段早于所述第一时刻（S304）；在检测出未设置所述第一存储成功标志或第一存储成功标志异常的情况下，删除在所述第一时间段内存储的所述第一数据集合并重新存储所述第一数据集合（S306）。

Description

存储数据的方法、装置、存储介质及电子装置
本申请要求在2018年11月06日提交中国专利局、申请号为201811315005.8的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请涉及通信领域，例如涉及一种存储数据的方法、装置、存储介质及电子装置。
背景技术
分布式环境下，往往对数据保存的完整性和一致性有着很严格的要求，比如不允许最终输出数据有重复记录、不允许有数据丢失等。
图1示出了分布式环境下的数据采集、数据传输、数据处理和数据保存功能。其中上游的数据采集节点(简称为采集节点)首先通过某种协议(比如流水号停等应答协议、通用分组无线业务隧道协议(General Packet Radio Service Tunnelling Protocol，GTP)、路由代表(Diameter)协议等)实时地从数据发生器采集数据，并将采集数据保存到磁盘(这部分数据称为原始数据)，同时再把采集数据转发到下游的数据处理节点(简称为处理节点)。采集节点确认处理节点接收到数据之后(接收到处理节点返回的响应消息)再向数据发生器回复数据已接收的响应，处理节点对接收到的数据进行再处理，然后再将处理后数据(这部分数据称为最终数据)保存到磁盘。
上面是分布式环境下一个正常的数据采集、处理和数据保存的过程，但因为数据源自身和网络、协议等原因处理节点往往会收到重复数据，为了保证最终数据的唯一性，处理节点在保存最终数据前往往需要判断该数据是否和之前保存的数据有重复，如果重复则不需要再做保存，实际上这也是一种很常用的数据排重方法。数据排重时考虑到最终数据记录可能很大、排重周期较长等因素，如果仅仅根据最终数据记录进行排重，对系统资源消耗很大，对系统性能也有较大影响，所以最理想的做法是根据几个关键字段来进行排重。排重关键字段组成的数据即排重数据，排重数据和最终数据一样也需要保存在磁盘上，以便于重启时从排重数据文件中读取排重信息进行正常的排重工作。
正常情况下处理节点是可以通过上述数据排重机制来保证最终数据的完整性以及唯一性，但如果处理节点有突发异常(比如掉电等)，数据就很难保证是否完整和唯一了(异常时内存中的相关最终数据和排重数据不能及时保存到磁盘)。下面我们来分析下上述这种分布式环境下节点出现掉电异常时数据重 复和数据丢失的一般处理情况及可能碰到的问题：
1、采集节点掉电
采集节点掉电重启后数据发生器会根据相应协议对采集节点掉电前发送但未收到响应的数据进行重发(已收到响应的数据则说明采集节点已经成功转发给处理节点了)，采集节点收到重发数据后转发给处理节点，处理节点将处理后的最终数据根据排重数据进行排重即可，所以单纯的采集节点掉电不存在数据重复和数据丢失问题。
2、处理节点掉电
因为处理节点内存中数据可能丢失，所以采集节点需要将缓存的历史发送原始数据(采集节点发送原始数据给处理节点时需要将最近的发送数据缓存起来)重发给处理节点进行处理和保存，但由于处理节点在掉电前保存的最终数据和排重数据不能保证一致，所以就会产生数据重复问题(掉电前保存的排重数据少于最终数据)或数据丢失问题(掉电前保存的排重数据多于最终数据)。
3、所有节点掉电
因为处理节点内存中数据可能丢失，所以采集节点需要读取掉电前保存的原始数据并发送给处理节点进行处理保存，但由于处理节点在掉电前保存的最终数据和排重数据不能保证一致，所以就产生数据重复问题(掉电前保存的排重数据少于最终数据)或数据丢失问题(掉电前保存的排重数据多于最终数据)。
针对相关技术中当系统中的处理节点掉电重启后，存储的数据的重复和丢失的问题，尚未提出有效的解决方案。
发明内容
本申请实施例提供了一种存储数据的方法、装置、存储介质及电子装置，以至少解决相关技术中系统中的处理节点掉电重启后，存储的数据的重复和丢失的问题。
根据本申请的一个实施例，提供了一种存储数据的方法，包括：在第一时刻检测到第一装置从掉电状态切换到上电状态；在所述第一装置处于所述上电状态下，检测是否设置有第一存储成功标志，其中，所述第一存储成功标志用于指示所述第一装置已成功存储第一时间段内待存储的第一数据集合，所述第一时间段早于所述第一时刻；在检测出未设置所述第一存储成功标志或第一存储成功标志异常的情况下，删除在所述第一时间段内存储的所述第一数据集合 并重新存储所述第一数据集合。
根据本申请的另一个实施例，提供了一种存储数据的装置，包括：第一检测模块，设置为在第一时刻检测到第一装置从掉电状态切换到上电状态；第二检测模块，设置为在所述第一装置处于所述上电状态下的情况下，检测是否设置有第一存储成功标志，其中，所述第一存储成功标志用于指示所述第一装置已成功存储第一时间段内待存储的第一数据集合，所述第一时间段早于所述第一时刻；存储模块，设置为在检测出未设置所述第一存储成功标志或第一存储成功标志异常的情况下，删除在所述第一时间段内存储的所述第一数据集合并重新存储所述第一数据集合。
根据本申请的又一个实施例，还提供了一种存储介质，所述存储介质中存储有计算机程序，其中，所述计算机程序被设置为运行时执行上述任一项方法实施例中的步骤。
根据本申请的又一个实施例，还提供了一种电子装置，包括存储器和处理器，所述存储器中存储有计算机程序，所述处理器被设置为运行所述计算机程序以执行上述任一项方法实施例中的步骤。
附图说明
图1是相关技术中的分布式环境下的数据采集、传输、处理和保存的流程图；
图2是本申请实施例提供的一种存储数据的方法的移动终端的硬件结构框图；
图3是本申请实施例提供的存储数据的流程图；
图4是本申请实施例提供的数据采集处理消息交互流程图；
图5是本申请实施例提供的处理节点对原始数据的处理和排重流程图；
图6是本申请实施例提供的数据存储流程图；
图7是本发明实施例提供的数据恢复流程图；
图8是本发明实施例提供的原始数据重发流程图；
图9是本发明实施例提供的处理节点数据存储的流程图；
图10是本发明实施例提供的存储数据的装置的结构框图。
具体实施方式
下文中将参考附图并结合实施例来说明本申请。
需要说明的是，本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。
本申请提供的方法实施例可以在移动终端、计算机终端或者类似的运算装置中执行。以运行在移动终端上为例，图2是本申请实施例提供的一种存储数据的方法的移动终端的硬件结构框图。如图2所示，移动终端20可以包括一个或多个(图2中仅示出一个)处理器202(处理器202可以包括但不限于微处理器(Microcontroller Unit，MCU)或可编程逻辑器件(Field Programmable Gate Array，FPGA)等的处理装置)和设置为存储数据的存储器204。在一实施例中，上述移动终端还可以包括用于通信功能的传输设备206以及输入输出设备208。本领域普通技术人员可以理解，图2所示的结构仅为示意，图2并不对上述移动终端的结构造成限定。例如，移动终端20还可包括比图2中所示更多或者更少的组件，或者具有与图2所示不同的配置。
存储器204可设置为存储计算机程序，例如，应用软件的软件程序以及模块，如本申请实施例中的存储数据的方法对应的计算机程序，处理器202通过运行存储在存储器204内的计算机程序，从而执行多种功能应用以及数据处理，即实现上述的方法。存储器204可包括高速随机存储器，还可包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中，存储器204可包括相对于处理器202远程设置的存储器，这些远程存储器可以通过网络连接至移动终端20。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
传输装置206设置为经由一个网络接收或者发送数据。上述的网络具体实例可包括移动终端20的通信供应商提供的无线网络。在一个实例中，传输装置206包括一个网络适配器(Network Interface Controller，NIC)，NIC可通过基站与其他网络设备相连从而可与互联网进行通讯。在一个实例中，传输装置206可以为射频(Radio Frequency，RF)模块，RF模块设置为通过无线方式与互联网进行通讯。
在本实施例中提供了一种运行于上述移动终端的存储数据的方法，图3是本申请实施例提供的存储数据的流程图，如图3所示，该流程包括如下步骤：
步骤S302，在第一时刻检测到第一装置从掉电状态切换到上电状态。
步骤S304，在所述第一装置处于所述上电状态下的情况下，检测是否设置有第一存储成功标志，其中，所述第一存储成功标志用于指示所述第一装置已 成功存储第一时间段内待存储的第一数据集合，所述第一时间段早于所述第一时刻。
步骤S306，在检测出未设置所述第一存储成功标志或第一存储成功标志异常的情况下，删除在所述第一时间段内存储的所述第一数据集合并重新存储所述第一数据集合。
通过上述步骤，由于当检测到第一装置从掉电状态到上电状态时，经过确认第一装置中未设置正常的第一存储成功标志后，通过第一装置重新存储第一数据集合。因此，可以解决在系统中的处理节点掉电重启后，存储的数据的重复和丢失的问题，达到保证数据不重复和不丢失的效果。
在一实施例中，上述步骤的执行主体可以为终端等，但不限于此。
在一个实施例中，在检测是否设置有第一存储成功标志之后，还包括：在检测出已设置所述第一存储成功标志且所述第一存储成功标志正确的情况下，所述第一装置存储第二时间段内待存储的第二数据集合，其中，所述第二时间段晚于所述第一时刻。
在本实施例中，处理节点(对应于第一装置)接收到原始数据后，开始数据处理和排重工作。在预定的时间段内或者处理一定数据量的原始数据后启动数据存储流程：步骤1)，处理节点将内存中的最终数据和排重数据记录到磁盘中，并标记为临时文件；步骤2)，新建并保存控制信息文件，控制信息文件内容可以为某固定值(对应与第一存储成功标志)；步骤3)，将保存的最终数据和排重数据临时文件重命名为正式文件。在该时间段内(对应于第一时间段)，处理节点掉电，处理节点在重新启动后，在发现掉电前的数据存储流程操作并未完成的情况下，处理节点需要先完成掉电前的数据存储操作，以保证最终数据和排重数据的一致性(最终数据和排重数据对应于第一数据集合)，之后，处理节点重复上述步骤，继续完成对后续原始数据的处理、排重工作和数据存储流程。处理节点在重新启动后，在检测到控制信息文件存在且文件内容正常(对应于已设置第一存储成功标志且所述第一存储成功标志正确)的情况下，处理节点重复上述步骤，继续完成对后续原始数据的处理、排重工作和数据存储流程(对应于第一装置存储第二时间段内待存储的第二数据集合)。
在一个实施例中，在检测出已设置所述第一存储成功标志且所述第一存储成功标志正确的情况下，所述方法还包括：在所述第一装置存储第二时间段内待存储的第二数据集合之前，所述第一装置删除所述第一存储成功标志；在所述第一装置存储第二时间段内待存储的第二数据集合之后，所述方法还包括：所述第一装置设置所述第一存储成功标志。在本实施例中，数据存储流程在上述实施例的数据存储流程的步骤3)之后还包括：步骤4)删除控制信息文件(对 应于第一存储成功标志)。也就是在对第一时间段内的最终数据和排重数据进行数据存储流程中的最后一步删除控制信息文件。在完成第一时间段内的数据存储流程后，处理节点继续处理剩余的原始数据并启动第二时间段内的数据存储流程。第二时间段内的数据存储流程与第一时间段内的数据存储流程相同。在本实施例中，在数据存储流程中也可以不删除第一存储成功标志，而在第一时间段内和第二时间段内分别设置不同的存储成功标志。在一实施例中，处理节点(对应于第一装置)在对第一时间段内的最终数据和排重数据进行数据存储流程中设置的是第一存储成功标志，在第二时间段内设置的是第二存储成功标志。第一存储成功标志和第二存储成功标志是不同的两种标志。例如，在第一时间段内创建第一控制信息文件，第一控制信息文件中的内容可以为123，在第二时间段内创建第二控制信息文件，第二控制信息文件中的内容是456。控制信息文件中的内容在此并不唯一，可以为任意内容，123和456仅代表第一控制信息文件和第二控制信息文件是不同的两个文件。
在一个实施例中，在第一时刻检测到第一装置从掉电状态切换到上电状态之前，所述方法还包括：所述第一装置获取第一处理数据集合与第一关键字数据集合，其中，第一数据集合包括第一处理数据集合与第一关键字数据集合，所述第一处理数据集合中的第一处理数据与所述第一关键字数据集合中的第一关键字数据一一对应，所述第一关键字数据用于判断所述第一装置中是否存储了所述第一处理数据；所述第一装置存储所述第一时间段内待存储的所述第一数据集合，其中，在所述第一装置存储所述第一时间段内待存储的所述第一数据集合的过程中或在所述第一装置成功存储所述第一时间段内待存储的所述第一数据集合之后，所述第一装置从上电状态切换到掉电状态。在本实施例中，处理节点从采集节点获取原始数据，对原始数据进行处理后得到最终数据。处理节点在保存最终数据前往往需要判断该数据和之前保存的数据是否有重复，如果重复则不需要再保存。在系统正常的情况下(处理节点没有掉电)，处理节点通过检查所述第一装置是否存在与该最终数据对应的排重数据决定是否存储该最终数据，若存在则不再存储该最终数据，若不存在则存储该最终数据。因此，排重数据是为了保证处理节点所存储的最终数据是唯一的。因此，处理节点(对应于第一装置)存储的第一数据集合中包括最终数据(对应于第一处理数据集合)和排重数据(对应于第一关键字数据集合)。其中，最终数据和排重数据是一一对应的。然而，当处理节点掉电的情况下，会出现存储的最终数据重复或者丢失的情况，例如，存储了排重数据却没保存最终数据，此时处理节点掉电，在处理节点重新启动后，发现存在排重数据，那么此时处理节点就不会再保存与该排重数据对应的最终数据，就会出现最终数据的丢失。另外一种情况是，存储了最终数据却没存储排重数据，此时处理节点掉电，在处理 节点重新启动后，发现没有与该最终数据对应的排重数据，那么处理节点在后续如果再接收到该最终数据，由于没有对应的排重数据，就会再存储一遍该最终数据，此时出现最终数据重复。
在一个实施例中，所述第一装置获取第一处理数据集合与第一关键字数据集合包括：所述第一装置接收到第二装置发送的原始数据；所述第一装置对所述原始数据进行处理，得到所述第一处理数据集合。在本实施例中，采集节点(对应于第二装置)在数据发生器中采集原始数据并发给处理节点(对应于第一装置)，处理节点对原始数据进行处理后得到最终数据(对应于第一处理数据)。
在一个实施例中，在所述第一装置接收到第二装置发送的原始数据之后，包括：所述第二装置存储发送给所述第一装置的所述原始数据。在本实施例中，采集节点(对应于第二装置)中记录最近已发送的原始数据。
下面结合具体实施例，对本申请进一步说明：
实施例一
数据采集处理流程(图4为数据采集处理消息交互流程图)包含如下步骤：
步骤4010：采集节点(对应于第二装置)从数据发生器实时采集原始数据，将原始数据写入到磁盘，并将原始数据发送给处理节点(对应于第一装置)，另外采集节点上也记录最近已发送的原始数据(如果处理节点发生掉电重启后，采集节点需要重发这部分原始数据给处理节点)。
步骤4020：处理节点收到采集节点的原始数据后，向采集节点回复响应消息，然后开始后续的数据处理和排重工作(如图5)。首先执行步骤5010：处理原始数据并形成最终数据(对应于第一处理数据集合)，然后执行步骤5020：根据最终数据的排重数据(对应于第一关键字数据集合)判断该排重数据是否存在于历史排重数据中，如果存在执行步骤5030：不保存该最终数据记录，如果不存在执行步骤5040：将最终数据和排重数据记录到内存中。
步骤4030：采集节点收到处理节点的数据已接收的响应消息后向数据发生器回复响应消息。
当需要产生最终数据文件时(对应于第一时间段)，需要启动数据存储流程。在数据存储流程引入了控制信息文件(对应于第一存储成功标志)的概念，控制信息文件是为后面的数据恢复流程所用。数据存储流程简图如图6，大致分为如下几步：
步骤6010：将内存中的最终数据(对应于第一处理数据)和排重数据(对应于第一关键字数据)记录到磁盘中，并标记为临时文件。
步骤6020：新建并保存控制信息文件，控制信息文件内容可以为某固定值(对应于第一存储成功标志)。
步骤6030：在控制信息文件存在且其中的第一存储成功标志正常的情况下，将步骤6010中保存的最终数据和排重数据临时文件重命名为正式文件。
步骤6040：删除控制信息文件。
数据恢复的第一步是处理节点异常恢复。如果处理节点在启动后发现之前有掉电现象并且发现掉电前的数据存储操作未完成(在存储过程中发生了掉电)，处理节点需要先完成掉电前的数据存储操作，以保证最终数据文件和排重数据文件的一致性。数据恢复流程图如图7，该流程包含以下两个分支：
1)控制信息文件不存在或控制信息文件内容异常，这时候执行步骤7010：删除数据存储过程中的临时文件，以保证最终数据和排重数据的一致性，对于删除的最终数据和排重数据可以通过后续的原始数据重发进行恢复。
2)控制信息文件存在且文件内容正常，继续完成掉电前存储操作，该存储操作包括：步骤7020：临时文件重命名，步骤7030：删除控制信息文件删除，并继续处理剩余的原始数据，并存储与剩余的原始数据对应的最终数据和排重数据(对应于通过所述第一装置存储第二时间段内待存储的第二数据集合)。
数据恢复的第二步就是原始数据重发，原始数据重发就是采集节点(对应于第二装置)将原始数据重新发送给处理节点(对应于第一装置)，防止处理节点掉电前因为数据储流程未完成而出现的最终数据丢失，图8为本申请实施例提供的原始数据重发流程图。原始数据重发流程包括：步骤8010：只有处理节点掉电，则采集节点需要将保存的历史发送原始数据重新发送给处理节点；步骤8020：如果采集节点等全部节点都掉电重启了，采集节点重启后就需要重新读取原始数据文件并发送给处理节点。
实施例二
图9为本申请实施例提供的处理节点(对应于第一装置)数据存储流程图。数据存储流程如下：
步骤9010：使用Direct I/O(直接I/O)方式(将内存中的数据直接写入到磁盘中，跳过操作系统的文件系统缓存)将内存中的最终数据(对应于第一处理数据)存储到磁盘，即最终数据临时文件。
步骤9020：使用Direct I/O方式将内存中的排重数据(对应于第一关键字数 据)存储到磁盘，即排重数据临时文件。
步骤9030：创建控制信息文件，文件内容填写为固定值“infoisok”(对应于第一存储成功标志)，并以Direct I/O方式存储到磁盘。
步骤9040：重命名最终数据临时文件为最终数据文件。
步骤9050：重命名排重数据临时文件为排重数据文件。
步骤9060：删除控制信息文件。
在一实施例中，数据恢复根据处理节点掉电时数据存储操作在不同阶段有不同的恢复方法，下面假设在图9的几个不同阶段发生了掉电，分别举四个数据恢复例子。
情况一
在图9的步骤9010或步骤9020之前处理节点掉电了，这样重启后控制信息文件(对应于第一存储成功标志)不存在(掉电前数据存储流程还没走到步骤9030)，数据恢复步骤如下：
1、如果存在最终数据临时文件(对应于第一处理数据集合)则删除最终数据临时文件。
2、采集节点(对应于第二装置)原始数据重发，图8为本申请实施提供的原始数据重发流程图。
3、处理节点(对应于第一装置)接收到采集节点的重发原始数据后正常处理数据(包括图5所示的数据处理流程和图9所示的数据存储流程)。
情况二
在图9的步骤9020或步骤9030之前处理节点掉电了，这样重启后控制信息文件(对应于第一存储成功标志)也不存在，数据恢复步骤如下：
1、删除最终数据临时文件(对应于第一处理数据集合)，如果存在排重数据临时文件(对应于第一关键字数据集合)则删除排重数据临时文件。
2、采集节点(对应于第二装置)原始数据重发，图8为本申请实施提供的原始数据重发流程图；
3、处理节点(对应于第一装置)接收到采集节点的重发原始数据后正常处理数据(包括图5所示的数据处理流程和图9所示的数据存储流程)。
情况三
在图9的步骤9030保存控制信息文件时处理节点(对应于第一装置)掉电了，这样重启后控制信息文件(对应于第一存储成功标志)不存在或文件内容不完整，数据恢复步骤如下：
1、删除最终数据临时文件(对应于第一处理数据集合)和排重数据临时文件(对应于第一关键字集合)。
2、采集节点原始数据重发，图8为本申请实施提供的原始数据重发流程图；
3、处理节点接收到采集节点的重发原始数据后正常处理数据(包括图5所示的数据处理流程和图9所示的数据存储流程)。
情况四
在图9的步骤9040与步骤9060之间处理节点(对应于第一装置)掉电了，这样重启后由于控制信息文件存在控制信息文件的其内容为“infoisok”(对应于第一存储成功标志)，与数据存储流程中的填写值一致，数据恢复步骤如下：
1、如果存在最终数据临时文件(对应于第一处理数据)，重命名最终数据临时文件为最终数据文件。
2、如果存在排重数据临时文件(对应于第一关键字数据)，重命名排重数据临时文件为排重数据文件。
3、删除控制信息文件(对应于第一存储成功标志)。
通过以上的实施方式的描述，本领域的技术人员可以了解到根据上述实施例的方法可借助软件加通用硬件平台的方式来实现，当然也可以通过硬件来实现。基于这样的理解，本申请的技术方案可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如只读存储器(Read-Only Memory，ROM)/随机存取存储器(Random Access Memory，RAM)、磁碟、光盘)中，包括多个指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本申请任意实施例所述的方法。
在本实施例中还提供了一种存储数据的装置，该装置用于实现上述实施例及实施方式，已经进行过说明的不再赘述。如以下所使用的，术语“模块”可以实现预定功能的软件和/或硬件的组合。尽管以下实施例所描述的装置可以以软件来实现，但是硬件，或者软件和硬件的组合的实现也是可能并被构想的。
图10是本申请实施例提供的存储数据的装置的结构框图，如图10所示， 该装置包括：第一检测模块1002，设置为在第一时刻检测到第一装置从掉电状态切换到上电状态；第二检测模块1004，设置为在所述第一装置处于所述上电状态下的情况下，检测是否设置有第一存储成功标志，其中，所述第一存储成功标志用于指示所述第一装置已成功存储第一时间段内待存储的第一数据集合，所述第一时间段早于所述第一时刻；删除模块1006，设置为在检测出未设置所述第一存储成功标志或第一存储成功标志异常的情况下，删除在所述第一时间段内存储的所述第一数据集合并重新存储所述第一数据集合。
在一个实施例中，所述装置还包括：存储模块，设置为在检测是否设置有第一存储成功标志之后，在检测出已设置所述第一存储成功标志且所述第一存储成功标志正确的情况下，所述第一装置存储第二时间段内待存储的第二数据集合，其中，所述第二时间段晚于所述第一时刻。
在一个实施例中，上述装置还设置为在检测出已设置所述第一存储成功标志且所述第一存储成功标志正确的情况下，所述第一装置存储第二时间段内待存储的第二数据集合之前，所述第一装置删除所述第一存储成功标志；在所述第一装置存储第二时间段内待存储的第二数据集合之后，所述第一装置设置所述第一存储成功标志。
在一个实施例中，所述第一装置设置为在第一时刻检测到第一装置从掉电状态切换到上电状态之前，获取第一处理数据集合与第一关键字数据集合，其中，所述第一数据集合包括第一处理数据集合与第一关键字数据集合，所述第一处理数据集合中的第一处理数据与所述第一关键字数据集合中的第一关键字数据一一对应，所述第一关键字数据用于判断所述第一装置中是否存储了所述第一处理数据；存储所述第一时间段内待存储的所述第一数据集合，其中，在所述第一装置存储所述第一时间段内待存储的所述第一数据集合的过程中或在所述第一装置成功存储所述第一时间段内待存储的所述第一数据集合之后，所述第一装置从上电状态切换到掉电状态。
在一个实施例中，所述第一装置是设置为通过如下方式获取第一处理数据集合与第一关键字数据集合：接收到第二装置发送的原始数据；对所述原始数据进行处理，得到所述第一处理数据集合。
在一个实施例中，所述第二装置设置为在所述第一装置接收到第二装置发送的原始数据之后，存储发送给所述第一装置的所述原始数据。
需要说明的是，上述多个模块是可以通过软件或硬件来实现的，对于后者，可以通过以下方式实现，但不限于此：上述模块均位于同一处理器中；或者，上述多个模块以任意组合的形式分别位于不同的处理器中。
本申请的实施例还提供了一种存储介质，该存储介质中存储有计算机程序，其中，该计算机程序被设置为运行时执行上述任一项方法实施例中的步骤。
在一实施例中，上述存储介质可以被设置为存储用于执行以下步骤的计算机程序：
S10，在第一时刻检测到第一装置从掉电状态切换到上电状态.
S20，在所述第一装置处于所述上电状态下的情况下，检测是否设置有第一存储成功标志，其中，所述第一存储成功标志用于指示所述第一装置已成功存储第一时间段内待存储的第一数据集合，所述第一时间段早于所述第一时刻；
S30，在检测出未设置所述第一存储成功标志的情况下，删除在所述第一时间段内存储的所述第一数据集合并重新存储所述第一数据集合。
在一实施例中，上述存储介质可以包括但不限于：U盘、ROM、RAM、移动硬盘、磁碟或者光盘等多种可以存储计算机程序的介质。
本申请的实施例还提供了一种电子装置，包括存储器和处理器，该存储器中存储有计算机程序，该处理器被设置为运行计算机程序以执行上述任一项方法实施例中的步骤。
在一实施例中，上述电子装置还可以包括传输设备以及输入输出设备，其中，该传输设备和上述处理器连接，该输入输出设备和上述处理器连接。
在一实施例中，上述处理器可以被设置为通过计算机程序执行以下步骤：
S10，在第一时刻检测到第一装置从掉电状态切换到上电状态.
S20，在所述第一装置处于所述上电状态下的情况下，检测是否设置有第一存储成功标志，其中，所述第一存储成功标志用于指示所述第一装置已成功存储第一时间段内待存储的第一数据集合，所述第一时间段早于所述第一时刻；
S30，在检测出未设置所述第一存储成功标志的情况下，删除在所述第一时间段内存储的所述第一数据集合并重新存储所述第一数据集合。
本实施例中的示例可以参考上述实施例及实施方式中所描述的示例，本实施例在此不再赘述。
显然，本领域的技术人员应该明白，上述的本申请的多个模块或多个步骤可以用通用的计算装置来实现，上述多个模块或多个步骤可以集中在单个的计算装置上，或者分布在多个计算装置所组成的网络上。在一实施例中，上述多个模块或多个步骤可以用计算装置可执行的程序代码来实现，从而，可以将上述多个模块或多个步骤存储在存储装置中由计算装置来执行，并且在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤，或者将上述多个模块 或多个步骤分别制作成多个集成电路模块，或者将上述多个模块或多个步骤中的多个模块或步骤制作成单个集成电路模块来实现。这样，本申请不限制于任何特定的硬件和软件结合。

Claims (10)

  1. 一种存储数据的方法，包括：
    第一装置在第一时刻检测到所述第一装置从掉电状态切换到上电状态；
    在所述第一装置处于所述上电状态下的情况下，所述第一装置检测是否设置有第一存储成功标志，其中，所述第一存储成功标志用于指示所述第一装置已成功存储第一时间段内待存储的第一数据集合，所述第一时间段早于所述第一时刻；
    在检测出未设置所述第一存储成功标志或所述第一存储成功标志异常的情况下，所述第一装置删除在所述第一时间段内存储的所述第一数据集合并重新存储所述第一数据集合。
  2. 根据权利要求1所述的方法，在所述第一装置检测是否设置有第一存储成功标志之后，还包括：
    在检测出已设置所述第一存储成功标志且所述第一存储成功标志正确的情况下，所述第一装置存储第二时间段内待存储的第二数据集合，其中，所述第二时间段晚于所述第一时刻。
  3. 根据权利要求2所述的方法，在检测出已设置所述第一存储成功标志且所述第一存储成功标志正确的情况下，且在所述第一装置存储第二时间段内待存储的第二数据集合之前，还包括：所述第一装置删除所述第一存储成功标志；
    在所述第一装置存储第二时间段内待存储的第二数据集合之后，还包括：所述第一装置设置所述第一存储成功标志。
  4. 根据权利要求1、2或3所述的方法，在所述第一装置在第一时刻检测到所述第一装置从掉电状态切换到上电状态之前，还包括：
    所述第一装置获取第一处理数据集合与第一关键字数据集合，其中，所述第一数据集合包括所述第一处理数据集合与所述第一关键字数据集合，所述第一处理数据集合中的第一处理数据与所述第一关键字数据集合中的第一关键字数据一一对应，所述第一关键字数据用于判断所述第一装置中是否存储了所述第一处理数据；
    所述第一装置存储所述第一时间段内待存储的所述第一数据集合，其中，在所述第一装置存储所述第一时间段内待存储的所述第一数据集合的过程中或在所述第一装置成功存储所述第一时间段内待存储的所述第一数据集合之后，所述第一装置从上电状态切换到掉电状态。
  5. 根据权利要求4所述的方法，其中，所述第一装置获取第一处理数据集合与第一关键字数据集合包括：
    所述第一装置接收到第二装置发送的原始数据；
    所述第一装置对所述原始数据进行处理，得到所述第一处理数据集合。
  6. 根据权利要求5所述的方法，在所述第一装置接收到第二装置发送的原始数据之后，包括：
    所述第二装置存储发送给所述第一装置的所述原始数据。
  7. 一种存储数据的装置，包括：
    第一检测模块，设置为在第一时刻检测到第一装置从掉电状态切换到上电状态；
    第二检测模块，设置为在所述第一装置处于所述上电状态下的情况下，检测是否设置有第一存储成功标志，其中，所述第一存储成功标志用于指示所述第一装置已成功存储第一时间段内待存储的第一数据集合，所述第一时间段早于所述第一时刻；
    删除模块，设置为在检测出未设置所述第一存储成功标志或所述第一存储成功标志异常的情况下，删除在所述第一时间段内存储的所述第一数据集合并重新存储所述第一数据集合。
  8. 根据权利要求7所述的装置，还包括：
    存储模块，设置为在检测是否设置有第一存储成功标志之后，并在检测出已设置所述第一存储成功标志且所述第一存储成功标志正确的情况下，通过所述第一装置存储第二时间段内待存储的第二数据集合，其中，所述第二时间段晚于所述第一时刻。
  9. 一种存储介质，存储有计算机程序，所述计算机程序被设置为运行时执行所述权利要求1至6任一项中所述的方法。
  10. 一种电子装置，包括存储器和处理器，所述存储器中存储有计算机程序，所述处理器被设置为运行所述计算机程序以执行所述权利要求1至6任一项中所述的方法。
PCT/CN2019/116019 2018-11-06 2019-11-06 存储数据的方法、装置、存储介质及电子装置 WO2020094063A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811315005.8A CN111143899A (zh) 2018-11-06 2018-11-06 存储数据的方法、装置、存储介质及电子装置
CN201811315005.8 2018-11-06

Publications (1)

Publication Number Publication Date
WO2020094063A1 true WO2020094063A1 (zh) 2020-05-14

Family

ID=70516546

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/116019 WO2020094063A1 (zh) 2018-11-06 2019-11-06 存储数据的方法、装置、存储介质及电子装置

Country Status (2)

Country Link
CN (1) CN111143899A (zh)
WO (1) WO2020094063A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113760587A (zh) * 2020-09-23 2021-12-07 北京沃东天骏信息技术有限公司 一种处理数据的方法和装置
CN115794842B (zh) * 2022-11-02 2024-04-05 北京明朝万达科技股份有限公司 数据处理方法、装置、电子设备及介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070073792A1 (en) * 2005-09-28 2007-03-29 Tony Nichols System and method for removing residual data from memory
CN101430705A (zh) * 2008-11-29 2009-05-13 中兴通讯股份有限公司 基于嵌入式数据库的存储方法和装置
US20100088745A1 (en) * 2008-10-06 2010-04-08 Fujitsu Limited Method for checking the integrity of large data items rapidly
CN105446664A (zh) * 2015-12-03 2016-03-30 深圳国微技术有限公司 一种对存储器进行文件存储方法和存储装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105468544B (zh) * 2014-09-11 2021-05-11 深圳市腾讯计算机系统有限公司 实现防掉电文件系统的方法、装置以及防掉电文件系统
CN104765628A (zh) * 2015-03-10 2015-07-08 惠州Tcl移动通信有限公司 一种移动终端更新触摸屏固件的方法及系统
CN105759679B (zh) * 2015-12-12 2019-07-23 重庆川仪自动化股份有限公司 掉电保护控制器及其掉电保持方法
CN106919474A (zh) * 2015-12-28 2017-07-04 中移(苏州)软件技术有限公司 一种缓存数据保护方法和装置
CN106569748B (zh) * 2016-10-27 2019-04-09 南方电网科学研究院有限责任公司 Flash文件系统的数据处理方法和装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070073792A1 (en) * 2005-09-28 2007-03-29 Tony Nichols System and method for removing residual data from memory
US20100088745A1 (en) * 2008-10-06 2010-04-08 Fujitsu Limited Method for checking the integrity of large data items rapidly
CN101430705A (zh) * 2008-11-29 2009-05-13 中兴通讯股份有限公司 基于嵌入式数据库的存储方法和装置
CN105446664A (zh) * 2015-12-03 2016-03-30 深圳国微技术有限公司 一种对存储器进行文件存储方法和存储装置

Also Published As

Publication number Publication date
CN111143899A (zh) 2020-05-12

Similar Documents

Publication Publication Date Title
WO2017107900A1 (zh) 一种虚拟机恢复的方法及虚拟机管理设备
US11397538B2 (en) Data migration method and apparatus
US20200341862A1 (en) Data backup method and apparatus
US7900006B2 (en) Maintaining checkpoints during backup of live system
WO2021226905A1 (zh) 一种数据存储方法、系统及存储介质
CN109783014B (zh) 一种存储数据的方法及装置
CN107623703B (zh) 全局事务标识gtid的同步方法、装置及系统
WO2020094063A1 (zh) 存储数据的方法、装置、存储介质及电子装置
EP3147797A1 (en) Data management method, node and system for database cluster
CN112965660B (zh) 一种双存储池信息反馈的方法、系统、设备及介质
CN109697140B (zh) 数据备份方法及装置、数据恢复方法及装置、存储介质
US7499987B2 (en) Deterministically electing an active node
CN102880530B (zh) 终端和终端备份及恢复方法
CN111858077A (zh) 一种存储系统中io请求日志的记录方法、装置及设备
KR101424568B1 (ko) 트랜잭션 재시작 가능한 클라이언트 장치와 데이터베이스 서버 및 방법
JP3323129B2 (ja) ファイル転送プロトコル方式、方法、および記録媒体
WO2013091162A1 (zh) 一种分布式存储数据恢复方法、装置及系统
CN113535482B (zh) 云备份链数据备份方法及装置、设备、可读介质
US20220129446A1 (en) Distributed Ledger Management Method, Distributed Ledger System, And Node
WO2024113534A1 (zh) 存储节点中存储资源的控制方法和装置，以及存储节点
WO2024113543A1 (zh) 数据处理方法、系统、装置、非易失性可读存储介质及电子设备
CN110647526B (zh) 批量数据处理方法、装置、计算机设备及存储介质
CN112966046B (zh) 数据同步方法和装置、电子设备和存储介质
JPH1040123A (ja) ジョブ管理方式と方法
CN113741928A (zh) 一种基于i2c的逻辑器件的固件升级方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19882403

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19882403

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 17.09.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19882403

Country of ref document: EP

Kind code of ref document: A1