CN109828862B - Method and device for replaying log - Google Patents
Method and device for replaying log Download PDFInfo
- Publication number
- CN109828862B CN109828862B CN201711180655.1A CN201711180655A CN109828862B CN 109828862 B CN109828862 B CN 109828862B CN 201711180655 A CN201711180655 A CN 201711180655A CN 109828862 B CN109828862 B CN 109828862B
- Authority
- CN
- China
- Prior art keywords
- data segment
- metadata
- log
- logs
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The application provides a method and a device for replaying logs, wherein the method comprises the following steps: determining a first data segment to be played back, wherein the first data segment comprises N logs, each log in the N logs comprises metadata and data, the metadata and the data of each log in the N logs are adjacent to each other on the first data segment, and N is a positive integer; determining a metadata set, the metadata set comprising metadata for each of the N logs; and playing back the metadata of each log in the N logs according to the metadata set. The method of the embodiment of the application is beneficial to shortening the playback time delay of the log by determining the metadata set under the condition that the write log is not affected.
Description
Technical Field
The present application relates to the field of communications, and more particularly, to a method and apparatus for playing back logs.
Background
When a server where data is located suddenly loses power, suddenly restarts or directly shuts down a database, and a session accessing the database is suddenly stopped (shutdown), so that after the server is restarted or a medium fails, the data cannot be recovered normally, at the moment, a log is used for sending a place, and the data is recovered to the moment of the failure of the server by using log playback, thereby ensuring the consistency of the data. And the playback time determines the time at which the system starts to operate normally.
In the prior art, under the condition that write performance is ensured by simultaneously issuing data and metadata in a log, the storage medium is read one by one during playback, so that playback time delay is increased along with the increase of the length of a data segment (segment), and the system cannot work normally for a long time due to overlong playback time.
Disclosure of Invention
The application provides a method for replaying a log, which is beneficial to shortening the replay time delay of the log by determining a metadata set under the condition that the writing of the log is not affected.
In a first aspect, a method of playing back a log is provided, the method comprising: determining a first data segment to be played back, wherein the first data segment comprises N logs, each log in the N logs comprises metadata and data, the metadata and the data of each log in the N logs are adjacent to each other on the first data segment, and N is a positive integer; determining a metadata set, the metadata set comprising metadata for each of the N logs; and playing back the metadata of each log in the N logs according to the metadata set.
According to the method for replaying the log, under the condition that the writing log is not affected, the metadata set is determined, so that the replaying time delay of the log can be shortened.
With reference to the first aspect, in certain possible implementations of the first aspect, the first data segment includes the metadata set.
According to the method for replaying the log, under the condition that the writing log is not affected, the metadata set in the data segment is determined, so that the replay time delay of the log is shortened.
With reference to the first aspect, in certain possible implementation manners of the first aspect, the determining a metadata set includes: the metadata set is obtained from a second data segment, the second data segment being different from the first data segment.
According to the method for replaying the log, under the condition that the writing log is not affected, the metadata set in the other data segment is determined, so that the replay time delay of the log is shortened.
With reference to the first aspect, in certain possible implementation manners of the first aspect, before the determining the metadata set, the method further includes: the metadata set is written to the first data segment upon determining that the log of the first data segment is full.
According to the method for replaying the log, the metadata set is added to the data section, so that the replay time delay of the log is shortened.
With reference to the first aspect, in certain possible implementation manners of the first aspect, before the determining the metadata set, the method further includes: upon determining that the log of the first data segment is full, the set of metadata is written to the second data segment.
According to the method for replaying the log, disclosed by the embodiment of the application, the metadata set is added in another data segment, so that the replay time delay of the log is shortened.
With reference to the first aspect, in some possible implementations of the first aspect, the metadata set is at an end of the first data segment. According to the method for replaying the log, the metadata set is added at the tail part of the data section, so that the replay time delay of the log is shortened.
With reference to the first aspect, in some possible implementation manners of the first aspect, the playing back, according to the metadata set, metadata of each log of the N logs includes: playing back the metadata of each log in the N logs; determining position information of the first data segment according to the metadata obtained by playback; and determining the first data segment according to the position information of the first data segment.
In a second aspect, there is provided an apparatus for playing back a log, the apparatus comprising: the first processing module is used for determining a first data segment to be played back, the first data segment comprises N logs, each log in the N logs comprises metadata and data, the metadata and the data of each log in the N logs are adjacent to each other in position on the first data segment, and N is a positive integer; a second processing module for determining a set of metadata, the set of metadata comprising metadata for each of the N logs; and the third processing module is used for playing back the metadata of each log in the N logs according to the metadata set.
With reference to the second aspect, in some possible implementations of the second aspect, the first data segment includes the metadata set.
With reference to the second aspect, in some possible implementations of the second aspect, the second processing module is specifically configured to: the metadata set is obtained from a second data segment, the second data segment being different from the first data segment.
With reference to the second aspect, in certain possible implementations of the second aspect, the apparatus further includes: and the fourth processing module is used for writing the metadata set into the first data segment when the first data segment is determined to be full in log writing.
With reference to the second aspect, in certain possible implementations of the second aspect, the apparatus further includes: and a fifth processing module, configured to write the metadata set to the second data segment when it is determined that the log of the first data segment is full.
With reference to the second aspect, in some possible implementations of the second aspect, the metadata set is at an end of the first data segment.
With reference to the second aspect, in some possible implementations of the second aspect, the third processing module is specifically configured to: playing back the metadata of each log in the N logs; determining position information of the first data segment according to the metadata obtained by playback; and determining the first data segment according to the position information of the first data segment.
In a third aspect, an apparatus for playing back a log is provided, the apparatus comprising a memory for storing instructions and a processor; the processor is configured to invoke instructions in the memory to perform the method as described in the first aspect and any of the possible implementations of the first aspect.
In a fourth aspect, there is provided a computer readable storage medium comprising instructions which, when run on a computer, cause a computer mechanism to perform the method of the first aspect or any one of the possible implementations of the first aspect.
In a fifth aspect, there is provided a computer program product comprising: computer program code which, when run by a computer, causes the computer to perform the method of the first aspect or any one of the possible implementations of the first aspect.
Drawings
Fig. 1 is a schematic diagram of a data segment structure according to an embodiment of the present application.
Fig. 2 is a schematic flow chart of a method of playing back a log of an embodiment of the application.
Fig. 3 is another schematic diagram of a data segment structure according to an embodiment of the present application.
Fig. 4 is a further schematic diagram of a data segment structure according to an embodiment of the present application.
Fig. 5 is another schematic flow chart diagram of a method of playing back a log of an embodiment of the present application.
Fig. 6 is a schematic block diagram of an apparatus for playback of a log of an embodiment of the present application.
Fig. 7 is another schematic block diagram of an apparatus for playing back logs of an embodiment of the present application.
Detailed Description
The technical scheme of the application will be described below with reference to the accompanying drawings.
The embodiment of the application is applicable to all scenes of nonvolatile memory log playback, including but not limited to storage systems, terminal equipment and the like.
The terminal device in the embodiments of the present application may refer to a user device, an access terminal, a subscriber unit, a subscriber station, a mobile station, a remote terminal, a mobile device, a user terminal, a wireless communication device, a user agent, or a user apparatus. The terminal device may also be a cellular telephone, a cordless telephone, a session initiation protocol (Session Initiation Protocol, SIP) phone, a wireless local loop (Wireless Local Loop, WLL) station, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a wearable device, a terminal device in a future 5G network or a terminal device in a future evolved public land mobile network (Public Land Mobile Network, PLMN), etc., as embodiments of the present application are not limited in this regard.
Fig. 1 is a schematic diagram of a data segment structure according to an embodiment of the present application, as shown in fig. 1, in a process of writing a log, data and metadata are issued together by a system, and when the log is played back, the log is played back piece by piece. The journals are stored in the format of a front-most data Segment (Segment) header, which in turn stores journals (logs), with the understanding that each journal includes the data (LogData) and metadata (LogMeta) of the journal.
Fig. 2 shows a schematic flow diagram of a method 100 of playing back logs according to an embodiment of the application, which method 100 may be performed by a processor as shown in fig. 1, which method 100 comprises:
s110, determining a first data segment to be played back, wherein the first data segment comprises N logs, each log in the N logs comprises metadata and data, the metadata and the data of each log in the N logs are adjacent in position on the first data segment, and N is a positive integer;
it should be appreciated that the first data segment may be the data segment in fig. 1.
It should also be appreciated that the metadata and data for each log may be concurrently issued by the storage system writing the data to the processor, which writes the metadata and data to one of the first data segment logs, where the metadata and data are located adjacent to one another in the first log.
S120, determining a metadata set, wherein the metadata set comprises metadata of each log in the N logs.
Optionally, the first data segment includes the metadata set.
Optionally, before the determining the metadata set, the method 100 further includes:
the metadata set is written to the first data segment upon determining that the log of the first data segment is full.
Specifically, the first data segment includes the set of metadata, and writing the set of metadata to a reserved location of the first data segment when the processor determines that the log of the first data segment is full may determine the set of metadata at the reserved location by determining the reserved location of the set of metadata.
Optionally, the metadata set is at the end of the first data segment.
It should be appreciated that the metadata set may be at the end of the first data segment or any other location of the first data segment, and the application is not limited thereto.
FIG. 3 is a schematic diagram of another data segment structure according to an embodiment of the present application, as shown in FIG. 3, where a metadata set (Meta Blocks) space is reserved at the end of each data segment, where the metadata set is used to store all metadata of the data segment, and the metadata set may issue a nonvolatile memory with the last log.
It should be understood that the size of the metadata set may be calculated according to the size of the data segment, the size of the data segment header, the size of the maximum log, and the size of the log metadata header, and may be otherwise derived, and the present application is not limited thereto.
It should also be appreciated that the metadata collection in the data segment structure diagram shown in fig. 3 is not limited to the end of the data segment, as long as it is in a fast-reading position.
Optionally, the determining the metadata set includes:
the metadata set is obtained from a second data segment, the second data segment being different from the first data segment.
Optionally, before determining the metadata set, the method 100 further comprises:
upon determining that the log of the first data segment is full, the set of metadata is written to the second data segment.
In particular, the second data segment includes the set of metadata, and metadata for the first data segment may be written to the second data segment when the processor determines that the first data segment is full.
It should be appreciated that before writing the metadata set to the second data segment, an index relationship between the first data segment and the second data segment may be established first, and when the log of the first data segment is full, metadata of N logs in the first data segment may be written to the corresponding second data segment.
It should also be appreciated that the set of metadata may be at any location of the second piece of data.
Fig. 4 is a schematic diagram showing a data Segment structure in one embodiment of the present application, as shown in fig. 4, a system reserves M data segments (segments) during initialization, where the M data segments are used for storing metadata, and when a certain data Segment (other than the M data segments) is full, the metadata set of the data Segment is issued to one data Segment of the M data segments, and when playback is performed, the corresponding metadata set information is read, and metadata information of the data Segment is restored.
For example, when initializing, 3 data segments are used for writing logs, namely a data segment1, a data segment 2 and a data segment 3, 1 data segment (a data segment 4) is reserved for storing all metadata of the data segment1, the data segment 2 and the data segment 3, when one of the data segment1, the data segment 2 and the data segment 3 is full, the system issues the metadata of the data segment to the data segment 4, and when playing back, metadata information corresponding to the data segment 4 is read, and metadata information of the data segment is restored.
It should be appreciated that the metadata collection in the data segment structure diagram shown in fig. 4 is not limited to being stored in separate data segments, as long as it is in a fast-reading location.
S130, reconstructing the metadata of each log in the N logs according to the metadata set.
Specifically, when the processor performs log playback, first determining a first data segment that needs to perform log playback, where the first data segment includes N logs, where each log in the N logs includes metadata and data, where the metadata and data of each log in the N logs are issued simultaneously by the system, N is a positive integer, and determining, according to the first data segment, a metadata set, where the metadata set includes metadata of each log in the N logs, where, for example, as shown in fig. 3, the metadata set of the first data segment may be at the tail of the first data segment; as another example, as shown in fig. 4, the metadata of the first data segment may be in another data segment. The processor may reconstruct the metadata of each of the N logs from the set of metadata.
Optionally, the playing back the metadata of each log in the N logs according to the metadata set includes:
playing back the metadata of each log in the N logs;
determining position information of the first data segment according to the metadata obtained by playback;
and determining the first data segment according to the position information of the first data segment.
Specifically, the metadata in the metadata set includes the location information (possibly including other information) of the first data segment, and the metadata of each log in the N logs is played back, so that the location information corresponding to the data in the first data segment in the storage disk can be obtained.
It should be understood that the location information in the storage disc corresponding to the first data segment may be the location of a management unit in the storage disc, where the data in the first data segment is stored.
According to the method for replaying the log, under the condition that the writing log is not affected, the metadata set is determined, so that the replaying time delay of the log can be shortened.
Fig. 5 shows a schematic flow chart of a method 200 of playing back logs according to an embodiment of the application, as shown in fig. 5, the method 200 may be performed by a processor, the method 200 comprising:
s210, starting log playback when the fault is recovered, restarted or the node is in a primary state;
s220, acquiring all to-be-played-back data segment identification (segment ID) sets;
s230, acquiring a minimum unreplayed data Segment identifier from the data Segment identifier set (for example, the minimum unreplayed data Segment identifier corresponds to Segment1, namely, data Segment 1);
s240, judging that the release of the data segment1 exists, and if the release of the data segment1 does not exist, ending playback; if yes, performing log playback of the data segment 1;
s250, judging whether the data segment1 is full (the metadata set is written), if so, performing S260 to read the metadata set of the data segment1 for metadata playback, and if not, performing S280 to play back the log piece by piece.
It will be appreciated that for the data segment structure shown in fig. 3, it may be determined whether the data segment1 is written with a set of metadata, and for the data segment structure shown in fig. 4, it may be determined whether a second data segment (different from the data segment 1) is written with a set of metadata.
S260, reading the reserved metadata set position of the data segment1 to obtain metadata;
s270, reconstructing metadata information of the data segment 1;
s280, reading logs one by one, obtaining metadata and reconstructing the metadata;
s290, the log playback ends.
According to the method for replaying the log, provided by the embodiment of the application, under the condition that the time delay of writing the log is not affected, the time delay of replaying the log is shortened by determining the metadata set during replaying the log.
The method of playing back the log according to the embodiment of the present application is described in detail above with reference to fig. 1 to 5, and the apparatus for playing back the log according to the embodiment of the present application is described in detail below with reference to fig. 6 and 7.
Fig. 6 shows a schematic block diagram of an apparatus 300 for playing back a log according to an embodiment of the application, as shown in fig. 6, the apparatus 300 comprising:
a first processing module 310, configured to determine a first data segment to be played back, where the first data segment includes N logs, each log of the N logs includes metadata and data, the metadata and the data of each log of the N logs are adjacent to each other in a position on the first data segment, and N is a positive integer;
a second processing module 320, configured to determine a metadata set, where the metadata set includes metadata of each log of the N logs;
and a third processing module 330, configured to play back the metadata of each of the N logs according to the metadata set.
Optionally, the first data segment includes the metadata set.
Optionally, the second processing module 320 is specifically configured to:
the metadata set is obtained from a second data segment, the second data segment being different from the first data segment.
Optionally, the apparatus 300 further comprises:
and the fourth processing module is used for writing the metadata set into the first data segment when the first data segment is determined to be full in log writing.
Optionally, the apparatus 300 further comprises:
and a fifth processing module, configured to write the metadata set to the second data segment when it is determined that the log of the first data segment is full.
Optionally, the metadata set is at the end of the first data segment.
Optionally, the third processing module 330 is specifically configured to:
playing back the metadata of each log in the N logs;
determining position information of the first data segment according to the metadata obtained by playback;
and determining the first data segment according to the position information of the first data segment.
According to the device for replaying the log, provided by the embodiment of the application, under the condition that the writing of the log is not affected, the metadata set is determined, so that the replay time delay of the log is shortened.
Fig. 7 shows a schematic block diagram of an apparatus 400 for playing back a log, the apparatus 400 comprising:
a memory 410 for storing instructions;
a processor 420 for invoking instructions in the memory 410 to perform operations in the corresponding method of the embodiment of the application.
Embodiments of the present application also provide a computer program product for use in an apparatus for playing back logs, the computer program product comprising a series of instructions which, when executed, enable the apparatus to perform operations corresponding to the apparatus of the above-described method.
In the embodiments of the present application, it should be noted that the above-described method embodiments of the present application may be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method embodiments may be implemented by integrated logic circuits of hardware in a processor or instructions in software form. The processor may be a general purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.
It will be appreciated that the memory in embodiments of the application may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM) which acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (Double Data Rate SDRAM), enhanced SDRAM (ESDRAM), synchronous DRAM (SLDRAM), and Direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.
It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application.
In addition, the terms "system" and "network" are often used interchangeably herein. The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.
It should be understood that in embodiments of the present application, "B corresponding to a" means that B is associated with a, from which B may be determined. It should also be understood that determining B from a does not mean determining B from a alone, but may also determine B from a and/or other information.
In the above embodiments, the implementation may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product may include one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, fiber optic, digital Subscriber (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means from one website, computer, server, or data center. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic Disk), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (8)
1. A method of playing back a log, comprising:
determining a first data segment to be played back, wherein the first data segment comprises N logs, each log in the N logs comprises metadata and data, the metadata of each log in the N logs are adjacent to the position of the data on the first data segment, and N is a positive integer;
determining a metadata set from the first data segment, the metadata set including metadata of each of the N logs, the metadata set being written to a reserved location of the first data segment when it is determined that the log of the first data segment is full, or determining the metadata set from a second data segment, the metadata set being written to the second data segment when it is determined that the log of the first data segment is full, the second data segment being different from the first data segment;
and playing back the metadata of each log in the N logs according to the metadata set.
2. The method of claim 1, wherein the reserved location is at a tail of the first data segment.
3. The method according to claim 1 or 2, wherein playing back the metadata of each of the N logs according to the metadata set includes:
playing back the metadata of each log in the N logs;
determining the position information of the first data segment according to the metadata obtained by playback;
and determining the first data segment according to the position information of the first data segment.
4. An apparatus for playing back a log, comprising:
the first processing module is used for determining a first data segment to be played back, the first data segment comprises N logs, each log in the N logs comprises metadata and data, the metadata and the data of each log in the N logs are adjacent to each other on the first data segment, and N is a positive integer;
a second processing module, configured to determine a metadata set from the first data segment, where the metadata set includes metadata of each log of the N logs, and the metadata set is written to a reserved location of the first data segment by the fourth processing module when it is determined that the log of the first data segment is full, or determine the metadata set from a second data segment, where the metadata set is written to a second data segment by the fifth processing module when it is determined that the log of the first data segment is full, where the second data segment is different from the first data segment;
and the third processing module is used for playing back the metadata of each log in the N logs according to the metadata set.
5. The apparatus of claim 4, wherein the reserved location is at the end of the first data segment.
6. The apparatus according to claim 4 or 5, wherein the third processing module is specifically configured to:
playing back the metadata of each log in the N logs;
determining the position information of the first data segment according to the metadata obtained by playback;
and determining the first data segment according to the position information of the first data segment.
7. An apparatus for playing back a log, comprising:
a memory for storing instructions;
a processor for invoking instructions in said memory to perform the method of any of the preceding claims 1-3.
8. A computer readable storage medium comprising instructions which, when run on a computer, cause the computer to perform the method of any of the preceding claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711180655.1A CN109828862B (en) | 2017-11-23 | 2017-11-23 | Method and device for replaying log |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711180655.1A CN109828862B (en) | 2017-11-23 | 2017-11-23 | Method and device for replaying log |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109828862A CN109828862A (en) | 2019-05-31 |
| CN109828862B true CN109828862B (en) | 2023-08-22 |
Family
ID=66858438
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711180655.1A Active CN109828862B (en) | 2017-11-23 | 2017-11-23 | Method and device for replaying log |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109828862B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102375853A (en) * | 2010-08-24 | 2012-03-14 | 中国移动通信集团公司 | Distributed database system, method for building index therein and query method |
| CN103077222A (en) * | 2012-12-31 | 2013-05-01 | 中国科学院计算技术研究所 | Method and system for ensuring consistence of distributed metadata in cluster file system |
| CN103516549A (en) * | 2013-09-27 | 2014-01-15 | 浪潮电子信息产业股份有限公司 | File system metadata log mechanism based on shared object storage |
| CN103902479A (en) * | 2014-03-27 | 2014-07-02 | 浪潮电子信息产业股份有限公司 | Quick reconstruction mechanism for metadata cache on basis of metadata log |
| CN104050268A (en) * | 2014-06-23 | 2014-09-17 | 西北工业大学 | Continuous data protection and recovery method with log space adjustable online |
| CN104301360A (en) * | 2013-07-19 | 2015-01-21 | 阿里巴巴集团控股有限公司 | Method, log server and system for recording log data |
| CN105138284A (en) * | 2015-08-11 | 2015-12-09 | 上海交通大学 | Virtual machine disk mirror image synchronization operation optimization system and method |
| CN105574104A (en) * | 2015-12-11 | 2016-05-11 | 上海爱数信息技术股份有限公司 | LogStructure storage system based on ObjectStore and data writing method thereof |
| CN105956090A (en) * | 2016-04-27 | 2016-09-21 | 中国科学技术大学 | I/O self-adaption-based file system log mode |
| CN106294866A (en) * | 2016-08-23 | 2017-01-04 | 北京奇虎科技有限公司 | A kind of log processing method and device |
-
2017
- 2017-11-23 CN CN201711180655.1A patent/CN109828862B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102375853A (en) * | 2010-08-24 | 2012-03-14 | 中国移动通信集团公司 | Distributed database system, method for building index therein and query method |
| CN103077222A (en) * | 2012-12-31 | 2013-05-01 | 中国科学院计算技术研究所 | Method and system for ensuring consistence of distributed metadata in cluster file system |
| CN104301360A (en) * | 2013-07-19 | 2015-01-21 | 阿里巴巴集团控股有限公司 | Method, log server and system for recording log data |
| CN103516549A (en) * | 2013-09-27 | 2014-01-15 | 浪潮电子信息产业股份有限公司 | File system metadata log mechanism based on shared object storage |
| CN103902479A (en) * | 2014-03-27 | 2014-07-02 | 浪潮电子信息产业股份有限公司 | Quick reconstruction mechanism for metadata cache on basis of metadata log |
| CN104050268A (en) * | 2014-06-23 | 2014-09-17 | 西北工业大学 | Continuous data protection and recovery method with log space adjustable online |
| CN105138284A (en) * | 2015-08-11 | 2015-12-09 | 上海交通大学 | Virtual machine disk mirror image synchronization operation optimization system and method |
| CN105574104A (en) * | 2015-12-11 | 2016-05-11 | 上海爱数信息技术股份有限公司 | LogStructure storage system based on ObjectStore and data writing method thereof |
| CN105956090A (en) * | 2016-04-27 | 2016-09-21 | 中国科学技术大学 | I/O self-adaption-based file system log mode |
| CN106294866A (en) * | 2016-08-23 | 2017-01-04 | 北京奇虎科技有限公司 | A kind of log processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109828862A (en) | 2019-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108563532B (en) | Data processing method and related device | |
| US8560778B2 (en) | Accessing data blocks with pre-fetch information | |
| CN106951521B (en) | Method, device and system for reading and writing log file | |
| CN110633110A (en) | Server starting method, equipment and storage medium | |
| US11379836B2 (en) | Methods and systems for recording data based on plurality of blockchain networks | |
| CN110837502B (en) | Data storage method, device and equipment in block chain type account book | |
| US11050550B2 (en) | Methods and systems for reading data based on plurality of blockchain networks | |
| CN103942126B (en) | Electronic equipment and its data back up method | |
| CN112954244A (en) | Method, device and equipment for realizing storage of monitoring video and storage medium | |
| CN110941514A (en) | Data backup method, data recovery method, computer equipment and storage medium | |
| CN113515531A (en) | Data access method, device, client and storage medium | |
| WO2018006721A1 (en) | Method and apparatus for storing log file | |
| CN112162773A (en) | Differential upgrading method and device, storage medium and terminal | |
| CN112988458B (en) | Data backup method and device, electronic equipment and storage medium | |
| CN110597461B (en) | Data storage method, device and equipment in block chain type account book | |
| JP2014526098A (en) | Method and system for downloading font files | |
| CN112182010A (en) | Dirty page refreshing method and device, storage medium and electronic equipment | |
| CN111803917A (en) | Resource processing method and device | |
| CN110377276B (en) | Source code file management method and device | |
| CN113535226B (en) | Application downloading method, terminal, server, computer device and storage medium | |
| CN109828862B (en) | Method and device for replaying log | |
| CN108491315A (en) | The page is resident statistical method, device and the computer readable storage medium of duration | |
| CN106293542B (en) | Method and device for decompressing file | |
| CN110187837B (en) | File access method, device and file system | |
| CN110597462A (en) | Data storage method, system, device and equipment in block chain type account book |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |