CN111752909A - Operation method, system and device of multi-version file - Google Patents
Operation method, system and device of multi-version file Download PDFInfo
- Publication number
- CN111752909A CN111752909A CN202010528171.7A CN202010528171A CN111752909A CN 111752909 A CN111752909 A CN 111752909A CN 202010528171 A CN202010528171 A CN 202010528171A CN 111752909 A CN111752909 A CN 111752909A
- Authority
- CN
- China
- Prior art keywords
- version
- target object
- operation log
- metadata
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1873—Versioning file systems, temporal file systems, e.g. file system supporting different historic versions of files
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses an operation method, a system and a device of a multi-version file, wherein the method comprises the following steps: configuring an object logic head for a multi-version file, wherein the object logic head is used for recording metadata of the multi-version file; wherein different multi-version files are provided with different object logical heads; when a target object in the multi-version file is operated, generating an operation log of the target object in the object logic head, and after the target object completes the operation, executing a completing action matched with the operation log on an instance of the target object in the object logic head; updating version information of the multi-version file in the object logical header, wherein the version information is used for at least characterizing the number of the latest version and pointing to the object of the latest version. The technical scheme provided by the application can improve the operating efficiency of the multi-version file.
Description
Technical Field
The invention relates to the technical field of internet, in particular to an operation method, a system and a device of a multi-version file.
Background
In the current distributed object storage system, a file version control function is usually provided to avoid the false deletion and false modification of files. After a storage space (bucket) version control function is started, all the past operations aiming at the file can be saved in a historical version form, so that the multi-version file is formed.
At present, when an operation is performed on a multi-version file, an operation log may be written in the bucket index, after the operation on the multi-version file is completed, the operation log may be read from the bucket index, a header object of the multi-version file may be modified according to the operation log, and after the modification is successful, the corresponding operation log may be deleted from the bucket index.
In practical application, a plurality of multi-version files need to be uploaded or deleted at the same time, and the operation process of each multi-version file needs to access the bucket index for multiple times. Meanwhile, the operations are usually in a mutually exclusive relationship, so that when accessing the bucket index, the operations of different multi-version files are actually serial operations, which greatly affects the operating efficiency of the multi-version files.
Disclosure of Invention
The application aims to provide an operation method, a system and a device of multi-version files, which can improve the operation efficiency of the multi-version files.
To achieve the above object, one aspect of the present application provides a method of operating a multi-version document, the method comprising: configuring an object logic head for a multi-version file, wherein the object logic head is used for recording metadata of the multi-version file; wherein different multi-version files are provided with different object logical heads; when a target object in the multi-version file is operated, generating an operation log of the target object in the object logic head, and after the target object completes the operation, executing a completing action matched with the operation log on an instance of the target object in the object logic head; updating version information of the multi-version file in the object logical header, wherein the version information is used for at least characterizing the number of the latest version and pointing to the object of the latest version.
To achieve the above object, another aspect of the present application further provides an operating system of a multi-version file, the system including: the object logic head configuration unit is used for configuring an object logic head for a multi-version file, and the object logic head is used for recording metadata of the multi-version file; wherein different multi-version files are provided with different object logical heads; an object operation unit, configured to, when an operation is performed on a target object in the multi-version file, generate an operation log of the target object in the object logic header, and after the target object completes the operation, perform a completion action matching the operation log on an instance of the target object in the object logic header; and the version information updating unit is used for updating the version information of the multi-version file in the object logical header, wherein the version information is at least used for representing the number of the latest version and pointing to the object of the latest version.
In order to achieve the above object, in another aspect, the present application further provides an operating apparatus in a multi-version file, where the operating apparatus includes a memory and a processor, the memory is used for storing a computer program, and the computer program, when executed by the processor, implements the operating method in the multi-version file.
As can be seen from the above, according to the technical solutions provided by one or more embodiments of the present application, different object logic heads may be configured for different multi-version files, and the object logic heads may record metadata of the corresponding multi-version files. Therefore, when different multi-version files need to be subjected to file operation, only the object logic head of the file needs to be accessed, and the same bucket index does not need to be accessed for multiple times. In particular, when a target object is operated on for a multi-versioned file, a corresponding operation log may be generated in the object logical header, and upon completion of the operation of the target object, a matching completion action may be performed on an instance of the target object in the object logical header. Subsequently, after the current operation of the multi-version file is completed, the version information of the multi-version file can be updated in the object logical header. Therefore, different multi-version files can manage the metadata through the object logic heads of the files, so that when the operation of the target object occurs, the operation process of other multi-version files does not need to be waited, and the operation efficiency of the multi-version files can be greatly improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic illustration of the steps of a method of operation of a multi-version document in an embodiment of the present invention;
FIG. 2 is a functional block diagram of an operating system of a multi-versioned file in an embodiment of the invention;
fig. 3 is a schematic structural view of an operation device of a multi-version document in the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clear, the technical solutions of the present application will be clearly and completely described below with reference to the detailed description of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art without any inventive work based on the embodiments in the present application are within the scope of protection of the present application.
The operation method of the multi-version file can be improved aiming at the existing distributed object storage system, so that the problem of low operation efficiency of the multi-version file is solved.
Referring to fig. 1, an operation method of the multi-version document provided by an embodiment of the present application may include the following steps.
S1: configuring an object logic head for a multi-version file, wherein the object logic head is used for recording metadata of the multi-version file; wherein different multi-version files are provided with different object logical headers.
In this embodiment, an object logic header (olh) of the multi-version file itself may be provided for each multi-version file, and metadata of the multi-version file may be recorded in the object logic header. Thus, different multi-version files can manage the metadata by the object logic head of the file, and the metadata does not need to be managed through a uniform interface.
In one embodiment, the configured object logical header may include three data types, where a first type of data may be used to store key-value pairs of the object, a second type of data may be used to store extended attributes of the object, and a third type of data may be used to store data of the object. For example, the three data types may be an omap type, an attr type, and a data type. The omap type may store key-value pairs of the object in a key-value pair database (rockdb, MangoDB, etc.), the attr type may store extended attributes of the object, and the data type may store data of the object.
In one embodiment, the first type of data may further include a first sub data type and a second sub data type, wherein the first sub data type may be used to store a number of a latest version and point to an object of the latest version, and the second sub data type may be used to store an operation log and a version instance of the object. In addition, the extended attribute of the second type data storage may include the number of the latest version, the object of the latest version, and whether the object of the latest version is a deletion instance, so that the second type data may be used to store the number of the latest version, point to the object of the latest version, and mark whether the object of the latest version is a deletion instance.
In one specific application example, two subtypes of the omap type and the extended attributes stored by the attr type may be as shown in tables 1 and 2:
table 1 omap type corresponding two subdata types
TABLE 2 extended attribute description for attr type storage
| Extending attribute names | Description of attribute values |
| user.olh.latest_version | Number of latest version |
| User.olh.target | Latest version of object |
| user.olh.is_removed | Whether the latest version of an object is a delete instance |
Of course, in practical applications, the object logical header of the multi-version file may further include fewer or more data types, and the functions of the data types may be changed according to actual situations. However, those skilled in the art should understand that reasonable changes to the form of the target logical header should also fall within the scope of the present application.
S3: when the operation is carried out on the target object in the multi-version file, an operation log of the target object is generated in the object logic head, and after the operation of the target object is completed, a completing action matched with the operation log is executed on the instance of the target object in the object logic head.
In the present embodiment, after the target logical head is configured for the multi-version file, a plurality of atomic operations (atomic operations) can be further designed to ensure that the target logical head can be effectively operated. Atomic operations refer to operations that are not interrupted by a thread scheduling mechanism. These atomic operations may perform operations such as deletion, uploading, etc. of objects for multi-version files, and may write or delete oplogs to or from the object logical header, and may perform operations on version instances of objects in the object logical header, etc.
In one specific application example, part of the atomic operations of a design may be as shown in Table 3:
table 3 partial atomic operation description
In the present embodiment, after an atomic operation is designed for a multi-version file, a flow such as uploading or deleting a file can be performed by using the atomic operation. Specifically, if a target object needs to be added to the current multi-version file, an operation log of the target object may be generated in an object logical header of the multi-version file. Then, when the target object is stored in the multi-version file of the distributed object storage system, it indicates that the uploading operation has been completed on the target object. At this time, the object logic head of the multi-version file needs to be modified accordingly. Specifically, the previously generated operation log of the target object may include information such as the version number of the current operation and whether the current operation is labeled with a deletion tag, and the operation log may be converted into a version instance of the corresponding object by combining the information. After the version instance of the object is obtained by conversion, the version instance of the object can be written into the object logical header, thereby completing the change of the metadata.
In another application example, if a target object needs to be deleted in a multi-version file, a corresponding oplog is also generated in the object logical header. Subsequently, after the target object is deleted from the distributed object storage system, the instance of the target object can be deleted from the object logical head according to the generated operation log, so that the change of the metadata can be completed.
From the above, the completion actions performed in the object logical head are different according to the operation types. If the operation log indicates that the target object is added in the multi-version file, the executed completing action may be to generate an instance of the target object according to the operation log and store the instance of the target object in the object logical head. And if the oplog representation deletes the target object in the multi-version file, the performed completing action may be deleting the instance of the target object in the object logical header. In this way, a completion action matching the operation log may be performed on the instance of the target object in the object logical header after the target object completes the operation.
In one embodiment, if the oplog indicates that the target object is deleted in the multi-version file, after the instance of the target object is deleted in the object logical header, if no other instance exists in the object logical header, all objects in the current multi-version file of the specification have been deleted, at which time the object logical header of the multi-version file may also be deleted, so that the metadata can be kept consistent with the data of the actual multi-version file.
In practical applications, multiple concurrent operations may exist for the same multi-version file. Taking two concurrent operations as an example, assume that a first target object and a second target object are concurrently operated for a multi-version file, and the second target object completes the operation after the first target object. In one application example, the two concurrent operations are both operations for adding a target object, and then the concurrent adding process of the two target objects can be as shown in table 4:
table 4 schematic table for concurrent adding of two target objects
After the concurrent addition, the objects VA and VB are newly added in the multi-version file.
In another application example, assuming two concurrent operations as adding a first target object and deleting a second target object, the procedure of the two concurrent operations may be as shown in table 5:
table 5 Simultaneous operation schematic table for adding and deleting objects
As can be seen from the above, through the two concurrent operations, VA is deleted from the original objects VA and VB in the multi-version file, and an object VC is added at the same time, the multi-version file at least includes the objects VB and VC, and the objects VA, the object VB, the object VC, the object VE, and the object VF used for illustration in this document and other paragraphs herein all represent different versions of the same multi-version file.
In another application example, assuming that both concurrent operations are delete operations, the procedure of the two concurrent operations may be as shown in table 6:
table 6 schematic diagram of operations to concurrently delete two objects
From the above, after the target object is deleted, if there is no version instance of other object in the object logical head, the object logical head can also be deleted. In practical application, when at least one of the operation types of the first target object and the second target object is a delete object type, after deleting the version instance of the object in the object logical head, if no other instance exists in the object logical head, the first object logical head may be deleted.
S5: updating version information of the multi-version file in the object logical header, wherein the version information is used for at least characterizing the number of the latest version and pointing to the object of the latest version.
In this embodiment, the version information of the multi-version file may be updated after the completion action of the instance of the target object is performed in the object logical header. Generally, the version information may include the number of the latest version, and the version information may mark the object to which the latest version points. Wherein, the number of the latest version can be gradually increased according to the operation times of the object in the multi-version file. For example, if the original version number of the multi-version file is 1, the number of the latest version may be 3 after sequentially adding the objects VA and VB to the multi-version file. Then, after the object VB is deleted from the multi-version file and the object VC is added again, the number of the latest version can be 5. It can be seen that, if the first target object and the second target object are operated concurrently for a multi-version file, the version numbers may be incremented one by one on the basis of the original version numbers for the operation of the first target object and the operation of the second target object to generate the numbers of the latest version. The object to which the latest version points may be determined according to the operation type of the object. Specifically, the latest version of the object may be determined in the multi-version file according to the operation type of the first target object and the operation type of the second target object, and the latest version of the object may be the object pointed to by the version information. For example, if the operation types of the first target object and the second target object are both the added object type, the second target object may be determined to be the latest version of the object because the second target object is added last. For example, in table 4, the latest version of the object may be VB. If one of the operation types of the first target object and the second target object is the added object type, and the other one is the deleted object type, the target object corresponding to the added object type may be determined to be the object of the latest version. For example, in table 5, the latest version of the object may be a VC. If the operation types of the first target object and the second target object are both deletion object types, the newly added object may be queried from the remaining objects of the multi-version file, and the newly added object may be determined to be the latest version of the object. For example, in Table 6, if VE and VF remain in the multi-version file after VB is deleted, and VF is added after VE, then the latest version of the object may be VF.
In the present embodiment, after the number of the latest version is generated and the object to which the latest version points is specified, the information can be recorded in the version information of the updated multi-version file, thereby completing the update process of the version information.
Further, the extended attributes in the object logical header may be updated after the operation of the target object is completed. For example, after completing the operation of the target object, the latest version number is 5, the latest version points to the object is VC, and the latest version points to the object without a delete marker, then the attr data types may be as shown in Table 7:
TABLE 7 schematic table of attr data types
In one embodiment, considering that objects are generally managed by buckets (storage areas) in the distributed object storage system, after the distributed object storage system is modified, the metadata of the multi-version file may be stored in the corresponding object logical header, but at the same time, the metadata of the multi-version file also needs to be recorded in the buckets. Since the same metadata needs to be stored in two different areas, there may be a problem that the two pieces of metadata are not consistent. In view of this, in the present embodiment, an automatic repair function of the metadata needs to be configured.
Specifically, the operation procedure of uploading or deleting the object may be as shown in table 8:
TABLE 8 schematic representation of the operation of objects
As can be seen from the above, after the operation log of the target object is generated in the object logic head, the operation log in the object logic head may be set to be in a pending state. Then, after the metadata operation log of the target object is generated in the storage area of the multi-version file, the metadata operation log may also be set to be in a pending state. After the target object completes the operation in the distributed object storage system, the corresponding operation log in the object logic header is deleted, and the state of the metadata operation log in the bucket is set to be the completion state.
In practical applications, one or more of the steps in table 8 may fail. For example:
1) olh _ prepare _ op success
2) Packet _ prepare _ op success
3) Uploading the objects: success of the method
4) olh _ complete _ op fail
5) Packet _ complete _ op failure
In this way, the object in the distributed object storage system has been uploaded successfully, but the metadata in the object logical header and the bucket fails to be processed, and at this time, the metadata needs to be repaired.
In one embodiment, a metadata operation log of a target object may be detected in a bucket, and whether metadata of the target object in a storage area and an object logical header needs to be repaired may be determined according to a current state of the metadata operation log. Specifically, if the current state of the metadata operation log is an undetermined state, it may be determined in the system whether the target object has completed the operation in the multi-version file, and if the operation has completed, it is determined that the metadata of the target object in the bucket and the object logical header needs to be repaired. If the current state of the metadata operation log is the completion state, the metadata is processed, and it can be determined that the metadata of the target object in the bucket and the object logic header does not need to be repaired.
For example, when olh _ complete _ op and bucket _ complete _ op fail, there is a metadata operation log of an object in the bucket, and when the bucket is processing the metadata operation log, the storage state of the object may be queried in the system. If the system indicates that the object has been successfully uploaded, the bucket can actively repair the metadata of the object in the bucket and the object logical header. Specifically, in the object logical head, the corresponding completing action may be executed according to the operation log in the object logical head, and the operation log may be deleted. In the bucket, the metadata operation log may be set to a completion state, thereby completing the repair process of the metadata.
Therefore, in the embodiment of the application, active repair can be performed when the metadata is inconsistent, so that the availability and the correctness of the distributed object storage system are improved.
Referring to fig. 2, the present application further provides an operating system of a multi-version file, the system comprising:
the object logic head configuration unit is used for configuring an object logic head for a multi-version file, and the object logic head is used for recording metadata of the multi-version file; wherein different multi-version files are provided with different object logical heads;
an object operation unit, configured to, when an operation is performed on a target object in the multi-version file, generate an operation log of the target object in the object logic header, and after the target object completes the operation, perform a completion action matching the operation log on an instance of the target object in the object logic header;
and the version information updating unit is used for updating the version information of the multi-version file in the object logical header, wherein the version information is at least used for representing the number of the latest version and pointing to the object of the latest version.
In one embodiment, the object operating unit includes:
an object adding module, configured to generate an instance of the target object according to the operation log if the operation log indicates that the target object is added in the multi-version file, and store the instance of the target object in the object logical head;
and the object deleting module is used for deleting the instance of the target object in the object logic head if the operation log represents that the target object is deleted in the multi-version file.
In one embodiment, the version information updating unit includes:
a number updating module, configured to increment the version number one by one on the basis of the original version number to generate a number of the latest version if a first target object and a second target object are concurrently operated for the multi-version file and the second target object completes the operation after the first target object;
and the information recording module is used for determining the object of the latest version in the multi-version file according to the operation type of the first target object and the operation type of the second target object, and recording the generated number of the latest version and the determined object of the latest version in the updated version information of the multi-version file.
In one embodiment, the system further comprises:
a state setting unit, configured to set the operation log in the object logic header to an undetermined state; generating a metadata operation log of the target object in a storage area of the multi-version file, and setting the metadata operation log to be in a pending state;
a log operating unit configured to delete the operation log in the object logical header and set a state of the metadata operation log in the storage area to a completion state after the target object completes an operation.
In one embodiment, the system further comprises:
and the metadata repairing unit is used for detecting the metadata operation log of the target object in the storage area and judging whether the metadata of the target object in the storage area and the object logic head needs to be repaired according to the current state of the metadata operation log.
Referring to fig. 3, the present application further provides an operating apparatus of the multi-version file, where the operating apparatus includes a memory and a processor, the memory is used for storing a computer program, and the computer program, when executed by the processor, implements the operating method of the multi-version file.
As can be seen from the above, according to the technical solutions provided by one or more embodiments of the present application, different object logic heads may be configured for different multi-version files, and the object logic heads may record metadata of the corresponding multi-version files. Therefore, when different multi-version files need to be subjected to file operation, only the object logic head of the file needs to be accessed, and the same bucket index does not need to be accessed for multiple times. In particular, when a target object is operated on for a multi-versioned file, a corresponding operation log may be generated in the object logical header, and upon completion of the operation of the target object, a matching completion action may be performed on an instance of the target object in the object logical header. Subsequently, after the current operation of the multi-version file is completed, the version information of the multi-version file may be updated in the object logical header to point to the latest version object of the multi-version file. Therefore, different multi-version files can manage the metadata through the object logic heads of the files, so that when the operation of the target object occurs, the operation process of other multi-version files does not need to be waited, and the operation efficiency of the multi-version files can be greatly improved. In addition, when the bucket processes the metadata operation log, the metadata in the bucket and the object logic head can be actively repaired according to the state of the metadata operation log and the actual operation result of the object in the system, so that the availability and the correctness of the distributed object storage system are improved.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for embodiments of the system and of the device, reference may be made to the introduction of embodiments of the method described above in contrast to the explanation.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above description is only an embodiment of the present application, and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (19)
1. A method of operating a multi-version document, the method comprising:
configuring an object logic head for a multi-version file, wherein the object logic head is used for recording metadata of the multi-version file; wherein different multi-version files are provided with different object logical heads;
when a target object in the multi-version file is operated, generating an operation log of the target object in the object logic head, and after the target object completes the operation, executing a completing action matched with the operation log on an instance of the target object in the object logic head;
updating version information of the multi-version file in the object logical header, wherein the version information is used for at least characterizing the number of the latest version and pointing to the object of the latest version.
2. The method of claim 1, wherein the object logical header contains three types of data, wherein the first type of data is used for storing key-value pairs of the object, the second type of data is used for storing extended attributes of the object, and the third type of data is used for storing data of the object.
3. The method of claim 2, wherein the first type of data comprises a first sub data type and a second sub data type, wherein the first sub data type is used for storing a latest version number and pointing to a latest version of an object, and the second sub data type is used for storing an operation log and a version instance of the object.
4. The method of claim 2, wherein the second type of data is used to store the number of the latest version of the object, the object pointing to the latest version, and whether the object marking the latest version is a deleted instance.
5. The method of claim 1, wherein performing a completion action in the object logical head on the instance of the target object that matches the oplog comprises:
if the operation log represents that the target object is added into the multi-version file, generating an instance of the target object according to the operation log, and storing the instance of the target object into the object logic head;
and if the operation log indicates that the target object is deleted in the multi-version file, deleting the instance of the target object in the object logic head.
6. The method according to claim 1 or 5, characterized in that the method further comprises:
and if the operation log represents that the target object is deleted in the multi-version file, after the instance of the target object is deleted in the object logic head, if no other instance exists in the object logic head, deleting the object logic head.
7. The method of claim 1, wherein if a first target object and a second target object are concurrently operated for the multi-version file and the second target object completes operations after the first target object, updating the version information of the multi-version file in the object logical header comprises:
for the operation of the first target object and the operation of the second target object, gradually increasing the version number one by one on the basis of the original version number to generate the number of the latest version;
and determining the object of the latest version in the multi-version file according to the operation type of the first target object and the operation type of the second target object, and recording the generated number of the latest version and the determined object of the latest version in the updated version information of the multi-version file.
8. The method of claim 7, wherein determining a latest version of an object in the multi-version file comprises:
if the operation types of the first target object and the second target object are both the types of the added objects, determining the second target object as the object of the latest version;
if one of the operation types of the first target object and the second target object is an added object type, and the other one is a deleted object type, determining the target object corresponding to the added object type as the object of the latest version;
and if the operation types of the first target object and the second target object are deletion object types, inquiring a newly added object in the remaining objects of the multi-version file, and determining the newly added object as the object of the latest version.
9. The method of claim 7, further comprising:
and when at least one of the operation types of the first target object and the second target object is a deleted object type, deleting the object logic head if other examples do not exist in the object logic head after deleting the version example of the object in the object logic head.
10. The method of claim 1, wherein after generating the oplog of the target object in the object logical head, the method further comprises:
setting the operation log in the object logic head to be in a pending state;
generating a metadata operation log of the target object in a storage area of the multi-version file, and setting the metadata operation log to be in a pending state;
wherein, when the target object completes an operation, the operation log in the object logical header is deleted, and the state of the metadata operation log in the storage area is set to a completion state.
11. The method of claim 10, wherein after the target object completes the operation, the method further comprises:
and detecting a metadata operation log of the target object in the storage area, and judging whether the metadata of the target object in the storage area and the object logic head needs to be repaired according to the current state of the metadata operation log.
12. The method of claim 11, wherein determining whether the metadata of the target object in the storage area and the object logical header needs to be repaired comprises:
if the current state of the metadata operation log is an undetermined state, judging whether the target object completes the operation in the multi-version file, and if the target object completes the operation, judging that the metadata of the target object in the storage area and the object logic head needs to be repaired;
and if the current state of the metadata operation log is a completion state, judging that the metadata of the target object in the storage area and the object logic head does not need to be repaired.
13. The method of claim 11 or 12, wherein repairing the metadata of the target object in the storage area and the object logical header comprises:
setting the metadata operation log to a completion state in the storage area;
and executing a finishing action matched with the operation log on the instance of the target object in the object logic head, and deleting the operation log in the object logic head.
14. An operating system of a multi-version file, the system comprising:
the object logic head configuration unit is used for configuring an object logic head for a multi-version file, and the object logic head is used for recording metadata of the multi-version file; wherein different multi-version files are provided with different object logical heads;
an object operation unit, configured to, when an operation is performed on a target object in the multi-version file, generate an operation log of the target object in the object logic header, and after the target object completes the operation, perform a completion action matching the operation log on an instance of the target object in the object logic header;
and the version information updating unit is used for updating the version information of the multi-version file in the object logical header, wherein the version information is at least used for representing the number of the latest version and pointing to the object of the latest version.
15. The system according to claim 14, wherein the object operating unit includes:
an object adding module, configured to generate an instance of the target object according to the operation log if the operation log indicates that the target object is added in the multi-version file, and store the instance of the target object in the object logical head;
and the object deleting module is used for deleting the instance of the target object in the object logic head if the operation log represents that the target object is deleted in the multi-version file.
16. The system according to claim 14, wherein the version information updating unit includes:
a number updating module, configured to increment the version number one by one on the basis of the original version number to generate a number of the latest version if a first target object and a second target object are concurrently operated for the multi-version file and the second target object completes the operation after the first target object;
and the information recording module is used for determining the object of the latest version in the multi-version file according to the operation type of the first target object and the operation type of the second target object, and recording the generated number of the latest version and the determined object of the latest version in the updated version information of the multi-version file.
17. The system of claim 14, further comprising:
a state setting unit, configured to set the operation log in the object logic header to an undetermined state; generating a metadata operation log of the target object in a storage area of the multi-version file, and setting the metadata operation log to be in a pending state;
a log operating unit configured to delete the operation log in the object logical header and set a state of the metadata operation log in the storage area to a completion state after the target object completes an operation.
18. The system of claim 17, further comprising:
and the metadata repairing unit is used for detecting the metadata operation log of the target object in the storage area and judging whether the metadata of the target object in the storage area and the object logic head needs to be repaired according to the current state of the metadata operation log.
19. An operating apparatus of a multi-version file, the apparatus comprising a memory for storing a computer program which, when executed by the processor, implements the method of any of claims 1 to 13.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010528171.7A CN111752909B (en) | 2020-06-11 | 2020-06-11 | Method, system and device for operating multi-version file |
| PCT/CN2020/103447 WO2021248640A1 (en) | 2020-06-11 | 2020-07-22 | Multi-version file operation method, system and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010528171.7A CN111752909B (en) | 2020-06-11 | 2020-06-11 | Method, system and device for operating multi-version file |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111752909A true CN111752909A (en) | 2020-10-09 |
| CN111752909B CN111752909B (en) | 2023-05-16 |
Family
ID=72675931
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010528171.7A Active CN111752909B (en) | 2020-06-11 | 2020-06-11 | Method, system and device for operating multi-version file |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111752909B (en) |
| WO (1) | WO2021248640A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112286880A (en) * | 2020-10-23 | 2021-01-29 | 北京金山云网络技术有限公司 | Data storage method, device, equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5649200A (en) * | 1993-01-08 | 1997-07-15 | Atria Software, Inc. | Dynamic rule-based version control system |
| JP2000181917A (en) * | 1998-12-18 | 2000-06-30 | Hitachi Ltd | Structured document managing method, executing device therefor and medium recording processing program therefor |
| US20070005874A1 (en) * | 2005-07-01 | 2007-01-04 | Dan Dodge | File system storing transaction records in flash-like media |
| CN106708734A (en) * | 2016-12-13 | 2017-05-24 | 腾讯科技（深圳）有限公司 | Software abnormality detection method and apparatus |
| CN110612702A (en) * | 2017-05-31 | 2019-12-24 | 思科技术公司 | Intent specification checking for inconsistencies |
| CN111090620A (en) * | 2019-12-06 | 2020-05-01 | 浪潮电子信息产业股份有限公司 | File storage method, device, equipment and readable storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7363540B2 (en) * | 2002-10-22 | 2008-04-22 | Microsoft Corporation | Transaction-safe FAT file system improvements |
| US10552402B2 (en) * | 2014-11-25 | 2020-02-04 | Amarnadh Sai Eluri | Database lockless index for accessing multi-version concurrency control data |
| CN110019134A (en) * | 2017-12-27 | 2019-07-16 | 阿里巴巴集团控股有限公司 | The multi-edition control method and equipment of database |
| CN109271194B (en) * | 2018-08-22 | 2022-07-26 | 五八有限公司 | Branch access method and device based on distributed version control system |
-
2020
- 2020-06-11 CN CN202010528171.7A patent/CN111752909B/en active Active
- 2020-07-22 WO PCT/CN2020/103447 patent/WO2021248640A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5649200A (en) * | 1993-01-08 | 1997-07-15 | Atria Software, Inc. | Dynamic rule-based version control system |
| JP2000181917A (en) * | 1998-12-18 | 2000-06-30 | Hitachi Ltd | Structured document managing method, executing device therefor and medium recording processing program therefor |
| US20070005874A1 (en) * | 2005-07-01 | 2007-01-04 | Dan Dodge | File system storing transaction records in flash-like media |
| CN106708734A (en) * | 2016-12-13 | 2017-05-24 | 腾讯科技（深圳）有限公司 | Software abnormality detection method and apparatus |
| CN110612702A (en) * | 2017-05-31 | 2019-12-24 | 思科技术公司 | Intent specification checking for inconsistencies |
| CN111090620A (en) * | 2019-12-06 | 2020-05-01 | 浪潮电子信息产业股份有限公司 | File storage method, device, equipment and readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 窦万峰,李春萍: "多版本技术中的对象标识及其压缩" * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112286880A (en) * | 2020-10-23 | 2021-01-29 | 北京金山云网络技术有限公司 | Data storage method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021248640A1 (en) | 2021-12-16 |
| CN111752909B (en) | 2023-05-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11314701B2 (en) | Resharding method and system for a distributed storage system | |
| US20210152638A1 (en) | Data processing method, apparatus, and system | |
| JP4669067B2 (en) | Dynamic fragment mapping | |
| CN109284073B (en) | Data storage method, device, system, server, control node and medium | |
| CN103902623A (en) | Method and system for accessing files on a storage system | |
| TWI730690B (en) | Method and device for simultaneously executing transactions in block chain, computer readable storage medium and computing equipment | |
| US9384202B1 (en) | Gateway module to access different types of databases | |
| US20240104059A1 (en) | Method for Service Processing and System, Device, and Medium | |
| CN109033365B (en) | Data processing method and related equipment | |
| CN109189759B (en) | Data reading method, data query method, device and equipment in KV storage system | |
| CN111752909B (en) | Method, system and device for operating multi-version file | |
| CN113253932B (en) | Read-write control method and system for distributed storage system | |
| CN108062323B (en) | Log reading method and device | |
| CN106874329A (en) | The implementation method and device of database table index | |
| CN102495838B (en) | Data processing method and data processing device | |
| CN115086757B (en) | Method, device, equipment and storage medium for withdrawing history editing operation | |
| CN114281242A (en) | Method, device and equipment for balancing memory load | |
| CN113641651A (en) | Business data management method, system and computer storage medium | |
| CN113032348A (en) | Spatial data management method, system and computer readable storage medium | |
| CN111444114B (en) | Method, device and system for processing data in nonvolatile memory | |
| CN110222105A (en) | Data summarization processing method and processing device | |
| CN114840545B (en) | Block chain fine-grained editing method supporting rapid state updating | |
| CN115357384B (en) | Space reclamation method and device for repeated data deleting storage system | |
| CN112711627B (en) | Data importing method, device and equipment of Greemplum database | |
| CN114880322B (en) | Financial data column type storage method, system, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |