CN113778329B - Data aggregation method, device, equipment and computer readable storage medium - Google Patents
Data aggregation method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN113778329B CN113778329B CN202110744253.XA CN202110744253A CN113778329B CN 113778329 B CN113778329 B CN 113778329B CN 202110744253 A CN202110744253 A CN 202110744253A CN 113778329 B CN113778329 B CN 113778329B
- Authority
- CN
- China
- Prior art keywords
- data
- aggregation
- target data
- target
- metadata
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000002776 aggregation Effects 0.000 title claims abstract description 102
- 238000004220 aggregation Methods 0.000 title claims abstract description 102
- 238000000034 method Methods 0.000 title claims abstract description 65
- 230000006870 function Effects 0.000 claims description 25
- 230000008569 process Effects 0.000 claims description 14
- 230000004048 modification Effects 0.000 claims description 10
- 238000012986 modification Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 230000004931 aggregating effect Effects 0.000 claims description 5
- 230000001960 triggered effect Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a data aggregation method, a device, equipment and a computer readable storage medium, under the condition that a multi-version function is started, the data aggregation method in the application can aggregate received target data with a storage capacity smaller than a preset threshold value to corresponding aggregated data, package the version number of the target data and the position of the body data of the target data in the aggregated data in bitmap information of the aggregated data, add identity information of the aggregated data in metadata of the target data, and even if the target data with a duplicate name exists, a user can quickly and accurately find the specific position of the body data in the target data according to the version number of each target data, namely, the application can identify the duplicate name target data with different version numbers, so that the duplicate name data is not subjected to coverage storage, the risk of data loss is reduced, and the user experience is improved.
Description
Technical Field
The present invention relates to the field of distributed storage systems, and in particular, to a data aggregation method, and a data aggregation method, apparatus, device, and computer readable storage medium.
Background
In a distributed storage system, in order to improve the storage space utilization rate, a plurality of target data with smaller storage capacity can be aggregated into one aggregated data with larger storage capacity by utilizing an aggregation function, and then the aggregated data is stored, but under the condition that a multi-version function in the distributed storage system is started, a user can create a plurality of data with the same name, and in this case, if a plurality of target data with the same name are aggregated into one aggregated data in the aggregation process, the target data with the later aggregation sequence can cover the target data which are already existing in the aggregated data and have the same name as the target data, so that the data is lost, and the user experience is reduced.
Therefore, how to provide a solution to the above technical problem is a problem that a person skilled in the art needs to solve at present.
Disclosure of Invention
The invention aims to provide a data aggregation method, which does not cover and store the renamed data in the aggregation process, reduces the risk of data loss and improves the user experience; another object of the present invention is to provide a data aggregation method, apparatus, device, and computer readable storage medium, in which the overlapping storage of the renamed data is not performed in the aggregation process, so that the risk of data loss is reduced, and the user experience is improved.
In order to solve the technical problems, the present invention provides a data aggregation method, including:
when target data with storage capacity smaller than a preset threshold value is received, aggregating body data of the target data to aggregation data corresponding to the target data;
judging whether the multi-version function of the distributed storage system is started or not;
if yes, integrating the position of the body data of the target data in the aggregated data and the version number of the target data into a structure form and adding the structure form into bitmap information of the aggregated data so as to search the specific position of the body data in the aggregated data through the bitmap information by the version number;
adding identity information of the aggregation data into metadata of the target data so as to determine the aggregation data of the body data of the target data through the identity information;
wherein the storage capacity of the aggregate data is greater than the preset threshold.
Preferably, after the determining whether the multi-version function of the distributed storage system is turned on, the data aggregation method further includes:
and if not, adding the position of the body data of the target data in the aggregation data to the bitmap information and executing the step of adding the identity information of the aggregation data in the metadata of the target data.
Preferably, the determining whether the multi-version function of the distributed storage system is turned on specifically includes:
judging whether a version number exists in metadata of the target data;
if so, the multi-version function of the distributed storage system is judged to be started.
Preferably, the data aggregation method further comprises:
and responding to a parameter modification instruction, and modifying the preset threshold according to the parameter modification instruction.
Preferably, the identity information of the aggregated data is specifically at least one of a name, a creation date, and a storage location.
Preferably, the distributed storage system is a distributed object storage system.
Preferably, the data aggregation method further comprises:
responding to a deleting instruction, and determining metadata of the target data according to the identity information of the target data appointed in the deleting instruction;
determining the aggregation data of the body data of the target data designated in the deleting instruction according to the determined identity information of the aggregation data in the metadata;
searching the position of the body data of the target data specified in the deleting instruction in the bitmap information of the determined aggregated data according to the determined version number in the metadata;
positioning the body data of the target data designated in the deleting instruction in the determined aggregated data according to the found position and deleting the body data;
deleting the determined metadata of the target data;
and deleting the data in the form of the structural body corresponding to the found position in the determined bitmap information of the aggregated data.
In order to solve the above technical problem, the present invention further provides a data aggregation device, including:
the aggregation module is used for aggregating the body data of the target data to the aggregation data corresponding to the target data when the target data with the storage capacity smaller than a preset threshold value is received;
the judging module is used for judging whether the multi-version function of the distributed storage system is started or not, and if yes, the updating module is triggered;
the updating module is used for integrating the position of the body data of the target data in the aggregated data and the version number of the target data into a structure form and adding the structure form to bitmap information of the aggregated data so as to search the specific position of the body data in the aggregated data in the bitmap information through the version number;
the adding module is used for adding the identity information of the aggregation data in the metadata of the target data so as to determine the aggregation data of the body data of the target data through the identity information;
wherein the storage capacity of the aggregate data is greater than the preset threshold.
In order to solve the above technical problem, the present invention further provides a data aggregation device, including:
a memory for storing a computer program;
a processor for implementing the steps of the data aggregation method as described above when executing the computer program.
To solve the above technical problem, the present invention also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the data aggregation method as described above.
The invention provides a data aggregation method, under the condition that a multi-version function is started, the data aggregation method in the application can aggregate the received target data with the storage capacity smaller than a preset threshold value to the corresponding aggregated data, the version number of the target data and the position of the body data of the target data in the aggregated data are packed and placed in bitmap information of the aggregated data, and identity information of the aggregated data is added in metadata of the target data, even if the renamed target data exist, due to the uniqueness of the version number of each target data, a user can quickly and accurately find the specific position of the body data in the target data according to the version number of the target data, namely, the application can identify the renamed target data with different version numbers, so that the overlapping storage of the renamed data is avoided, the risk of data loss is reduced, and the user experience is improved.
The invention also provides a data aggregation device, equipment and a computer readable storage medium, which have the same beneficial effects as the data aggregation method.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required in the prior art and the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a data aggregation method provided by the invention;
FIG. 2 is a schematic diagram of a data aggregation device according to the present invention;
fig. 3 is a schematic structural diagram of a data aggregation device according to the present invention.
Detailed Description
The core of the invention is to provide a data aggregation method, which does not cover and store the renamed data in the aggregation process, reduces the risk of data loss and improves the user experience; the invention further provides a data aggregation method, a device, equipment and a computer readable storage medium, wherein the method, the device and the equipment do not cover and store the renamed data in the aggregation process, so that the risk of data loss is reduced, and the user experience is improved.
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, fig. 1 is a flow chart of a data aggregation method provided by the present invention, where the data aggregation method includes:
s101: when target data with the storage capacity smaller than a preset threshold value is received, aggregating body data of the target data to aggregation data corresponding to the target data;
specifically, in view of the technical problems in the background art, the invention is to aggregate each small data (data with a storage capacity smaller than a preset threshold) in the distributed storage system under the open multi-version function into aggregated data, and in the process, no coverage operation is performed on the same-name data to prevent data loss, so that in the step, when the target data with the storage capacity smaller than the preset threshold is received, the body data of the target data is aggregated into the aggregated data corresponding to the target data, that is, whether the data with repeated names exists before or not, the body data of the target data is aggregated, thereby realizing the aggregation of all the small data, and specific distinguishing measures are embodied in the subsequent steps.
S102: judging whether the multi-version function of the distributed storage system is started or not;
specifically, in consideration of that there may be a plurality of data with the same name in the case of the multi-version function being started, the corresponding processing manner is different, so that it is required to determine whether the multi-version function of the distributed storage system is started in this step, so as to trigger a subsequent action through the determination result.
S103: if yes, integrating the position of the body data of the target data in the aggregated data and the version number of the target data into a structure form and adding the structure form to bitmap information of the aggregated data, and searching the specific position of the body data in the aggregated data through the bitmap information by the version number;
specifically, in the case of opening the multi-version function, in the present application, to distinguish each target data by its unique version number (i.e. ID), theoretically, the position of the body data of the target data can be found according to the position by finding the position of the target data from the bitmap information, but assuming that there are a plurality of small data with the same name in the bitmap information, the position data in the bitmap information "position" cannot be found by name at this time, so in this step, the "position" data of the target data and its unique version number are integrated into a structure and added into the bitmap information of the aggregated data, and thus, the "position" data of the body data of the target data bound with the version number can be found in the bitmap information only by the version number of the target data, so that the body data can be found in the aggregated data by using the "position" data.
S104: adding identity information of the aggregation data into metadata of the target data so as to determine the aggregation data of the body data of the target data through the identity information;
wherein the storage capacity of the aggregate data is greater than a preset threshold.
Specifically, in the process of searching the target data, the user needs to search the metadata through the identity information of the target data, after the identity information of the aggregate data in the metadata is found, the bitmap information of the corresponding aggregate data can be found through the identity information of the aggregate data, and finally, the 'position' data is determined from the bitmap information through the version number in the metadata of the target data, so that the body data of the target data can be found in the aggregate data finally.
The invention provides a data aggregation method, under the condition that a multi-version function is started, the data aggregation method in the application can aggregate the received target data with the storage capacity smaller than a preset threshold value to the corresponding aggregated data, the version number of the target data and the position of the body data of the target data in the aggregated data are packed and placed in bitmap information of the aggregated data, and identity information of the aggregated data is added in metadata of the target data, even if the renamed target data exist, due to the uniqueness of the version number of each target data, a user can quickly and accurately find the specific position of the body data in the target data according to the version number of the target data, namely, the application can identify the renamed target data with different version numbers, so that the overlapping storage of the renamed data is avoided, the risk of data loss is reduced, and the user experience is improved.
Based on the above embodiments:
as a preferred embodiment, after determining whether the multi-version function of the distributed storage system is turned on, the data aggregation method further includes:
if not, adding the position of the body data of the target data in the aggregate data to the bitmap information and executing the step of adding the identity information of the aggregate data in the metadata of the target data.
Specifically, under the condition that the multi-version function is not started, it is proved that no newly uploaded target data with the same name exists in the distributed system, so that the situation that the target data with the same name before can be covered is not needed to be considered, at the moment, the position of the body data of the target data in the aggregate data can be directly added to the bitmap information, the step of adding the identity information of the aggregate data in the metadata of the target data is executed, and the unique 'position' data of the target data can be found from the bitmap information through the name in the metadata of the target data in the subsequent searching process without distinguishing the bitmap information through the version number.
As a preferred embodiment, determining whether the multi-version function of the distributed storage system is turned on is specifically:
judging whether a version number exists in metadata of target data;
if so, the multi-version function of the distributed storage system is judged to be started.
Specifically, the judging mode is simple, quick and accurate.
Of course, in addition to this determination method, the determination may be performed in a manner of "determining whether the configuration parameter in the bucket corresponding to the target data configures the multi-version function to be on" which is not limited herein.
As a preferred embodiment, the data aggregation method further includes:
and responding to the parameter modification instruction, and modifying the preset threshold according to the parameter modification instruction.
Specifically, in order to facilitate the user to modify the preset threshold value autonomously, a modification interface is opened in the embodiment of the invention, and the user can modify the preset threshold value through the parameter modification instruction, so that the working efficiency and the user experience are improved.
The preset threshold may be various data, for example, 512kb, etc., which is not limited herein.
Specifically, the size of the aggregate data may be set autonomously, for example, may be set to 4MB, which is not limited in the embodiment of the present invention.
The parameter modification instruction may be sent by the user through a man-machine interaction device, and the man-machine interaction device may be specifically of a plurality of types, for example, may be a mobile terminal, etc., which is not limited herein.
As a preferred embodiment, the identity information of the aggregated data is specifically at least one of a name, a creation date, and a storage location.
Specifically, at least one of the name, creation date, and storage location can accurately find the aggregated data, and the data amount is small.
Of course, the identity information of the aggregated data may be of various types other than at least one of the name, the creation date, and the storage location, and the embodiments of the present invention are not limited herein.
As a preferred embodiment, the distributed storage system is a distributed object storage system.
In particular, the distributed object storage system has a characteristic of wide use.
Of course, the distributed storage system may be other types besides a distributed object storage system, such as a distributed file storage system, and the embodiment of the present invention is not limited herein.
As a preferred embodiment, the data aggregation method further includes:
responding to the deleting instruction, and determining metadata of the target data according to the identity information of the target data appointed in the deleting instruction;
determining the aggregate data of the body data of the target data designated in the deleting instruction according to the determined identity information of the aggregate data in the metadata;
searching the position of the body data of the target data appointed in the deleting instruction in the bitmap information of the determined aggregated data according to the version number in the determined metadata;
positioning and deleting the body data of the target data appointed in the instruction in the determined aggregation data according to the found position;
deleting the metadata of the determined target data;
deleting the data in the form of the structural body corresponding to the found position in the determined bitmap information of the aggregated data.
Specifically, when deleting target data, the target data needs to be searched first, and this process has been described before, which is not described herein, and the embodiment of the invention can delete body data directly after searching body data of target data, and delete the residual metadata and data in the form of structure in bitmap information, thereby realizing complete deletion, reducing space occupation and improving user experience.
Referring to fig. 2, fig. 2 is a schematic structural diagram of a data aggregation device according to the present invention, where the data aggregation device includes:
an aggregation module 21, configured to aggregate, when receiving target data with a storage capacity smaller than a preset threshold, body data of the target data to aggregate data corresponding to the target data;
the judging module 22 is configured to judge whether the multi-version function of the distributed storage system is on, and if yes, trigger the updating module 23;
an updating module 23, configured to integrate the location of the body data of the target data in the aggregate data and the version number of the target data into a structure form and add the structure form to the bitmap information of the aggregate data, so as to find the specific location of the body data in the aggregate data in the bitmap information through the version number;
an adding module 24, configured to add identity information of the aggregate data to metadata of the target data, so as to determine, by using the identity information, the aggregate data in which the body data of the target data is located;
wherein the storage capacity of the aggregate data is greater than a preset threshold.
For the description of the data aggregation apparatus provided in the embodiment of the present invention, reference is made to the foregoing embodiment of the data aggregation method, and the embodiment of the present invention is not repeated herein.
Referring to fig. 3, fig. 3 is a schematic structural diagram of a data aggregation device provided by the present invention, where the data aggregation device includes:
a memory 31 for storing a computer program;
a processor 32 for implementing the steps of the data aggregation method in the previous embodiment when executing a computer program.
For the description of the data aggregation device provided by the embodiment of the present invention, reference is made to the foregoing embodiment of the data aggregation method, and the embodiment of the present invention is not repeated herein.
The present invention also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the data aggregation method as in the previous embodiments.
For the description of the computer readable storage medium provided in the embodiment of the present invention, reference is made to the foregoing embodiments of the data aggregation method, and the embodiments of the present invention are not repeated here.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section. It should also be noted that in this specification the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method of data aggregation, comprising:
when target data with storage capacity smaller than a preset threshold value is received, aggregating body data of the target data to aggregation data corresponding to the target data;
judging whether the multi-version function of the distributed storage system is started or not;
if yes, integrating the position of the body data of the target data in the aggregated data and the version number of the target data into a structure form and adding the structure form into bitmap information of the aggregated data so as to search the specific position of the body data in the aggregated data through the bitmap information by the version number;
adding identity information of the aggregation data into metadata of the target data so as to determine the aggregation data of the body data of the target data through the identity information;
wherein the storage capacity of the aggregate data is greater than the preset threshold;
in the process of searching the target data, firstly, searching metadata of the target data through identity information of the target data, after acquiring identity information of aggregation data in metadata of the target data, searching bitmap information of the aggregation data through the identity information of the aggregation data, and determining the position of body data of the target data in the aggregation data from the bitmap information through a version number in the metadata of the target data so as to find the body data of the target data from the aggregation data.
2. The data aggregation method according to claim 1, wherein after determining whether the multi-version function of the distributed storage system is on, the data aggregation method further comprises:
and if not, adding the position of the body data of the target data in the aggregation data to the bitmap information and executing the step of adding the identity information of the aggregation data in the metadata of the target data.
3. The data aggregation method according to claim 2, wherein the determining whether the multi-version function of the distributed storage system is turned on is specifically:
judging whether a version number exists in metadata of the target data;
if so, the multi-version function of the distributed storage system is judged to be started.
4. A data aggregation method according to claim 3, characterized in that the data aggregation method further comprises:
and responding to a parameter modification instruction, and modifying the preset threshold according to the parameter modification instruction.
5. The data aggregation method of claim 1, wherein the identity information of the aggregated data is specifically at least one of a name, a creation date, and a storage location.
6. The data aggregation method of claim 1, wherein the distributed storage system is a distributed object storage system.
7. The data aggregation method according to any one of claims 1 to 6, characterized in that the data aggregation method further comprises:
responding to a deleting instruction, and determining metadata of the target data according to the identity information of the target data appointed in the deleting instruction;
determining the aggregation data of the body data of the target data designated in the deleting instruction according to the determined identity information of the aggregation data in the metadata;
searching the position of the body data of the target data specified in the deleting instruction in the bitmap information of the determined aggregated data according to the determined version number in the metadata;
positioning the body data of the target data designated in the deleting instruction in the determined aggregated data according to the found position and deleting the body data;
deleting the determined metadata of the target data;
and deleting the data in the form of the structural body corresponding to the found position in the determined bitmap information of the aggregated data.
8. A data aggregation apparatus, comprising:
the aggregation module is used for aggregating the body data of the target data to the aggregation data corresponding to the target data when the target data with the storage capacity smaller than a preset threshold value is received;
the judging module is used for judging whether the multi-version function of the distributed storage system is started or not, and if yes, the updating module is triggered;
the updating module is used for integrating the position of the body data of the target data in the aggregated data and the version number of the target data into a structure form and adding the structure form to bitmap information of the aggregated data so as to search the specific position of the body data in the aggregated data in the bitmap information through the version number;
the adding module is used for adding the identity information of the aggregation data in the metadata of the target data so as to determine the aggregation data of the body data of the target data through the identity information;
wherein the storage capacity of the aggregate data is greater than the preset threshold;
in the process of searching the target data, firstly, searching metadata of the target data through identity information of the target data, after acquiring identity information of aggregation data in metadata of the target data, searching bitmap information of the aggregation data through the identity information of the aggregation data, and determining the position of body data of the target data in the aggregation data from the bitmap information through a version number in the metadata of the target data so as to find the body data of the target data from the aggregation data.
9. A data aggregation apparatus, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the data aggregation method according to any one of claims 1 to 7 when executing said computer program.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the data aggregation method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110744253.XA CN113778329B (en) | 2021-06-30 | 2021-06-30 | Data aggregation method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110744253.XA CN113778329B (en) | 2021-06-30 | 2021-06-30 | Data aggregation method, device, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113778329A CN113778329A (en) | 2021-12-10 |
| CN113778329B true CN113778329B (en) | 2023-12-22 |
Family
ID=78836066
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110744253.XA Active CN113778329B (en) | 2021-06-30 | 2021-06-30 | Data aggregation method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113778329B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101496000A (en) * | 2006-07-31 | 2009-07-29 | 微软公司 | Two-way and multi-master synchronization over web syndications |
| CN107958079A (en) * | 2017-12-14 | 2018-04-24 | 郑州云海信息技术有限公司 | Aggregate file delet method, system, device and readable storage medium storing program for executing |
| CN110377584A (en) * | 2019-06-24 | 2019-10-25 | 武汉理工大学 | A kind of access method and device of the data structure edition compatibility based on metadata |
| CN111966637A (en) * | 2020-08-21 | 2020-11-20 | 苏州浪潮智能科技有限公司 | Method, device and equipment for opening file and readable medium |
-
2021
- 2021-06-30 CN CN202110744253.XA patent/CN113778329B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101496000A (en) * | 2006-07-31 | 2009-07-29 | 微软公司 | Two-way and multi-master synchronization over web syndications |
| CN107958079A (en) * | 2017-12-14 | 2018-04-24 | 郑州云海信息技术有限公司 | Aggregate file delet method, system, device and readable storage medium storing program for executing |
| CN110377584A (en) * | 2019-06-24 | 2019-10-25 | 武汉理工大学 | A kind of access method and device of the data structure edition compatibility based on metadata |
| CN111966637A (en) * | 2020-08-21 | 2020-11-20 | 苏州浪潮智能科技有限公司 | Method, device and equipment for opening file and readable medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113778329A (en) | 2021-12-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107015985B (en) | Data storage and acquisition method and device | |
| CN107247722B (en) | File scanning method and device and intelligent terminal | |
| CN107391045B (en) | Directory space quota method and system | |
| CN107800757B (en) | User behavior recording method and device | |
| CN111198856B (en) | File management method, device, computer equipment and storage medium | |
| WO2018006721A1 (en) | Method and apparatus for storing log file | |
| CN108897859A (en) | A kind of metadata retrieval method, apparatus, equipment and computer readable storage medium | |
| CN105718307A (en) | Process management method and process management device | |
| CN102937923B (en) | A kind of method of automatic preservation user interface state | |
| CN113778329B (en) | Data aggregation method, device, equipment and computer readable storage medium | |
| US11853229B2 (en) | Method and apparatus for updating cached information, device, and medium | |
| US10241927B2 (en) | Linked-list-based method and device for application caching management | |
| CN111552438A (en) | Object writing method, device, server and storage medium | |
| CN107330031A (en) | A kind of method of data storage, device and electronic equipment | |
| US9626371B2 (en) | Attribute selectable file operation | |
| CN113868194B (en) | Management method, device, system and medium for local user group of storage equipment | |
| CN106371867B (en) | Financial equipment software updating method and system | |
| CN111061699B (en) | Method and device for updating version of electric energy meter database | |
| CN109343953B (en) | Memory management method and device and electronic equipment | |
| CN112286876A (en) | Log file capturing method and device and computer readable storage medium | |
| CN106970964B (en) | GPS data information query method and system based on shared memory | |
| CN110968467A (en) | Remote automatic test method for GPU and algorithm | |
| CN113127479A (en) | Method and device for loading Elasticissearch index, computer equipment and storage medium | |
| CN111629378A (en) | Method and device for intercepting risk short messages based on bloom filter | |
| CN110321358A (en) | A kind of method and device of user data reorganization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |