CN114595486B - Zero data identification method and device, readable storage medium and electronic equipment - Google Patents
Zero data identification method and device, readable storage medium and electronic equipment Download PDFInfo
- Publication number
- CN114595486B CN114595486B CN202210502419.1A CN202210502419A CN114595486B CN 114595486 B CN114595486 B CN 114595486B CN 202210502419 A CN202210502419 A CN 202210502419A CN 114595486 B CN114595486 B CN 114595486B
- Authority
- CN
- China
- Prior art keywords
- data
- identified
- zero
- calculation
- groups
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/062—Securing storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
Abstract
The invention discloses a zero data identification method, a device, a readable storage medium and electronic equipment, wherein after data to be identified is obtained, weak hash calculation is carried out on the data to be identified firstly, the weak hash calculation value of the data to be identified is compared with the value of all-zero data weak hash calculation, when the calculation values are equal, the data to be identified is judged to be all-zero data, but the weak hash algorithm possibly has the condition of data collision, namely non-zero data can also be identified as all-zero data, so that the exclusive OR calculation is carried out on the data to be identified to eliminate the all-zero data with identification errors in the weak hash algorithm, and the algorithm for identifying the zero data of a simplified storage is simplified by combining the weak hash algorithm and the recursive exclusive OR calculation, the accuracy of the zero data identification of the storage is improved, and the effect of improving the performance of the storage is achieved.
Description
Technical Field
The present invention relates to the field of memory data processing, and in particular, to a zero data identification method, apparatus, readable storage medium, and electronic device.
Background
Zero data is a special type of data that is of particular use in some typical application scenarios. The storage device has the advantages that the original data in the storage device are destroyed by sending zero data through the test tool, and the space recycling of the storage device is realized by sending the zero data.
However, the current algorithm for identifying zero data is complex, so that the normal writing performance of the memory is inevitably affected while the memory, such as an SSD (Solid State Disk or Solid State Drive), identifies zero data. Therefore, most of the conventional storages do not have the zero data identification function, and the performance of the storages cannot be further improved.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the zero data identification method and device, the readable storage medium and the electronic equipment are provided, the algorithm of storage zero data identification is simplified, and the performance of the storage is improved.
In order to solve the technical problems, the invention adopts the technical scheme that:
a zero data identification method, comprising the steps of:
acquiring data to be identified;
carrying out weak hash calculation on the data to be identified;
judging whether the hash value of the data to be identified is equal to the hash value of all-zero data or not, and if so, performing XOR calculation on the data to be identified;
and judging whether the XOR calculation result of the single-bit data in the data to be identified is zero, if so, determining that the data to be identified is all-zero data, and if not, determining that the data to be identified is non-all-zero data.
In order to solve the technical problem, the invention adopts another technical scheme as follows:
a null data recognition device comprising:
the acquisition module is used for acquiring data to be identified;
the Hash calculation module is used for carrying out weak Hash calculation on the data to be identified;
the first judgment module is used for judging whether the hash value of the data to be identified is equal to the hash value of all-zero data or not, and if so, performing exclusive-or calculation on the data to be identified;
the exclusive or calculation module is used for carrying out exclusive or calculation on the data to be identified;
and the second judgment module is used for judging whether the XOR calculation result of the single-bit data in the data to be identified is zero, if so, the data to be identified is all-zero data, and if not, the data to be identified is non-all-zero data.
In order to solve the technical problem, the invention adopts another technical scheme as follows:
a computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of a method for zero data recognition as described above.
In order to solve the technical problem, the invention adopts another technical scheme as follows:
an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of a method for zero data identification as described above when executing the computer program.
The invention has the beneficial effects that: after the data to be identified is obtained, weak hash calculation is carried out on the data to be identified, the weak hash calculation value of the data to be identified is compared with the value of all-zero data weak hash calculation, when the calculation values are equal, the data to be identified is judged to be all-zero data, but the weak hash algorithm possibly has the condition of data collision, namely non-zero data can be identified as all-zero data, the exclusive-or calculation is carried out on the data to be identified, all-zero data with identification errors in the weak hash algorithm are eliminated, the algorithm for simplifying zero data identification of the storage is simplified by combining the weak hash algorithm and the recursive exclusive-or calculation, the accuracy of the storage for identifying the zero data is improved, and the effect of improving the performance of the storage is achieved.
Drawings
FIG. 1 is a flowchart illustrating steps of a method for identifying null data according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a zero data identification device according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an electronic device in an embodiment of the invention;
fig. 4 is a logic relationship diagram of sub-data in a zero data identification method according to an embodiment of the present invention.
Detailed Description
In order to explain technical contents, achieved objects, and effects of the present invention in detail, the following description is made with reference to the accompanying drawings in combination with the embodiments.
Referring to fig. 1, a zero data identification method includes the steps of:
acquiring data to be identified;
carrying out weak hash calculation on the data to be identified;
judging whether the hash value of the data to be identified is equal to the hash value of all-zero data, and if so, performing exclusive or calculation on the data to be identified;
and judging whether the XOR calculation result of the single-bit data in the data to be identified is zero, if so, determining that the data to be identified is all-zero data, and if not, determining that the data to be identified is non-all-zero data.
As can be seen from the above description, the beneficial effects of the present invention are: after the data to be identified is obtained, weak hash calculation is carried out on the data to be identified, the weak hash calculation value of the data to be identified is compared with the value of all-zero data weak hash calculation, when the calculation values are equal, the data to be identified is judged to be all-zero data, but the weak hash algorithm possibly has the condition of data collision, namely non-zero data can be identified as all-zero data, the exclusive-or calculation is carried out on the data to be identified, all-zero data with identification errors in the weak hash algorithm are eliminated, the algorithm for simplifying zero data identification of the storage is simplified by combining the weak hash algorithm and the recursive exclusive-or calculation, the accuracy of the storage for identifying the zero data is improved, and the effect of improving the performance of the storage is achieved.
Further, the performing the xor calculation on the data to be identified includes:
judging whether the data to be identified is the single-bit data or not, if not, dividing the data to be identified into a plurality of groups of subdata with the same data volume;
combining a plurality of groups of the subdata, and carrying out XOR calculation on the combined subdata to obtain a plurality of groups of XOR calculation results;
carrying out OR calculation on a plurality of groups of XOR calculation results, if the result is zero, acquiring any subdata as the data to be identified, and returning to the step of judging whether the data to be identified is the single-bit data until the data to be identified is the single-bit data;
and if the result is not zero, the data to be identified is not zero data.
It can be known from the above description that, by determining whether the data to be identified is the single-bit data, if not, dividing the data to be identified into a plurality of sub-data sets with the same data volume, then combining the sub-data sets and performing xor calculation to obtain a plurality of xor calculation results, then performing or calculating the obtained xor results, if the result is zero, then entering any sub-data set as the current data to be identified into the determining step again until the data to be identified is the minimum order of magnitude, i.e. 1byte, and if the result is not zero, indicating that the data is not zero data and stopping the determination, by gradually recurrently determining the data to be identified acquired from the memory whether the data is zero data, and only xor and/or calculation are involved in the whole determining process, not only the zero data identification algorithm is simplified but also whether the data is zero data can be accurately determined, thereby improving the performance of the reservoir.
Further, the dividing the data to be identified into a plurality of groups of subdata with the same data volume includes:
acquiring the current storage magnitude of the data to be identified;
obtaining the division quantity according to the quantity relation between the current storage magnitude and the next-stage storage magnitude corresponding to the current storage magnitude;
and dividing the data to be identified into a plurality of groups of subdata with the same data volume according to the dividing number.
According to the above description, the current storage magnitude and the next storage magnitude of the data to be identified are obtained, the division number is obtained according to the number relation between the storage magnitudes, and then the data to be identified is divided according to the corresponding division number, so that recursive number division can be realized, zero data judgment can be performed in a step-by-step progressive manner, the judgment of the zero data is simplified, and the identification precision is improved.
Further, the combining the plurality of sets of sub-data includes:
establishing a logical relationship between each subdata and any other subdata;
and combining the subdata according to the logic relation.
As can be seen from the above description, after the logical relationship between each piece of sub-data and any other piece of sub-data is established, the sub-data is combined according to the corresponding logical relationship, so that a direct or indirect xor logical calculation relationship exists between each piece of sub-data and any other piece of sub-data, thereby ensuring the accuracy of zero data identification.
Further, the combining the multiple groups of the sub-data and performing xor calculation on the combined sub-data to obtain multiple groups of xor calculation results includes:
sequentially carrying out non-overlapping combination on two adjacent groups of subdata to obtain a plurality of groups of first combinations;
sequentially combining two groups of the sub data at intervals to obtain a plurality of groups of second combinations;
and calculating the XOR result of the first combination and the second combination to obtain a plurality of groups of XOR calculation results.
As can be seen from the above description, by combining two adjacent sets of sub-data in a non-overlapping manner, a direct operation logic exists between each sub-data in the first combination and the adjacent set of sub-data, and combining two sets of sub-data between each other, so that corresponding operation logics are formed between the sets of mutually unrelated sub-data in the first combination and the sub-data in the other first combination, thereby realizing that a direct or indirect operation logic relationship exists between each sub-data and any other sub-data, and improving the accuracy of zero data identification.
Further, the combining the multiple groups of the sub-data and performing xor calculation on the combined sub-data to obtain multiple groups of xor calculation results further includes:
sequentially overlapping and combining the two adjacent groups of subdata to obtain a third combination;
and calculating the XOR result of the plurality of groups of the third combinations to obtain a plurality of groups of the XOR calculation results.
As can be seen from the above description, by overlapping and combining two adjacent sets of sub-data, each sub-data can generate a direct xor operation logical relationship with another adjacent set of sub-data, and generate an indirect xor operation logical relationship with the inter-phase sub-data, so that a direct or indirect operation logical relationship exists between each sub-data and any other sub-data, and the accuracy of zero data identification is improved.
Further, the weak hash calculation includes: an information-digest algorithm 5 and a secure hash algorithm.
As can be seen from the above description, the weak hash calculation can be performed on the data to be identified by using a plurality of different hash algorithms such as the information-digest algorithm 5 and the secure hash algorithm 1, so that the adaptability of the system is improved.
Another embodiment of the present invention provides a null data identification device, including:
the acquisition module is used for acquiring data to be identified;
the Hash calculation module is used for carrying out weak Hash calculation on the data to be identified;
the first judgment module is used for judging whether the hash value of the data to be identified is equal to the hash value of all-zero data or not, and if so, performing exclusive-or calculation on the data to be identified;
the exclusive or calculation module is used for carrying out exclusive or calculation on the data to be identified;
and the second judgment module is used for judging whether the XOR calculation result of the single-bit data in the data to be identified is zero, if so, the data to be identified is all-zero data, and if not, the data to be identified is non-all-zero data.
Another embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of a zero data recognition method as described above.
Another embodiment of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of a zero data identification method as described above.
The above zero data identification method, apparatus, computer readable storage medium and electronic device of the present invention can be applied to the zero data identification of various types of SSD products, and are described below by specific embodiments:
example one
Referring to fig. 1, a zero data identification method includes the steps of:
s1, acquiring data to be identified; if the data to be identified is transmitted in, 4K data;
s2, performing weak hash calculation on the data to be identified; wherein the weak Hash (Hash) calculation comprises: the information-Digest Algorithm 5 (Message-Digest Algorithm 5, MD 5) and the Secure Hash Algorithm (SHA 1) may also perform the determination based on a CRC (Cyclic Redundancy Check) value in a DIF (Data Integrity Field, Data consistency protection);
s3, judging whether the hash value of the data to be identified is equal to the hash value of all-zero data or not, and if so, performing XOR calculation on the data to be identified; the all-zero data calculated by different hash algorithms have different values, so that the same hash algorithm is needed to be adopted for calculation and comparison during comparison, namely if the transmitted 4K data is calculated in an MD5 mode, the 4K all-zero data is also calculated in an MD5 mode; meanwhile, the weak Hash algorithm may have a data collision condition, that is, the non-all-zero data may also be compared by the Hash value, that is, the Hash value calculated by the non-all-zero data is the same as the all-zero data, so that it is necessary to further judge whether the data to be identified is the all-zero data by the exclusive or calculation to improve the accuracy of zero data identification;
s4, judging whether the XOR calculation result of the single-bit data in the data to be identified is zero, if so, judging that the data to be identified is all-zero data, and if not, judging that the data to be identified is non-all-zero data; judging whether 8 bits in 1byte data are all zero, if so, determining that the data are zero data, and if all bit data in the data to be identified are zero, determining that the data to be identified are all-zero data;
the judgment in step S4 may be set according to specific accuracy requirements; if the required precision is 64bytes magnitude, circularly judging until the XOR calculation result of the 64bytes magnitude data in the data to be identified is zero; and if the highest precision is required, identifying that the XOR calculation result of the single-bit data in the data to be identified is zero, namely, circularly judging and setting the final magnitude of the data to be identified according to a specific magnitude precision requirement.
Example two
The difference between the embodiment and the embodiment one is that how to perform the xor calculation on the data to be identified is specifically defined;
the step S3 includes the following steps:
s31, judging whether the data to be identified is the single-bit data, if not, dividing the data to be identified into a plurality of groups of subdata with the same data volume; as the data to be identified is 4K data, namely not single-bit data, a corresponding dividing step is executed; specifically, the dividing steps are as follows S311-S313:
s311, acquiring the current storage magnitude of the data to be identified; if the currently transmitted data to be identified is 4K data, namely the currently stored magnitude is 4K;
s312, obtaining the division quantity according to the quantity relation between the current storage magnitude and the next storage magnitude corresponding to the current storage magnitude; obtaining a division number of 8 according to the current storage magnitude 4K and the next storage magnitude sector (512B), namely 4K =8 × 512B;
s313, dividing the data to be identified into a plurality of groups of subdata with the same data volume according to the division quantity; the 4K data are sequentially divided into sector _0, sector _1, sector _2, sector _3, sector _4, sector _5, sector _6 and sector _7, and each group of subdata comprises 512B data volume;
s32, combining the plurality of sub-data sets, and performing xor calculation on the combined sub-data set to obtain a plurality of xor calculation results, specifically:
s321, establishing a logic relationship between each subdata and any other subdata; if the logical relationship between sector _0 and any group of sector _1 to sector _7 is established;
s322, combining the subdata according to the logic relationship;
in an alternative embodiment: referring to fig. 4, in step S321A, a logical relationship between sector _0 and sector _7 is established;
step S322a, sequentially combining two adjacent groups of subdata without overlapping to obtain a plurality of groups of first combinations; if the sector _0 and sector _1, the sector _2 and sector _3, the sector _4 and sector _5, and the sector _6 and sector _7 are combined, a first combination is obtained; wherein, a direct XOR logical operation relation exists between the subdata in each combination in the first combination;
s322b, sequentially combining the two groups of sub-data which are alternated to obtain a plurality of groups of second combinations; if the sector _0 and sector _2, the sector _2 and sector _4, and the sector _4 and sector _6 are combined, a second combination is obtained; the sub-data in each second combination has a direct exclusive-or logical operation relationship, and the first combinations are connected with each other through the second combination, so that an indirect exclusive-or logical operation relationship is generated between each first combination;
s322c, calculating the XOR result of the first combination and the second combination to obtain a plurality of sets of XOR calculation results; the obtained XOR result is respectively parity _0, parity _1, parity _2, parity _3, parity _4, parity _5 and parity _ 6;
in another alternative embodiment, the steps S321-S322 may also establish a logical relationship between each of the sub-data and any other sub-data by means of a nested xor operation for the step S321B:
step S322B, sequentially overlapping and combining the two adjacent groups of subdata to obtain a third combination; sequentially combining sector _0 with sector _1, sector _1 with sector _2, sector _2 with sector _3, sector _3 with sector _4, sector _4 with sector _5, sector _5 with sector _6, and sector _6 with sector _7 to obtain a third combination; calculating the XOR result of the plurality of groups of the third combinations to obtain a plurality of groups of XOR calculation results parity _0-parity _ 6;
in order to ensure the accuracy of the calculation result, different combinations may be added, but the addition of the combinations also increases the calculation amount of the system, so the logical relationship between each piece of sub-data and any other piece of sub-data is preferably established with the least number of sets;
s33, carrying out OR calculation on the multiple groups of XOR calculation results, if the result is zero, acquiring any subdata as the data to be identified, and returning to the step S31 until the data to be identified is single-bit data; carrying out OR calculation on the XOR calculation result parity _0-parity _6, wherein if the obtained OR calculation result is zero, the XOR calculation result parity _0-parity _6 is zero, that is, the data of the eight groups of sub data sector _0-sector _7 are the same as all-zero data; then, taking any one group of the subdata in sector _0-sector _7 as the data to be identified, and returning to the step S31;
if the or calculation result is not zero, it indicates that at least one set of xor calculation results in the xor calculation results parity _0-parity _6 is not zero, that is, at least one set of the sub data in the eight sets of the sub data sector _0-sector _7 is different from other sub data and is non-zero data;
returning to step S31:
judging whether a preset identification magnitude is reached, if the current preset identification magnitude is the same as the data magnitude of the data to be identified, stopping circulation and outputting a judgment result that the data to be identified is zero data;
if the corresponding preset identification magnitude is not reached, continuing the circulation process: as described above, the data to be identified is sector _0, and the data magnitude of sector _0 is 512B, which is not single-bit data, so that the corresponding dividing step is performed: if the result of the calculation is zero, the 64Bytes are continuously divided by 8Bytes, and so on, step-by-step recursion is performed to finally judge whether each bit in the single byte is zero;
this embodiment provides another specific example as follows:
if the input is 4k of data A;
a first round: splitting the data A into 8 512Bytes, and generating data B with a result of 512Bytes through the exclusive OR and the OR calculation; simultaneously taking one 512Bytes of data C, judging whether the data B and the data C are all 0, and if so, carrying out the next round;
and a second round: respectively splitting the data B/C into 8 64Bytes to generate data D with 64Bytes as a result; meanwhile, one data E of 64Bytes is taken, whether the data D and the data E are all 0 is judged, and if yes, the next round is carried out;
and a third round: splitting the data D/E into 8Bytes to generate data F with the result of 8 Bytes; meanwhile, one 8Bytes data G is taken, whether the data F and the data G are all 0 is judged, and if yes, the next round is carried out;
fourth wheel: splitting the data F/G into 8 1Bytes to generate data H with a result of 1 Byte; meanwhile, one 1byte data K is taken, whether the data H and the data K are all 0 is judged, and if yes, the next round is carried out;
and a fifth round: determine H, K if data is all 0 s;
after five rounds of operation are finished, returning the result whether the 8 512Bytes corresponding to the data A are equal or not; in the process of judging whether the data is zero data, the priority judgment can be carried out or whether the result data B is 0 (split to a single bit) is carried out, and if the result data B is 0, the priority judgment can be carried out to judge whether the data C is 0.
EXAMPLE III
Referring to fig. 2, a zero data identification apparatus includes:
the acquisition module is used for acquiring data to be identified;
the Hash calculation module is used for carrying out weak Hash calculation on the data to be identified;
the first judgment module is used for judging whether the hash value of the data to be identified is equal to the hash value of all-zero data or not, and if so, performing exclusive-or calculation on the data to be identified;
the exclusive or calculation module is used for carrying out exclusive or calculation on the data to be identified;
and the second judgment module is used for judging whether the XOR calculation result of the single-bit data in the data to be identified is zero, if so, the data to be identified is all-zero data, and if not, the data to be identified is non-all-zero data.
Example four
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of a method for zero data recognition as described in the first or second embodiment.
EXAMPLE five
Referring to fig. 3, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps of a zero data identification method as described in the first embodiment or the second embodiment.
In summary, according to the zero data identification method, the apparatus, the readable storage medium and the electronic device provided by the present invention, after acquiring the data to be identified, the weak hash calculation is performed on the data to be identified, the weak hash calculation value of the data to be identified is compared with the weak hash calculation value of all-zero data, when the calculation values are equal, the data to be identified is determined to be all-zero data, but the weak hash algorithm may have a data collision condition, i.e. non-zero data may also be identified as all-zero data, the data to be identified is subjected to the xor calculation to eliminate all-zero data with identification errors in the weak hash algorithm, and the data to be identified acquired from the memory is subjected to the stepwise recursion to determine whether the data is zero data layer by layer, and the whole determination process only involves xor and/or calculation, so that not only the zero data identification algorithm is simplified, but also whether the data is zero data can be accurately determined, thereby improving the performance of the storage; and can be implemented using hardware without incurring performance penalties for the SSD.
In the above embodiments provided in the present application, it should be understood that the disclosed method, apparatus, computer-readable storage medium, and electronic device may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical division, and other divisions may be realized in practice, for example, a plurality of components or modules may be combined or integrated into another apparatus, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or components or modules, and may be in an electrical, mechanical or other form.
The components described as separate parts may or may not be physically separate, and parts displayed as components may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the components can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing module, or each component may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It should be noted that, for the sake of simplicity, the above-mentioned method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present invention is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no acts or modules are necessarily required of the invention.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all equivalent changes made by using the contents of the present specification and the drawings, or applied directly or indirectly to the related technical fields, are included in the scope of the present invention.
Claims (9)
1. A zero data identification method, comprising the steps of:
acquiring data to be identified;
carrying out weak hash calculation on the data to be identified;
judging whether the hash value of the data to be identified is equal to the hash value of all-zero data or not, and if so, performing XOR calculation on the data to be identified;
judging whether the XOR calculation result of the single-bit data in the data to be identified is zero, if so, determining that the data to be identified is all-zero data, and if not, determining that the data to be identified is non-all-zero data;
the performing the xor calculation on the data to be identified includes:
judging whether the data to be identified is the single-bit data or not, if not, dividing the data to be identified into a plurality of groups of subdata with the same data volume;
combining a plurality of groups of the subdata, and carrying out XOR calculation on the combined subdata to obtain a plurality of groups of XOR calculation results;
carrying out OR calculation on a plurality of groups of XOR calculation results, if the result is zero, acquiring any subdata as the data to be identified, and returning to the step of judging whether the data to be identified is the single-bit data until the data to be identified is the single-bit data;
and if the result is not zero, the data to be identified is not zero data.
2. The method according to claim 1, wherein the dividing the data to be identified into a plurality of sub-data sets with the same data size comprises:
acquiring the current storage magnitude of the data to be identified;
obtaining the division quantity according to the quantity relation between the current storage magnitude and the next-stage storage magnitude corresponding to the current storage magnitude;
and dividing the data to be identified into a plurality of groups of subdata with the same data volume according to the division number.
3. The method of claim 1, wherein the combining the plurality of sub-data sets comprises:
establishing a logical relationship between each subdata and any other subdata;
and combining the subdata according to the logic relation.
4. The method of claim 3, wherein the combining the plurality of sub-data sets and performing XOR computation on the combined sub-data sets to obtain a result of the XOR computation comprises:
sequentially carrying out non-overlapping combination on two adjacent groups of subdata to obtain a plurality of groups of first combinations;
sequentially combining two groups of the sub data at intervals to obtain a plurality of groups of second combinations;
and calculating the XOR result of the first combination and the second combination to obtain a plurality of groups of XOR calculation results.
5. The method of claim 3, wherein the combining the plurality of sub-data sets and performing XOR computation on the combined sub-data to obtain a result further comprises:
sequentially overlapping and combining the two adjacent groups of subdata to obtain a third combination;
and calculating the XOR result of the plurality of groups of the third combinations to obtain a plurality of groups of the XOR calculation results.
6. The zero data identification method according to claim 1, wherein the weak hash calculation comprises: an information-digest algorithm 5 and a secure hash algorithm.
7. A null data recognition device, comprising:
the acquisition module is used for acquiring data to be identified;
the Hash calculation module is used for carrying out weak Hash calculation on the data to be identified;
the first judgment module is used for judging whether the hash value of the data to be identified is equal to the hash value of all-zero data or not, and if so, performing exclusive-or calculation on the data to be identified;
the exclusive or calculation module is used for carrying out exclusive or calculation on the data to be identified;
the second judgment module is used for judging whether the XOR calculation result of the single-bit data in the data to be identified is zero, if so, the data to be identified is all-zero data, and if not, the data to be identified is non-all-zero data;
the performing the xor calculation on the data to be identified includes:
judging whether the data to be identified is the single-bit data or not, if not, dividing the data to be identified into a plurality of groups of subdata with the same data volume;
combining a plurality of groups of the subdata, and carrying out XOR calculation on the combined subdata to obtain a plurality of groups of XOR calculation results;
carrying out OR calculation on a plurality of groups of XOR calculation results, if the result is zero, acquiring any subdata as the data to be identified, and returning to the step of judging whether the data to be identified is the single-bit data until the data to be identified is the single-bit data;
and if the result is not zero, the data to be identified is not zero data.
8. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of a method for zero data recognition according to any one of claims 1 to 6.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of a method for zero data recognition according to any one of claims 1 to 6 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210502419.1A CN114595486B (en) | 2022-05-10 | 2022-05-10 | Zero data identification method and device, readable storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210502419.1A CN114595486B (en) | 2022-05-10 | 2022-05-10 | Zero data identification method and device, readable storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114595486A CN114595486A (en) | 2022-06-07 |
| CN114595486B true CN114595486B (en) | 2022-08-05 |
Family
ID=81813590
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210502419.1A Active CN114595486B (en) | 2022-05-10 | 2022-05-10 | Zero data identification method and device, readable storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114595486B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5179711A (en) * | 1989-12-26 | 1993-01-12 | International Business Machines Corporation | Minimum identical consecutive run length data units compression method by searching consecutive data pair comparison results stored in a string |
| US9177177B1 (en) * | 2012-10-04 | 2015-11-03 | Symantec Corporation | Systems and methods for securing storage space |
| US10248623B1 (en) * | 2015-03-30 | 2019-04-02 | EMC IP Holding Company LLC | Data deduplication techniques |
| CN111459919A (en) * | 2020-04-26 | 2020-07-28 | 深圳佰维存储科技股份有限公司 | Data query method, loading assembly, search assembly and storage medium |
| CN112463032A (en) * | 2020-11-09 | 2021-03-09 | 苏州浪潮智能科技有限公司 | Performance optimization method, system and device for deduplication module of storage system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8443028B2 (en) * | 2009-06-02 | 2013-05-14 | Exelis Inc. | Circuits and methods for performing exponentiation and inversion of finite field elements |
| WO2019079358A1 (en) * | 2017-10-16 | 2019-04-25 | KOMENDA, J. Kyle | Density coordinate hashing for volumetric data |
-
2022
- 2022-05-10 CN CN202210502419.1A patent/CN114595486B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5179711A (en) * | 1989-12-26 | 1993-01-12 | International Business Machines Corporation | Minimum identical consecutive run length data units compression method by searching consecutive data pair comparison results stored in a string |
| US9177177B1 (en) * | 2012-10-04 | 2015-11-03 | Symantec Corporation | Systems and methods for securing storage space |
| US10248623B1 (en) * | 2015-03-30 | 2019-04-02 | EMC IP Holding Company LLC | Data deduplication techniques |
| CN111459919A (en) * | 2020-04-26 | 2020-07-28 | 深圳佰维存储科技股份有限公司 | Data query method, loading assembly, search assembly and storage medium |
| CN112463032A (en) * | 2020-11-09 | 2021-03-09 | 苏州浪潮智能科技有限公司 | Performance optimization method, system and device for deduplication module of storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114595486A (en) | 2022-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11403284B2 (en) | System for data sharing platform based on distributed data sharing environment based on block chain, method of searching for data in the system, and method of providing search index in the system | |
| CN110089035B (en) | Storage controller, data processing chip and data processing method | |
| EP0793174B1 (en) | Error detection and correction method and apparatus for computer memory | |
| US9507662B2 (en) | Expanded error correction codes | |
| US20200401480A1 (en) | Erasure Code-Based Data Processing Method and Apparatus | |
| CN114281793A (en) | Data verification method, device and system | |
| CN110750381A (en) | Error correction method and device based on NAND FLASH memory | |
| CN114138231B (en) | Method, circuit and SOC for executing matrix multiplication operation | |
| CN111475402A (en) | Program function testing method and related device | |
| CN114595486B (en) | Zero data identification method and device, readable storage medium and electronic equipment | |
| CN116049315A (en) | Block chain storage method, system, device and storage medium | |
| JP2000181807A (en) | Method and device for inspecting data in recording medium | |
| CN115421965A (en) | Consistency checking method and device, electronic equipment and storage medium | |
| CN106354581B (en) | A kind of cyclic redundancy check method and multi-core processor | |
| CN114564336A (en) | Data consistency checking method, device, equipment and storage medium | |
| WO2017166845A1 (en) | Data processing device and data processing method | |
| CN116048868A (en) | Code generation method, device, equipment and storage medium | |
| Vajha et al. | Small-d MSR codes with optimal access, optimal sub-packetization and linear field size | |
| WO2020088211A1 (en) | Data compression method and related apparatus, and data decompression method and related apparatus | |
| US10263638B2 (en) | Lossless compression method for graph traversal | |
| CN117079703B (en) | Method and device for testing embedded memory of chip and electronic equipment | |
| CN115114062B (en) | Fault detection method, device, equipment and storage medium for instruction word line | |
| CN108512554B (en) | Bandwidth optimization method and device of line-level layered decoding structure | |
| CN116662063B (en) | Error correction configuration method, error correction method, system, equipment and medium for flash memory | |
| CN115114062A (en) | Method, apparatus, device and storage medium for detecting fault of instruction word line |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |