CN112463214B - Data processing method and device, computer readable storage medium and electronic equipment - Google Patents
Data processing method and device, computer readable storage medium and electronic equipment Download PDFInfo
- Publication number
- CN112463214B CN112463214B CN201910848505.6A CN201910848505A CN112463214B CN 112463214 B CN112463214 B CN 112463214B CN 201910848505 A CN201910848505 A CN 201910848505A CN 112463214 B CN112463214 B CN 112463214B
- Authority
- CN
- China
- Prior art keywords
- stored
- data
- queried
- byte array
- code
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 claims abstract description 35
- 238000012545 processing Methods 0.000 claims abstract description 32
- 238000004364 calculation method Methods 0.000 claims abstract description 6
- 238000003491 array Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 7
- 239000002699 waste material Substances 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 230000006837 decompression Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 101100108191 Vibrio parahaemolyticus serotype O3:K6 (strain RIMD 2210633) add gene Proteins 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 101150060298 add2 gene Proteins 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
- G06F9/30003—Arrangements for executing specific machine instructions
- G06F9/3004—Arrangements for executing specific machine instructions to perform operations on memory
- G06F9/30043—LOAD or STORE instructions; Clear instruction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
- G06F7/60—Methods or arrangements for performing computations using a digital non-denominational number representation, i.e. number representation without radix; Computing devices using combinations of denominational and non-denominational quantity representations, e.g. using difunction pulse trains, STEELE computers, phase computers
- G06F7/72—Methods or arrangements for performing computations using a digital non-denominational number representation, i.e. number representation without radix; Computing devices using combinations of denominational and non-denominational quantity representations, e.g. using difunction pulse trains, STEELE computers, phase computers using residue arithmetic
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Mathematical Analysis (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- General Engineering & Computer Science (AREA)
- Computational Mathematics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention relates to a data processing method and device, a computer readable storage medium and electronic equipment, and relates to the technical field of data processing, wherein the method comprises the following steps: carrying out hash operation on data to be stored to obtain hash codes to be stored, which correspond to the data to be stored; performing redundancy calculation on the total number of nodes included in the hash code to be stored and the byte array to obtain a position code to be stored; and when the byte array corresponding to the to-be-stored position code is determined to be unoccupied, storing the to-be-stored data and the to-be-stored hash code into the byte array corresponding to the to-be-stored position code. The embodiment of the invention improves the utilization rate of the Java heap memory.
Description
Technical Field
The embodiment of the invention relates to the technical field of data processing, in particular to a data processing method, a data processing device, a computer readable storage medium and electronic equipment.
Background
In the data processing process, the value to be processed is often required to be stored in a Set container in a mode of increasing the value; the Set container is HashSet provided by JDK (Java Development Kit, development kit of Java language).
However, the above method has the following disadvantages: because the HashSet is realized based on HashSet, when the value to be processed is required to be stored in HashSet, the value to be processed is required to be stored in the node object, so that a node object is required to be created when one value to be processed is stored, the reference address of the node object is required to be stored, the node object and the reference address are irrelevant to the value to be processed, further, the waste of Java heap memory is caused, and the utilization rate of Java heap memory is reduced.
Therefore, it is desirable to provide a new data processing method and apparatus.
It should be noted that the information of the present invention in the above background section is only for enhancing the understanding of the background of the present invention and thus may include information that does not form the prior art that is already known to those of ordinary skill in the art.
Disclosure of Invention
The present invention is directed to a data processing method, a data processing apparatus, a computer readable storage medium, and an electronic device, and more particularly, to overcome at least some of the problems of low Java heap memory utilization caused by the limitations and disadvantages of the related art.
According to one aspect of the present disclosure, there is provided a data processing method including:
Carrying out hash operation on data to be stored to obtain hash codes to be stored, which correspond to the data to be stored;
performing redundancy calculation on the total number of nodes included in the hash code to be stored and the byte array to obtain a position code to be stored;
and when the byte array corresponding to the to-be-stored position code is determined to be unoccupied, storing the to-be-stored data and the to-be-stored hash code into the byte array corresponding to the to-be-stored position code.
In an exemplary embodiment of the present disclosure, storing the data to be stored and the hash code to be stored into a byte array corresponding to the position code to be stored includes:
dividing the byte array into a plurality of storage blocks with the same length;
and storing the data to be stored, the length value of the data to be stored and the hash code into each storage block respectively.
In an exemplary embodiment of the present disclosure, the data processing method further includes:
when the byte array corresponding to the position code to be stored is determined to be occupied, judging whether the stored hash code in the byte array corresponding to the position code to be stored is identical to the hash code to be stored or not;
When the stored hash code is determined to be the same as the hash code to be stored, storing the data to be stored into a next byte array corresponding to the position code to be stored;
and generating a logic linked list according to the position information of the next byte array, and storing the logic linked list into the byte array corresponding to the position code to be stored.
In an exemplary embodiment of the present disclosure, the data processing method further includes:
storing the data to be stored into a next byte array corresponding to the position code to be stored when the stored hash code is determined to be different from the hash code to be stored;
and storing the settlement identification corresponding to the data to be stored into the byte array corresponding to the position code to be stored.
According to one aspect of the present disclosure, there is provided a data processing method including:
carrying out hash operation on data to be queried to obtain hash codes to be queried, and carrying out residual operation on the hash codes to be queried and the total number of nodes included in a byte array to obtain position codes to be queried;
determining whether stored data in a byte array corresponding to the position code to be queried is identical to the data to be queried when determining that the stored hash code in the byte array corresponding to the position code to be queried is identical to the hash code to be queried;
And returning a query result corresponding to the data to be queried when the stored data is determined to be the same as the data to be queried.
In an exemplary embodiment of the present disclosure, the data processing method further includes:
when the stored data is different from the data to be queried, performing remainder operation on the stored hash codes and the total number of nodes to obtain a position code to be verified;
if the position code to be verified is consistent with the position code to be queried, acquiring target position information of the data to be queried from a byte array corresponding to the position code to be queried;
and returning a query result corresponding to the data to be queried when the stored data in the byte array corresponding to the target position information is confirmed to be the same as the data to be queried.
In an exemplary embodiment of the present disclosure, the data processing method further includes:
if the position code to be verified is inconsistent with the position code to be queried, acquiring a residence identifier of the data to be queried from a byte array corresponding to the position code to be queried;
judging whether the data to be queried is stored in a container corresponding to a byte array corresponding to the position code to be queried according to the residence identifier;
When the data to be queried is confirmed to be stored in a container corresponding to the byte array corresponding to the position code to be queried, matching the data to be queried with stored data in all byte arrays included in the container;
and returning a query result corresponding to the data to be queried when the data to be queried is successfully matched with stored data in all byte arrays included in the container.
According to an aspect of the present disclosure, there is provided a data processing apparatus comprising:
the first operation module is used for carrying out hash operation on data to be stored to obtain hash codes to be stored, which correspond to the data to be stored;
the second operation module is used for performing residual operation on the hash codes to be stored and the total number of nodes included in the byte array to obtain position codes to be stored;
and the first storage module is used for storing the data to be stored into the byte array corresponding to the position code to be stored when the byte array corresponding to the position code to be stored is determined to be unoccupied.
According to an aspect of the present disclosure, there is provided a data processing apparatus comprising:
the third operation module is used for carrying out hash operation on the data to be queried to obtain hash codes to be queried, and carrying out residual operation on the hash codes to be queried and the total number of nodes included in the byte array to obtain position codes to be queried;
The first judging module is used for determining whether the stored data in the byte array corresponding to the position code to be queried is identical to the data to be queried or not when the stored hash code in the byte array corresponding to the position code to be queried is identical to the hash code to be queried;
and the query result returning module is used for returning a query result corresponding to the data to be queried when the stored data is determined to be the same as the data to be queried.
According to one aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the data processing method of any one of the above.
According to one aspect of the present disclosure, there is provided an electronic device including:
a processor; and
a memory for storing executable instructions of the processor;
wherein the processor is configured to perform the data processing method of any of the above via execution of the executable instructions.
On one hand, hash operation is carried out on data to be stored to obtain hash codes to be stored corresponding to the data to be stored, and then surplus operation is carried out on the hash codes to be stored and the total number of nodes included in a byte array to obtain position codes to be stored; finally, when the byte array corresponding to the position code to be stored is unoccupied, the data to be stored and the hash code to be stored are stored in the byte array corresponding to the position code to be stored, so that the problems that in the prior art, a node object is required to be created and the reference address of the node object is required to be stored as a result of storing a value to be processed, the node object and the reference address are irrelevant to the value to be processed, and further Java heap memory is wasted, the utilization rate of the Java heap memory is reduced, the waste of the Java heap memory is reduced, and the utilization rate of the Java heap memory is improved are solved; on the other hand, obtaining the position code to be stored by performing redundancy operation on the total number of nodes included in the hash code to be stored and the byte array; when the byte array corresponding to the position code to be stored is unoccupied, the data to be stored and the hash code to be stored are stored in the byte array corresponding to the position code to be stored, so that the problems that a node object is required to be created when a value to be processed is stored in the prior art, the storage process is complicated and the storage efficiency is low are solved, the step of storing the data to be stored is simplified, and the storage efficiency is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention. It is evident that the drawings in the following description are only some embodiments of the present invention and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art.
Fig. 1 schematically shows a structural example diagram of a JDK native HashSet according to an exemplary embodiment of the present invention.
Fig. 2 schematically shows a flow chart of a data processing method according to an exemplary embodiment of the invention.
Fig. 3 schematically illustrates a structural example diagram of a lightweight hash association container according to an exemplary embodiment of the present invention.
Fig. 4 schematically illustrates an exemplary view of an application scenario of the data processing method illustrated in fig. 2 according to an exemplary embodiment of the present invention.
Fig. 5 schematically shows a flow chart of another data processing method according to an exemplary embodiment of the invention.
Fig. 6 schematically shows an exemplary diagram of an application scenario of the data processing method shown in fig. 5 according to an exemplary embodiment of the present invention.
Fig. 7 schematically shows an example diagram of an application scenario of another data processing method according to an example embodiment of the present invention.
Fig. 8 schematically shows a flow chart of another data processing method according to an exemplary embodiment of the invention.
Fig. 9 schematically illustrates an exemplary view of an application scenario of the data processing method illustrated in fig. 8 according to an exemplary embodiment of the present invention.
Fig. 10 schematically shows a flow chart of another data processing method according to an exemplary embodiment of the invention.
Fig. 11 schematically illustrates an exemplary view of an application scenario of the data processing method illustrated in fig. 10 according to an exemplary embodiment of the present invention.
Fig. 12 schematically shows a flow chart of another data processing method according to an exemplary embodiment of the invention.
Fig. 13 schematically illustrates an exemplary view of an application scenario of the data processing method illustrated in fig. 12 according to an exemplary embodiment of the present invention.
Fig. 14 schematically shows a block diagram of a data processing apparatus according to an example embodiment of the invention.
Fig. 15 schematically shows a block diagram of another data processing apparatus according to an example embodiment of the invention.
Fig. 16 schematically shows an electronic device for implementing the above-described data processing method according to an example embodiment of the invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known aspects have not been shown or described in detail to avoid obscuring aspects of the invention.
Furthermore, the drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor devices and/or microcontroller devices.
In the existing data processing process, a value using add (value) mode is often required to be stored in a Set container, and whether the value is contained in the Set is judged in a contains (value) mode later; the Set container commonly used in Java is HashSet provided by JDK, which has the advantage that the temporal complexity of the contacts operation is O (1).
However, the above approach has the following drawbacks: on the one hand, the HashSet100 is based on HashMap implementation, as JDK native. The data structure is [ object array object [ ] 101+linked list ], and the specific reference is shown in FIG. 1. Specifically, as shown in fig. 1, since the HashSet data structure is [ Object array+linked list ], and the value needs to be stored in the Node Object 102, storing a value needs to create a Node Object, and storing the reference address (8 bytes of each reference address) of the Node Object in the Object [ ], where the Node Object and the reference address are irrelevant to the value data, and the Java heap memory is wasted.
On the other hand, when using the Set container, most of storage types are simple data types (such as String, intel), for simple data types, real effective information is only a part of objects, for example, when storing int type data (4 byte), the int seal box needs to be stored as an intel object (12 byte), and when each Node object contains Value object, extra object description information and an object reference address (8 byte) stored in the Node object are wasted, after the storage type is converted into the object, the storage space of the JVM heap is far larger than metadata, and the multi-stored information is not effective information required by data processing, so that the waste of Java heap memory is caused.
In yet another aspect, the data structure of HashSet [ object array+linked list ] causes objects to be discretely distributed in the JVM heap memory, and when the value is determined to exist by using the containers method, the cache (L1, L2, L3) of the CPU is not fully utilized, resulting in a slower speed of data from the memory to the cache of the CPU.
In this exemplary embodiment, a data processing method is provided first, where the method may operate on a server, a server cluster, or a cloud server; of course, those skilled in the art may also operate the method of the present invention on other platforms as required, and this is not a particular limitation in the present exemplary embodiment. Referring to fig. 2, the data processing method may include the steps of:
s210, carrying out hash operation on data to be stored to obtain hash codes to be stored, which correspond to the data to be stored.
And S220, performing redundancy operation on the total number of nodes included in the hash code to be stored and the byte array to obtain a position code to be stored.
And S230, when the byte array corresponding to the position code to be stored is determined to be unoccupied, storing the data to be stored and the hash code to be stored into the byte array corresponding to the position code to be stored.
In the data processing method, on one hand, hash operation is performed on data to be stored to obtain hash codes to be stored corresponding to the data to be stored, and then surplus operation is performed on the hash codes to be stored and the total number of nodes included in the byte array to obtain position codes to be stored; finally, when the byte array corresponding to the position code to be stored is unoccupied, the data to be stored and the hash code to be stored are stored in the byte array corresponding to the position code to be stored, so that the problems that in the prior art, a node object is required to be created and the reference address of the node object is required to be stored as a result of storing a value to be processed, the node object and the reference address are irrelevant to the value to be processed, and further Java heap memory is wasted, the utilization rate of the Java heap memory is reduced, the waste of the Java heap memory is reduced, and the utilization rate of the Java heap memory is improved are solved; on the other hand, obtaining the position code to be stored by performing redundancy operation on the total number of nodes included in the hash code to be stored and the byte array; when the byte array corresponding to the position code to be stored is unoccupied, the data to be stored and the hash code to be stored are stored in the byte array corresponding to the position code to be stored, so that the problems that a node object is required to be created when a value to be processed is stored in the prior art, the storage process is complicated and the storage efficiency is low are solved, the step of storing the data to be stored is simplified, and the storage efficiency is improved.
Hereinafter, each step involved in the data processing method of the exemplary embodiment of the present invention will be explained and described in detail with reference to the accompanying drawings.
First, the LightHashSet to be used in the exemplary embodiment of the present invention will be explained and explained. Referring to fig. 3, in a LightHashSet (lightweight hash association container) 300, a plurality of Byte arrays Byte [ ]301 may be included, and a plurality of logical Node nodes 302 may be included in each Byte array 301; each of the logical Node nodes may be configured to store data to be stored (Value) 303, a data length of the data to be stored 304, (hashCode), storage location identification information 306 of the data to be stored, and a residence identifier 307 of the data to be stored.
For example, in fig. 3, 13 Node nodes may be included in a byte array, then a data length 304 of the data to be stored may be used in the Node 1, a hash code 305 of the data to be stored may be used in the Node 2-Node 4 to store the data to be stored (Value) 303, a storage location identification information (Next table) 306 of the data to be stored may be used in the Node 9-Node 12, and a residence identification 307 of the data to be stored may be used in the Node 13. It should be noted that the byte data may be divided into several storage blocks of equal length through a logical linked list, and each block is a Node. The number of nodes included in each byte data group can be determined according to the Value; the value may not occupy the entire space, and the length of the space is specified at the time of lightHashSet initialization.
In step S210, hash operation is performed on the data to be stored to obtain a hash code to be stored corresponding to the data to be stored.
In the present exemplary embodiment, the data type of the data to be stored may include, for example, an intelger (Integer) form, and may also include a String (String) form, which is not particularly limited in this example.
In step S220, a remainder operation is performed on the hash code to be stored and the total number of nodes included in the byte array to obtain a position code to be stored.
In the present exemplary embodiment, the remainder operation is also referred to as a modulo operation (mod); for example, assuming a and b are two integers, the result of a% b is the remainder of a/b. Specifically, 5%3 =2 (quotient 1 residual 2), 9%4 =1 (quotient 2 residual 1), 3%7 =3 (quotient 0 residual 3). Furthermore, by using the hash code hashCode to be stored to make a remainder on the total number of Node nodes, the position code to be stored can be determined according to the remainder operation result.
In step S230, when it is determined that the byte array corresponding to the to-be-stored position code is unoccupied, the to-be-stored data and the to-be-stored hash code are stored into the byte array corresponding to the to-be-stored position code.
In this example embodiment, after the above-mentioned to-be-stored position code is obtained, a byte array corresponding to the to-be-stored position code may be determined, and whether the byte data is in an occupied state may be determined; when the byte array is in the unoccupied state, the data to be stored and the hash code to be stored can be directly stored into the byte array. Specifically, storing the data to be stored and the hash code to be stored into the byte array may include: firstly, dividing the byte array into a plurality of storage blocks with the same length; and secondly, the data to be stored, the length value of the data to be stored and the hash codes are respectively stored into the storage blocks. As shown in fig. 4, when the hashCode% total node number=1 and the byte array corresponding to 1 is in the unoccupied state, the data to be stored may be directly stored into the byte array corresponding to 1; alternatively, when hashCode% total number of nodes=2 and when hashCode% total number of nodes=4, the storage may be performed directly.
The storage block is Node; when the byte array corresponding to the position code to be stored is unoccupied, the data to be stored can be considered to be stored in the byte array, so that the storage position identification information and the settlement identification of the data to be stored are not required to be stored in the last byte array corresponding to the byte array. It can be known that, when the position code of the next byte array corresponding to the byte array is exactly the same as the position code to be stored of other data to be stored, and the next byte array is in the unoccupied state, the storage block (Node) in the byte array for storing the storage position identification information and the residence identification can be in the empty state. However, since most of the byte arrays corresponding to the position codes to be stored are in an occupied state, it is necessary to configure a memory block for storing the storage position identification information and the residence identification.
Fig. 5 schematically illustrates another data processing method according to an example embodiment of the present disclosure. Referring to fig. 5, the data processing method may further include steps S510 to S530, which will be described in detail below.
In step S510, when it is determined that the byte array corresponding to the to-be-stored position code is occupied, it is determined whether the stored hash code in the byte array corresponding to the to-be-stored position code is identical to the to-be-stored hash code.
In step S520, when it is determined that the stored hash code is the same as the to-be-stored hash code, the to-be-stored data is stored into a next byte array corresponding to the to-be-stored position code.
In step S530, a logical linked list is generated according to the position information of the next byte array, and the logical linked list is stored into the byte array corresponding to the position code to be stored.
Hereinafter, step S510 to step S530 will be explained and explained. Referring to fig. 6, for example, when add1 operation is performed, data to be stored may be directly stored into a node array corresponding to hashCode% total number of nodes=1; when add2 operation is performed, the node array with hashCode% total number=1 is already occupied, so that the node array can be stored into the Next position, if the Next position is empty, the node array is stored in the Next position, the Next node in the home position stores the array index (position information) of the Next position, then a logic linked list can be generated according to the position information, and the logic linked list is stored in a byte array with hashCode% total number=1 pair, so that the query speed is improved. Also, since the data is 0 at the time of byte [ ] initialization, it collides with the 0 th tuple index, it can be forcibly specified that it is represented by-1 when next points to 0.
It should be further noted here that, since the data to be stored is an integer or a character string, the required storage block is fixed; the memory blocks required for their corresponding hash codes are also fixed (e.g., ha Xicun codes may be 400 in length); thus, in each byte array, the number of memory blocks required by the logical linked list may determine the total number of memory blocks included in each byte array. Thus, the above is committed to dividing each byte array into multiple memory blocks by a logical linked list.
Further, the data processing method may further include: storing the data to be stored into a next byte array corresponding to the position code to be stored when the stored hash code is determined to be different from the hash code to be stored; and storing the settlement identification corresponding to the data to be stored into the byte array corresponding to the position code to be stored. In detail:
referring to fig. 7, when the add operation 3 is required, it is found that the node array corresponding to hashCode% node total number=2 is occupied and the hashCode of the stored data is different from the hashCode corresponding to the add operation 3, so that the storing into the next position can be tried, if the next position is empty, the storing into the next position is performed, and the settlement identifier of the original position (i.e. byte array corresponding to the second hashCode% node total number=1) is set to 1, which proves that the position has been tried to store.
Fig. 8 schematically illustrates another data processing method according to an example embodiment of the present disclosure. Referring to fig. 8, the data processing method may include steps S810 to S830, which will be described in detail below.
In step S810, hash operation is performed on the data to be queried to obtain a hash code to be queried, and residual operation is performed on the hash code to be queried and the total number of nodes included in the byte array to obtain a position code to be queried.
In step S820, when it is determined that the stored hash code in the byte array corresponding to the position code to be queried is identical to the hash code to be queried, it is determined whether the stored data in the byte array corresponding to the position code to be queried is identical to the data to be queried.
In step S830, when it is determined that the stored data is identical to the data to be queried, a query result corresponding to the data to be queried is returned.
Hereinafter, the steps S810 to S830 will be explained. Referring to fig. 9, when hashCode% total number of nodes=2, it may be determined whether the hash code stored in the byte array is the same as the hash code to be queried; if so, determining whether the values are the same; after the values are the same, a true query result may be returned. It should be noted that the calculation process of the position code to be queried in step S810 is similar to the calculation process of the position code to be stored described above, and will not be repeated here.
Fig. 10 schematically illustrates another data processing method according to an example embodiment of the present disclosure. Referring to fig. 10, the data processing method may include steps S910 to S930, which will be described in detail below.
In step S1010, when it is determined that the stored data is different from the data to be queried, a remainder operation is performed on the stored hash code and the total number of nodes to obtain a position code to be verified.
In step S1020, if the position code to be verified is consistent with the position code to be queried, the target position information of the data to be queried is obtained from the byte array corresponding to the position code to be queried.
In step S1030, when it is determined that the stored data in the byte array corresponding to the target location information is identical to the data to be queried, a query result corresponding to the data to be queried is returned.
Hereinafter, step S1010 to step S1030 will be explained. Referring to fig. 11, when a query (containers) operation is performed, a position code to be queried is obtained using the same method as the stored procedure; when the comparison hashCode is the same but the value is different, the data in the node array corresponding to the array subscript (position information) stored in the array node is taken out for comparison, and when the node array with the same value is found, a true query result is returned. And when the next node corresponding to the position information is found to be 0, returning a false query result. Further, since the byte [ ] is initialized with data 0, it collides with the 0 th tuple index, and thus if-1 is taken, it is converted to 0 for subsequent operations.
Fig. 12 schematically illustrates another data processing method according to an example embodiment of the present disclosure. As shown in fig. 12, the data processing method may include steps S1210 to S1230, which will be described in detail below.
In step S1210, if the position code to be verified is inconsistent with the position code to be queried, the hosting identifier of the data to be queried is obtained from the byte array corresponding to the position code to be queried.
In step S1220, it is determined whether the data to be queried is stored in a container corresponding to the byte array corresponding to the position code to be queried according to the residence identifier.
In step S1230, when it is confirmed that the data to be queried is stored in the container corresponding to the byte array corresponding to the position code to be queried, the data to be queried is matched with stored data in all byte arrays included in the container.
In step S1240, when it is confirmed that the data to be queried successfully matches the stored data in all byte arrays included in the container, a query result corresponding to the data to be queried is returned.
Hereinafter, the steps S1210 to S1240 will be explained. Referring to fig. 13, when a query (containers) operation is performed, a position code to be queried is obtained using the same method as the stored procedure; judging whether the settlement mark stored in the byte array is 1 when the hashCode is different, and directly returning a false query result when the settlement mark is 0, which indicates that the true data owner at the position does not store the data owner; when the settlement mark is 1, the true data master of the position tries to store, the halfcode is compared to find downwards until the Node nodes with the same halfcode are found, whether the value is the same is judged, and a turn query result is returned when the value is the same.
The data processing method provided by the invention has at least the following advantages:
on one hand, the technical scheme of the invention solves the problem that JVM (Java Virtual Machine ) heap memory is excessively large when a large amount of data is stored in the HashSet container, and the problems of data storage (add) and low performance of judging whether the data exists (containers) or not.
On the other hand, the technical scheme of the invention uses java basic type byte as the bottom layer storage data structure of the LightHashSet container, thereby reducing the memory use of the JVM heap. In addition, because the data with the same value of HashCode% size is stored continuously as far as possible by using a logic linked list, the CPU cache utilization rate and the contans method efficiency can be improved during the contans processing.
In another aspect, the LightHashSet is a Set interface inheriting JDK native as the HashSet native to JDK, and can be applied to various application scenarios, and can be directly replaced in a scenario where the HashSet native to JDK has been used.
Furthermore, the user can design and realize a self-defined compression and decompression method by inheriting the abstract class mode to open the data compression and decompression interface, so that the memory use and lifting efficiency is further reduced.
Furthermore, the memory consumption of the container is reduced through a lightweight data structure (the memory usage is 50% of that of the HashSet which is native to JDK). And, the LightHashSet utilizes the related data stored in adjacent byte [ ] blocks, fully utilizes the CPU cache, and can achieve that the add/containers method is faster than the JDK native HashSet.
The disclosure also provides a data processing device. Referring to fig. 14, the data processing apparatus may include a first operation module 1410, a second operation module 1420, and a first storage module 1430. Wherein:
the first operation module 1410 may be configured to perform hash operation on data to be stored to obtain a hash code to be stored corresponding to the data to be stored.
The second operation module 1420 may be configured to perform a remainder operation on the hash code to be stored and the total number of nodes included in the byte array to obtain a position code to be stored.
The first storage module 1430 may be configured to store the data to be stored into the byte array corresponding to the position code to be stored when it is determined that the byte array corresponding to the position code to be stored is unoccupied.
In an exemplary embodiment of the present disclosure, storing the data to be stored and the hash code to be stored into a byte array corresponding to the position code to be stored includes:
dividing the byte array into a plurality of storage blocks with the same length; and storing the data to be stored, the length value of the data to be stored and the hash code into each storage block respectively.
In an exemplary embodiment of the present disclosure, the data processing apparatus further includes:
and the second judging module can be used for judging whether the stored hash codes in the byte array corresponding to the position codes to be stored are identical with the hash codes to be stored or not when the byte array corresponding to the position codes to be stored is occupied.
And the second storage module is used for storing the data to be stored into a next byte array corresponding to the position code to be stored when the stored hash code is determined to be the same as the hash code to be stored.
And the third storage module can be used for generating a logic linked list according to the position information of the next byte array and storing the logic linked list into the byte array corresponding to the position code to be stored.
In an exemplary embodiment of the present disclosure, the data processing apparatus further includes:
the fourth storage module is used for storing the data to be stored into a next byte array corresponding to the position code to be stored when the stored hash code is determined to be different from the hash code to be stored;
and the fifth storage module can be used for storing the residence identification corresponding to the data to be stored into the byte array corresponding to the position code to be stored.
The present disclosure also provides another data processing apparatus. Referring to fig. 15, the data processing apparatus may include a third operation module 1510, a first determination module 1520, and a first query result return module 1530. Wherein:
the third operation module 1510 may be configured to perform hash operation on data to be queried to obtain a hash code to be queried, and perform remainder operation on the hash code to be queried and the total number of nodes included in the byte array to obtain a position code to be queried.
The first determining module 1520 may be configured to determine, when it is determined that the stored hash code in the byte array corresponding to the position code to be queried is the same as the hash code to be queried, whether the stored data in the byte array corresponding to the position code to be queried is the same as the data to be queried.
The first query result returning module 1530 may be configured to return a query result corresponding to the data to be queried when it is determined that the stored data is the same as the data to be queried.
In an exemplary embodiment of the present disclosure, the data processing apparatus further includes:
and the fourth operation module can be used for performing residual operation on the stored hash codes and the total number of nodes to obtain the position codes to be verified when the stored data are determined to be different from the data to be queried.
The target position information acquisition module may be configured to acquire target position information of the data to be queried from a byte array corresponding to the position code to be queried if the position code to be verified is consistent with the position code to be queried.
And the second query result returning module is used for returning the query result corresponding to the data to be queried when the stored data in the byte array corresponding to the target position information is identical to the data to be queried.
In an exemplary embodiment of the present disclosure, the data processing apparatus further includes:
and the settlement identifier acquisition module can be used for acquiring the settlement identifier of the data to be queried from the byte array corresponding to the position code to be queried if the position code to be verified is inconsistent with the position code to be queried.
The third judging module can be used for judging whether the data to be queried is stored in a container corresponding to the byte array corresponding to the position code to be queried according to the residence identifier;
the matching module can be used for matching the data to be queried with stored data in all byte arrays included in a container when the data to be queried is confirmed to be stored in the container corresponding to the byte array corresponding to the position code to be queried;
and the third query result returning module is used for returning the query result corresponding to the data to be queried when the data to be queried is successfully matched with the stored data in all byte arrays included in the container.
The specific details of each module in the above data processing apparatus have been described in detail in the corresponding data processing method, so that the details are not repeated here.
It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functions of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the invention. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.
Furthermore, although the steps of the methods of the present invention are depicted in the accompanying drawings in a particular order, this is not required to either imply that the steps must be performed in that particular order, or that all of the illustrated steps be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform, etc.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present invention may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, a mobile terminal, or a network device, etc.) to perform the method according to the embodiments of the present invention.
In an exemplary embodiment of the present invention, an electronic device capable of implementing the above method is also provided.
Those skilled in the art will appreciate that the various aspects of the invention may be implemented as a system, method, or program product. Accordingly, aspects of the invention may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system.
An electronic device 1600 according to such an embodiment of the invention is described below with reference to fig. 16. The electronic device 1600 shown in fig. 16 is merely an example, and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in fig. 16, the electronic device 1600 is embodied in the form of a general purpose computing device. The components of the electronic device 1600 may include, but are not limited to: the at least one processing unit 1610, the at least one memory unit 1620, and a bus 1630 connecting the different system components (including the memory unit 1620 and the processing unit 1610).
Wherein the storage unit stores program code that is executable by the processing unit 1610 such that the processing unit 1610 performs steps according to various exemplary embodiments of the present invention described in the above-described "exemplary methods" section of the present specification. For example, the processing unit 1610 may perform step S210 as shown in fig. 2: carrying out hash operation on data to be stored to obtain hash codes to be stored, which correspond to the data to be stored; step S220: performing redundancy calculation on the total number of nodes included in the hash code to be stored and the byte array to obtain a position code to be stored; step S230: and when the byte array corresponding to the to-be-stored position code is determined to be unoccupied, storing the to-be-stored data and the to-be-stored hash code into the byte array corresponding to the to-be-stored position code.
The memory unit 1620 may include readable media in the form of volatile memory units, such as Random Access Memory (RAM) 16201 and/or cache memory 16202, and may further include Read Only Memory (ROM) 16203.
The storage unit 1620 may also include a program/utility 16204 having a set (at least one) of program modules 16205, such program modules 16205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
Bus 1630 may be a local bus representing one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or using any of a variety of bus architectures.
Electronic device 1600 may also communicate with one or more external devices 1700 (e.g., keyboard, pointing device, bluetooth device, etc.), as well as with one or more devices that enable a user to interact with the electronic device 1600, and/or with any device (e.g., router, modem, etc.) that enables the electronic device 1600 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 1650. Also, electronic device 1600 can communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet, through network adapter 1660. As shown, network adapter 1660 communicates with other modules of electronic device 1600 over bus 1630. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 1600, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present invention may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, a terminal device, or a network device, etc.) to perform the method according to the embodiments of the present invention.
In an exemplary embodiment of the present invention, a computer-readable storage medium having stored thereon a program product capable of implementing the method described above in the present specification is also provided. In some possible embodiments, the various aspects of the invention may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the invention as described in the "exemplary methods" section of this specification, when said program product is run on the terminal device.
A program product for implementing the above-described method according to an embodiment of the present invention may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
Furthermore, the above-described drawings are only schematic illustrations of processes included in the method according to the exemplary embodiment of the present application, and are not intended to be limiting. It will be readily appreciated that the processes shown in the above figures do not indicate or limit the temporal order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, among a plurality of modules.
Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
Claims (9)
1. A method of data processing, comprising:
carrying out hash operation on data to be stored to obtain hash codes to be stored, which correspond to the data to be stored;
performing redundancy calculation on the total number of nodes included in the hash code to be stored and the byte array to obtain a position code to be stored;
When the byte array corresponding to the position code to be stored is determined to be unoccupied, storing the data to be stored and the hash code to be stored into the byte array corresponding to the position code to be stored;
when the byte array corresponding to the position code to be stored is determined to be occupied, judging whether the stored hash code in the byte array corresponding to the position code to be stored is identical to the hash code to be stored or not;
when the stored hash code is determined to be the same as the hash code to be stored, storing the data to be stored into a next byte array corresponding to the position code to be stored;
and generating a logic linked list according to the position information of the next byte array, and storing the logic linked list into the byte array corresponding to the position code to be stored.
2. The data processing method according to claim 1, wherein storing the data to be stored and the hash code to be stored into a byte array corresponding to the position code to be stored includes:
dividing the byte array into a plurality of storage blocks with the same length;
And storing the data to be stored, the length value of the data to be stored and the hash code into each storage block respectively.
3. The data processing method according to claim 1, characterized in that the data processing method further comprises:
storing the data to be stored into a next byte array corresponding to the position code to be stored when the stored hash code is determined to be different from the hash code to be stored;
and storing the settlement identification corresponding to the data to be stored into the byte array corresponding to the position code to be stored.
4. A method of data processing, comprising:
carrying out hash operation on data to be queried to obtain hash codes to be queried, and carrying out residual operation on the hash codes to be queried and the total number of nodes included in a byte array to obtain position codes to be queried;
determining whether stored data in a byte array corresponding to the position code to be queried is identical to the data to be queried when determining that the stored hash code in the byte array corresponding to the position code to be queried is identical to the hash code to be queried;
Returning a query result corresponding to the data to be queried when the stored data is determined to be the same as the data to be queried;
when the stored data is different from the data to be queried, performing remainder operation on the stored hash codes and the total number of nodes to obtain a position code to be verified;
if the position code to be verified is consistent with the position code to be queried, acquiring target position information of the data to be queried from a byte array corresponding to the position code to be queried;
and returning a query result corresponding to the data to be queried when the stored data in the byte array corresponding to the target position information is confirmed to be the same as the data to be queried.
5. The data processing method according to claim 4, characterized in that the data processing method further comprises:
if the position code to be verified is inconsistent with the position code to be queried, acquiring a residence identifier of the data to be queried from a byte array corresponding to the position code to be queried;
judging whether the data to be queried is stored in a container corresponding to a byte array corresponding to the position code to be queried according to the residence identifier;
When the data to be queried is confirmed to be stored in a container corresponding to the byte array corresponding to the position code to be queried, matching the data to be queried with stored data in all byte arrays included in the container;
and returning a query result corresponding to the data to be queried when the data to be queried is successfully matched with stored data in all byte arrays included in the container.
6. A data processing apparatus, comprising:
the first operation module is used for carrying out hash operation on data to be stored to obtain hash codes to be stored, which correspond to the data to be stored;
the second operation module is used for performing residual operation on the hash codes to be stored and the total number of nodes included in the byte array to obtain position codes to be stored;
the first storage module is used for storing the data to be stored into the byte array corresponding to the position code to be stored when the byte array corresponding to the position code to be stored is determined to be unoccupied;
the second judging module is used for judging whether the stored hash codes in the byte array corresponding to the position codes to be stored are identical to the hash codes to be stored or not when the byte array corresponding to the position codes to be stored is occupied;
The second storage module is used for storing the data to be stored into a next byte array corresponding to the position code to be stored when the stored hash code is determined to be the same as the hash code to be stored;
and the third storage module is used for generating a logic linked list according to the position information of the next byte array and storing the logic linked list into the byte array corresponding to the position code to be stored.
7. A data processing apparatus, comprising:
the third operation module is used for carrying out hash operation on the data to be queried to obtain hash codes to be queried, and carrying out residual operation on the hash codes to be queried and the total number of nodes included in the byte array to obtain position codes to be queried;
the first judging module is used for determining whether the stored data in the byte array corresponding to the position code to be queried is identical to the data to be queried or not when the stored hash code in the byte array corresponding to the position code to be queried is identical to the hash code to be queried;
the query result returning module is used for returning a query result corresponding to the data to be queried when the stored data is determined to be the same as the data to be queried;
The fourth operation module is used for performing remainder operation on the stored hash codes and the total number of nodes to obtain the position codes to be verified when the stored data are determined to be different from the data to be queried;
the target position information acquisition module is used for acquiring the target position information of the data to be queried from a byte array corresponding to the position code to be queried if the position code to be verified is consistent with the position code to be queried;
and the second query result returning module is used for returning a query result corresponding to the data to be queried when the stored data in the byte array corresponding to the target position information is confirmed to be the same as the data to be queried.
8. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the data processing method of any of claims 1-5.
9. An electronic device, comprising:
a processor; and
a memory for storing executable instructions of the processor;
wherein the processor is configured to perform the data processing method of any of claims 1-5 via execution of the executable instructions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910848505.6A CN112463214B (en) | 2019-09-09 | 2019-09-09 | Data processing method and device, computer readable storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910848505.6A CN112463214B (en) | 2019-09-09 | 2019-09-09 | Data processing method and device, computer readable storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112463214A CN112463214A (en) | 2021-03-09 |
| CN112463214B true CN112463214B (en) | 2023-11-03 |
Family
ID=74807452
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910848505.6A Active CN112463214B (en) | 2019-09-09 | 2019-09-09 | Data processing method and device, computer readable storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112463214B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7403137B1 (en) * | 2007-03-30 | 2008-07-22 | Juniper Networks, Inc. | Memory efficient indexing for disk-based compression |
| CN102968498A (en) * | 2012-12-05 | 2013-03-13 | 华为技术有限公司 | Method and device for processing data |
| CN106164898A (en) * | 2014-10-11 | 2016-11-23 | 华为技术有限公司 | Data processing method and device |
| WO2018099107A1 (en) * | 2016-12-02 | 2018-06-07 | 深圳市中兴微电子技术有限公司 | Hash table management method and device, and computer storage medium |
| CN109144712A (en) * | 2017-06-19 | 2019-01-04 | 北京信威通信技术股份有限公司 | Memory pool building, memory allocation method and device |
| CN110046040A (en) * | 2019-04-09 | 2019-07-23 | 厦门网宿有限公司 | Distributed task scheduling processing method and system and storage medium |
| CN110162523A (en) * | 2019-04-04 | 2019-08-23 | 阿里巴巴集团控股有限公司 | Date storage method, system, device and equipment |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060248079A1 (en) * | 2005-04-28 | 2006-11-02 | Freescale Semiconductor Incorporated | Method and apparatus for finding a perfect hash function and making minimal hash table for a given set of keys |
| US8793257B2 (en) * | 2009-05-24 | 2014-07-29 | Roger Frederick Osmond | Method for improving the effectiveness of hash-based data structures |
| US8868926B2 (en) * | 2012-04-06 | 2014-10-21 | Exablox Corporation | Cryptographic hash database |
-
2019
- 2019-09-09 CN CN201910848505.6A patent/CN112463214B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7403137B1 (en) * | 2007-03-30 | 2008-07-22 | Juniper Networks, Inc. | Memory efficient indexing for disk-based compression |
| CN102968498A (en) * | 2012-12-05 | 2013-03-13 | 华为技术有限公司 | Method and device for processing data |
| CN106164898A (en) * | 2014-10-11 | 2016-11-23 | 华为技术有限公司 | Data processing method and device |
| WO2018099107A1 (en) * | 2016-12-02 | 2018-06-07 | 深圳市中兴微电子技术有限公司 | Hash table management method and device, and computer storage medium |
| CN109144712A (en) * | 2017-06-19 | 2019-01-04 | 北京信威通信技术股份有限公司 | Memory pool building, memory allocation method and device |
| CN110162523A (en) * | 2019-04-04 | 2019-08-23 | 阿里巴巴集团控股有限公司 | Date storage method, system, device and equipment |
| CN110046040A (en) * | 2019-04-09 | 2019-07-23 | 厦门网宿有限公司 | Distributed task scheduling processing method and system and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 基于一致哈希算法的用电信息采集系统研究;邹保平;;电力信息化(第06期);全文 * |
| 基于改进型MBF的命名数据网PIT存储结构研究;许亚平;李卓;刘开华;马东来;杨奕康;;重庆邮电大学学报(自然科学版)(第01期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112463214A (en) | 2021-03-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111090628A (en) | Data processing method and device, storage medium and electronic equipment | |
| CN110688096B (en) | Method and device for constructing application program containing plug-in, medium and electronic equipment | |
| US11321090B2 (en) | Serializing and/or deserializing programs with serializable state | |
| CN114564509B (en) | Serialization method, deserialization method, device, chip, and medium | |
| US11784946B2 (en) | Method for improving data flow and access for a neural network processor | |
| CN112558984A (en) | Code compiling method and device, electronic equipment and server | |
| US20200371827A1 (en) | Method, Apparatus, Device and Medium for Processing Data | |
| US10133713B2 (en) | Domain specific representation of document text for accelerated natural language processing | |
| CN109033456B (en) | Condition query method and device, electronic equipment and storage medium | |
| CN111198880A (en) | Data storage method and device based on redis and electronic equipment | |
| CN112463214B (en) | Data processing method and device, computer readable storage medium and electronic equipment | |
| CN116225419A (en) | Code-free development method, device, storage medium and equipment for back-end transaction processing | |
| CN113127430A (en) | Mirror image information processing method and device, computer readable medium and electronic equipment | |
| CN116346777A (en) | Dictionary tree domain name matching method, device, equipment and storage medium | |
| CN107320959B (en) | Game role identification information generation method, device, medium and electronic equipment | |
| US11573899B1 (en) | Transparent interleaving of compressed cache lines | |
| CN113141407B (en) | Page resource loading method and device and electronic equipment | |
| CN111949648A (en) | Memory cache data system and data indexing method | |
| CN111753548B (en) | Information acquisition method and device, computer storage medium and electronic equipment | |
| US11797277B2 (en) | Neural network model conversion method server, and storage medium | |
| CN118277686A (en) | Page rendering method and device and electronic equipment | |
| CN111562940B (en) | Project data construction method and device | |
| CN110727391A (en) | Transaction data storage method and device, block chain system, medium and electronic equipment | |
| US11687449B2 (en) | Concurrent marking garbage collection | |
| EP4191407A1 (en) | Method and device for generating and registering ui service package and loading ui service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |