KR102597640B1 - Optimized hopscotch multiple hash tables for efficient memory in-line deduplication application - Google Patents

Abstract

In the memory deduplication method, each of the plurality of hash tables corresponds to a hash function, each of the plurality of hash tables consists of physical hash buckets, and each physical bucket consists of ways and stores data. Identifying a plurality of hash tables, each of a plurality of virtual buckets consisting of a portion of the physical hash buckets and at least one of the physical hash buckets and another virtual bucket among the plurality of virtual buckets. Identifying a plurality of shared virtual buckets, identifying each of the plurality of physical buckets having data allocated to and stored in the corresponding virtual bucket among the virtual buckets, generating a hash value Hashing a data line according to a corresponding hash function among the hash functions, determining whether the corresponding virtual bucket among the virtual buckets of the hash table has available space for a data block according to the hash value. determining, when the corresponding virtual bucket among the virtual buckets does not have available space until the corresponding virtual bucket among the virtual buckets has space for the data block, from the corresponding virtual bucket among the virtual buckets It may include sequentially moving data to adjacent virtual buckets among virtual buckets, and storing the data block in the corresponding virtual bucket.

Description

{OPTIMIZED HOPSCOTCH MULTIPLE HASH TABLES FOR EFFICIENT MEMORY IN-LINE DEDUPLICATION APPLICATION}

The present invention relates to Hapscotch multiple hash tables, and more specifically, to Hapscotch multiple hash tables optimized for memory efficiency that immediately processes deduplication applications.

Data deduplication refers to reducing unnecessary data in a memory device to reduce the capacity cost of the memory device. In data deduplication, a data object/item (e.g., a data file) is divided into one or more data lines/chunks/blocks. By associating multiple data blocks comprised of the same data into one stored data block, redundant copies of data blocks can be reduced or eliminated by the computer memory, thereby reducing the overall amount of redundant blocks of data in the memory device. Reducing unnecessary copies of data can increase read speeds and memory bandwidth, and potentially result in power savings.

Accordingly, if redundant data copies can be reduced to a single data copy, the overall usable capacity of the memory device is increased while using the same amount of physical resources. As a result, economical use of memory devices enables a reduction in the number of data rewrites, and write requests for duplicate data blocks already stored in memory can be discarded, making it possible to use memory devices with data deduplication applied. Lifespan can be extended for a long time by effectively increasing write endurance.

A common data deduplication methodology is a CPU-centric approach that can use in-memory deduplication technology in which the deduplication engine is integrated into the CPU or memory controller (MC). This methodology allows duplicates to be recognized by the CPU processor and provides deduplication memory operations (e.g., Content lookups, Reference Count Updates, etc.) under the control of the memory controller. Implement a Deduplicated Cache (DDC) that works with the memory controller to try. The deduplication methodology is also a cache that caches conversion lines to increase data reads by removing conversion fetches from critical paths, and a direct conversion buffer (which may be similar to a lookaside buffer). Direct Translation Buffer (DTB) can be implemented.

Deduplication is usually used for hard drives. However, there is interest in providing fine-grained deduplication in areas of volatile memory such as dynamic random access memory (DRAM).

The information described above in this background art is merely to aid understanding of the invention and may therefore include information that does not constitute prior art.

The present invention is intended to solve the above-described technical problem, and the present invention can provide a plurality of hash tables optimized for efficient memory that immediately processes deduplication applications.

Aspects of embodiments of the present invention relate to memory deduplication in dynamic random access memory (DRAM) systems.

According to an embodiment of the present invention, a memory deduplication method is provided, wherein each of a plurality of hash tables corresponds to a hash function, each of the plurality of hash tables consists of physical hash buckets, and each physical hash bucket is provided. Identifying a plurality of hash tables, wherein the bucket is composed of waders and stores data, each of the plurality of virtual buckets is composed of a portion of the physical hash buckets and is at least similar to another virtual bucket among the plurality of virtual buckets. Identifying a plurality of virtual buckets sharing one of the physical hash buckets. Identifying each of the plurality of physical buckets having data allocated and stored in a corresponding virtual bucket among the virtual buckets, hashing a data line according to a corresponding hash function among the hash functions to generate a hash value, Determining whether a corresponding virtual bucket among the virtual buckets of the hash table has available space for a data block according to the hash value, when the corresponding virtual bucket among the virtual buckets has space for the data block sequentially moving data from a corresponding virtual bucket among the virtual buckets to an adjacent virtual bucket among the virtual buckets when a corresponding virtual bucket among the virtual buckets does not have available space, and a corresponding virtual bucket among the virtual buckets It may include storing the data block in a bucket.

The method may further include updating an address lookup table memory to change one or more lookup addresses corresponding to the moved data block.

Each of the plurality of hash tables may further include a reference count line, a signature line, and a sumword line.

The method may further include generating a sumword vector to represent the physical hash buckets containing data corresponding to the virtual bucket.

The step of generating the sumword vector includes, for each of the plurality of virtual buckets, each of the plurality of physical hash buckets of the relative virtual bucket among the plurality of virtual buckets. It may include using a binary indicator to indicate whether it contains a data block associated with the virtual bucket.

The method uses log2(H) bits for each physical hash bucket to indicate which of the plurality of physical hash buckets contains data corresponding to a certain virtual bucket among the plurality of virtual buckets. A step of generating a composed sumword value may be further included.

The step of generating the sumword value consists of pseudo-addresses for each of the plurality of physical hash buckets containing data in a location expressed in association with pairs of the plurality of physical hash buckets and virtual buckets. It may include generating a two-dimensional array.

The plurality of hash tables may be stored in volatile memory.

The volatile memory may include dynamic random access memory (DRAM).

The method may further include indexing the hash table with a physical line ID (PLID) consisting of a virtual bucket usage value field of log2(H) bits, where the value corresponds to a corresponding virtual bucket among the plurality of virtual buckets. It may be the same as the number of data blocks in the bucket.

The method may further include increasing the virtual bucket usage value field by 1 when writing an item to the corresponding virtual bucket among the plurality of virtual buckets.

The method may further include storing the data block in a buffer memory when the corresponding hash table is full.

According to an embodiment of the present invention, a deduplication DRAM memory module is provided for reducing duplication of data blocks in a memory for deduplication memory, and the deduplication DRAM memory module includes a hash table memory for storing deduplicated data blocks, an address lookup table memory for storing deduplicated data blocks and corresponding addresses, and receiving read requests to cause the deduplicated DRAM memory module to retrieve data blocks from the hash table memory, and the deduplicated DRAM memory. A module may include a processor that receives write requests to store the data blocks in the hash table memory.

The hash table memory may be composed of a three-dimensional array of hash tables stored in the hash table memory, each of the hash tables composed of physical hash buckets, and each physical hash bucket composed of waders. and the deduplicated data blocks can be stored.

Each of the hash tables further includes a plurality of virtual buckets, and each virtual bucket may be composed of two or more physical hash buckets.

The deduplication DRAM memory module may move the deduplicated data blocks between adjacent virtual buckets among the plurality of virtual buckets within a corresponding hash table.

The deduplication DRAM memory module can perform data deduplication without externally provided commands.

The deduplication DRAM memory module may further include a buffer memory for storing data when the hash table memory is full.

According to an embodiment of the present invention, a deduplication DRAM memory module is provided for reducing duplication of data blocks in a memory for deduplication memory, and the deduplication DRAM memory module includes a plurality of hash tables, each of which has physical hash buckets. Each physical hash bucket is composed of waders and stores the data blocks, and includes a three-dimensional array of hash tables stored in the deduplication DRAM memory module, a processor, and memory, wherein the memory contains instructions. and when the instructions are executed by the processor, the memory causes the deduplication DRAM memory module to move a pre-stored deduplicated data block between adjacent virtual buckets of a hash table among the plurality of hash tables. Each of the virtual buckets may be composed of two or more physical hash buckets.

The memory further stores instructions, and when the instructions are executed by the processor, the memory causes a deduplication DRAM memory module to store instructions in one of the virtual buckets from which the prestored deduplicated data block is moved. This may cause the incoming data to be stored in .

According to the deduplication DRAM memory module according to an embodiment of the present invention, memory accesses can be reduced and the lifespan of the DRAM system can be extended.

These and other aspects of the invention may be understood by reference to the specification, claims, and accompanying drawings.
1 is a block diagram of a deduplication DRAM system architecture according to an embodiment of the present invention.
Figure 2 is a block diagram of memory types within the deduplication memory module of the embodiment of Figure 1.
Figure 3 is a block diagram of a hash table of the hash table memory of the embodiment of Figure 2.
Figure 4 is a block diagram of a plurality of hash table arrays according to an embodiment of the present invention.
5A, 5B, and 5C depict two-dimensional arrays for generating Hopwords for associating virtual buckets with specific physical buckets according to embodiments of the present invention.
Figure 6 is a block diagram of a physical line ID (PLID) for addressing data blocks of a hash table memory according to an embodiment of the present invention.
Figure 7 is a flowchart showing a process of writing data to a plurality of hash table arrays of a memory module using the Hapscotch method according to an embodiment of the present invention.
Figure 8 is a flowchart showing a process of reading data from a plurality of hash table arrays of a memory module according to an embodiment of the present invention.

The features of the present invention and how to achieve them will become clear by referring to the detailed description of the embodiments and the accompanying drawings. Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings, where like reference numerals designate like elements. However, the present invention may be implemented in many different forms and is not limited to the embodiments merely illustrated herein. Rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the nature and functionality of the invention to those skilled in the art. Accordingly, processes, elements, and techniques that are not necessary for a person skilled in the art to fully understand the features and functions of the present invention may not be described. Unless specifically noted, like reference numbers indicate like elements in the accompanying drawings and written description, and thus description thereof will not be repeated. In the drawings, the relative sizes of components, layers and regions may be exaggerated for clarity.

Although the terms first, second, third, etc. are used herein to describe various elements, components, regions, layers and/or sections, these elements, components, regions, layers and/or sections Alternatively, the sections will not be understood as being limited by these terms. These terms are used to distinguish one element, component, region, layer, or section from another element, component, region, layer, or section. Accordingly, a first component, component, region, layer, or section described below may refer to a second component, component, region, layer, or section without departing from the spirit and scope of the present invention.

“Under”, “below”, “lower”, “particular part” or “particular part” are used to facilitate description to describe an element or its characteristic relationship with other component(s) or feature(s) shown in the drawing. Spatial and relative terms such as “below”, “above”, and “above” may be used here. It will be understood that spatial and relative terms are intended to include other orientations of the device in use or operation in addition to the orientation depicted in the drawings. For example, if the device in the drawing is turned over, components described as “below” or “under” or “particularly below” other components or features will be positioned “above” the other components or features. You lose. Accordingly, example terms “below” or “below a particular portion” may include both upward and downward directions. Devices may be adjusted differently (eg, rotated 90 degrees in different directions) and spatially relative descriptors should be interpreted accordingly.

When an element, layer, region, or component is referred to as “to,” “coupled to,” or “connected to” another element, layer, region, or component, it refers to “the other element, layer, region, or component.” “, “directly coupled to,” “directly connected to,” or there may be one or more intervening elements, layers, regions, or components. Additionally, when an element or layer is referred to as between two elements or layers, it may be that only the element or layer is between the two elements or layers, or one or more intervening elements or layers may also be present. .

In the following examples, the x-axis, y-axis, and z-axis are not limited to the three axes of the rectangular coordinate system and can be interpreted in a broad sense. For example, the x-axis, y-axis, and z-axis may be orthogonal to each other, or may represent different directions that are not orthogonal to each other.

The terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the invention. As used herein, the singular forms “a” and “a” are intended to include the plural forms as well, unless the context clearly dictates otherwise. When the terms “consisting of,” “consisting of,” “comprising,” and “comprising” are used herein, such terms refer to defined features, integers, steps, operations, elements, and /or specifies the presence of components, but does not preclude the addition or presence of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. As used herein, the term “and/or” includes any and all combinations associated with one or more listed items. Expressions like “at least one” modify the entire list of elements and do not modify individual elements of the list.

As used herein, the terms “generally,” “about,” and similar terms are used as terms of approximation and not as terms of degree. This is to take into account inherent variations in measured or calculated values as recognized by those skilled in the art. Additionally, the use of “may” when describing embodiments of the invention means “one or more embodiments of the invention.” As used herein, the terms “use,” “used,” and “used” may be considered synonyms for the terms “use,” “used,” and “utilized,” respectively. Additionally, the term “exemplary” means an example or illustration.

Certain embodiments may be implemented differently, and certain process sequences may be performed differently from the described sequence. For example, two sequential processes described may alternatively be performed simultaneously or may be performed in the reverse order from that described.

Electronic or electrical devices and/or any other related devices or elements according to embodiments of the invention described herein may include any suitable hardware, firmware (e.g., Application Specific Integrated Circuit; ASIC), software, Alternatively, it may be implemented using a combination of software, firmware, and hardware. For example, the various elements of these devices may be formed from a single integrated circuit (IC) chip or from separate IC chips. Additionally, various elements of these devices may be implemented on a flexible printed circuit film, a tape carrier package (TCP), a printed circuit board (PCB), or formed on a single substrate. Additionally, various elements of these devices may be processes or threads that execute on one or more processors or on one or more computing devices that execute computer program instructions and interact with other system elements to perform the various functions described herein. It can be (Thread).

Computer program instructions are stored in memory implemented in a computing device using standard memory devices such as random access memory (RAM), for example. Computer program instructions may also be stored on, for example, a CD-ROM, a flash drive, or other non-transitory computer readable media. Additionally, those skilled in the art will understand that, without departing from the spirit and scope of the exemplary embodiments of the present invention, the functions of various computing devices may be integrated or integrated into a single computing device, and the functions of a particular computing device may be integrated into one or more other computing devices. It must be recognized that it may be distributed across .

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification have the same meaning as commonly understood by a person skilled in the art to which the present invention pertains. These terms, as defined in commonly used dictionaries, should be construed to have meanings consistent with their meanings in the context of this specification and/or related art and, unless expressly defined herein, not in an idealized or overly formal sense. It should not be interpreted.

1 is a block diagram of a deduplication DRAM system architecture according to an embodiment of the present invention.

Referring to Figure 1, in order to function as a computer memory, deduplication memory uses a function known as “translation” to record the relationship between the content of the original data and the set of unique deduplicated memory blocks. is performed, and the recorded relationships are recorded in compressed form. For example, addresses of original data may be stored in a lookup table.

In general, it is difficult for the processor 110 of the CPU to directly access the physical memory (e.g., the deduplication DRAM memory module 130), and the above-described access requires an array of memory lines. ) is instead managed by the memory controller 120. CPU-centric deduplication systems rescue cache data inside the CPU before the data reaches the memory system.

The deduplication DRAM system architecture (100) according to an embodiment of the present invention uses memory-centered deduplication rather than the conventional CPU-centered deduplication, and the deduplication DRAM memory module 130 is a processor. This means performing memory deduplication without commands from (110). The deduplication DRAM system architecture 100 also uses a configurable deduplication algorithm stored in the deduplication DRAM memory module 130 to increase the capacity gain of the memory, thereby providing a high capacity memory solution. That is, unlike CPU-centered deduplication, the deduplication DRAM system architecture 100 according to an embodiment of the present invention provides all deduplication intelligence ( Intelligence). Accordingly, deduplication can be performed within the deduplication DRAM memory module 130 without the CPU module 140 knowing, thereby increasing the capacity of the deduplication DRAM memory module 130. That is, since deduplication is fine-grained and operates within volatile memory (e.g., deduplication DRAM memory module 130), all deduplication information according to an embodiment of the present invention is redundant. The deduplication occurs within the DRAM memory module 130 itself, and on the other hand, the kernel module (140) within the CPU may not know the details of the deduplication operations performed within the deduplication DRAM memory module 130.

Although embodiments of the present invention have been described using DRAM as the deduplication DRAM memory module 130, it should be understood that other types of memory may be used in other embodiments of the present invention. Additionally, the deduplication DRAM system architecture 100 according to an embodiment of the present invention is capable of supporting various types of memory and interfacing. That is, the deduplication DRAM memory module 130 according to an embodiment of the present invention may be associated with various other types of memory interfaces through the memory controller 120 (for example, DDR4 (Double Data Rate Fourth-Generation Synchronous Dynamic Random-Access Memory), Peripheral Component Interconnect Express (PCIe), a serial expansion bus standard for connecting a computer and one or more peripheral devices, DDR-T, and KTI). Accordingly, it should be noted that other architectures may be used to integrate deduplication DRAM memory module 130 into deduplication DRAM system architecture 100.

Additionally, in order to implement embodiments of the present invention, there may be some changes to the existing DRAM memory module (e.g., driver upgrade), but the software implementation is based on the operating system (OS)/CPU. The deduplication DRAM system architecture 100 according to an embodiment of the present invention can be used without physically changing the module 140 or the processor 110.

The deduplication DRAM system architecture 100 according to an embodiment of the present invention includes deduplication, content addressability, security, processor-in-memory (PIM), and associated addresses. For DRAM's intelligent protocols such as RAS (Row Address Strobe), which is a signal sent to DRAM to inform DRAM that the data bit in DRAM is a row address stored in a cell located at the intersection of column address and row address. A System on Chip (SoC) can be implemented in the deduplication DRAM memory module 130.

Deduplication DRAM system architecture 100 also allows processor 110 to implement virtual density management, smart data placement, and DRAM intelligent application programming interfaces (APIs) associated with memory controller 120. , etc. may have Smart System Software that causes it to be possible.

The deduplication DRAM memory module 130 has various form factors (e.g., DIMM (Dual In-line Memory Module), 2.5In, FHHL (Full Height Half Length), HHHL (Half Height Half Length), It may have additional 3DS DRAM components such as high-capacity DRAM memory modules (Full Height Full Length (FHFL), etc.).

Therefore, by providing a memory-centric deduplication system using the deduplication DRAM system architecture 100 according to an embodiment of the present invention, the deduplicate write process can be performed directly at the memory interface. There is, and by doing so, the capacity of the deduplication DRAM memory module 130 is increased.

FIG. 2 is a block diagram of memory types in the deduplication DRAM memory module of the embodiment of FIG. 1, and FIG. 3 is a block diagram of a hash table of the hash table memory of the embodiment of FIG. 2.

Referring to FIG. 2, the deduplication DRAM memory module 130 according to an embodiment of the present invention may have a deduplication algorithm architecture, where the memory area inside the deduplication memory DRAM module 130 is divided into three different areas. It is classified as The three other areas are Address Lookup Table Memory (ALUTM, 210) to indicate the location where deduplicated blocks of data are stored, and a hash to store deduplicated data blocks. It includes a table memory (Hash Table Memory, 220), and an Overflow/Buffer Memory (230) for storing data when the hash ways of the hash table are full.

When a block of data is input to the deduplication DRAM memory module 130, the deduplication algorithm is used to determine whether the data block is new or a data block without any corresponding address in the ALUTM 210 that has not been previously stored. It can work. To perform this operation, the deduplication algorithm will access ALUTM 210. To ensure that identical data blocks are stored as only one entry, a pointer (e.g., a Physical Line ID (PLID), as described in Figure 5 below) within ALUTM 210 is stored in the hash table memory. The same data block storage location is indicated at 220. That is, the ALUTM 210 is a storage device for locations (eg, addresses) associated with a lookup address mapping pointer (eg, PLID) in a hash table. Therefore, if a data block has previously been stored in hash table memory 220, it is possible for a pointer in ALUTM 210 to point to an address in hash table memory 220 where the same data block is stored, thereby creating a redundant copy of the data block. The need to store is eliminated, and the capacity of the deduplication DRAM memory module 130 is increased.

Referring to Figure 3, memory deduplication uses a relatively efficient but simple multi-way hash table/hash array to ensure a high level of deduplication and, correspondingly, a large memory capacity of the deduplication DRAM memory module 130. (Multiple-way Hash Table/Hash Array, 380) can be used. The hash table memory 220 of the deduplicated DRAM memory module 130 according to an embodiment of the present invention has one or more hash tables 380 and is used for their usefulness in determining whether a data block is unique. do. The hash table 380 can be thought of as a two-dimensional array consisting of hash buckets (rows, 310) and hash ways (columns, 320). That is, the hash table 380 according to an embodiment of the present invention includes m hash buckets 310 in rows, and each hash bucket 310 has data lines/slots indicating the capacity of the hash bucket 310. Contains n columns of fields/items/hash wads 320 (m and n are integers).

Data blocks are stored in the hash ways 320 of the hash table memory 220, and the address pointers of the ALUTM 210 contain values representing a specific hash bucket 310 and a specific hash way 320 associated with a specific data block. You can save it. Accordingly, an address (e.g., a 64-bit address) can be indexed into the ALUTM 210, and from there, the associated hash of the hash bucket 310 of the hash table 380 storing the data block corresponding to the address. Way 320 may be determined.

Therefore, during a write process (e.g., writing 64-bytes of data), a write request (i.e., a request to write incoming data consisting of one or more data blocks) is received. Then, a hash value is calculated for the incoming data using a hash function/hash algorithm (i.e., the incoming data is “hashed”) so that the corresponding hash bucket 310 and hash way 320 can be determined. . Accordingly, the hash value indicates where the data block is placed, or, if the data block (e.g., a 64-byte data block) is redundant, the hash value indicates where the data block is already stored in the hash table memory 220. Indicates the location. As data content is added to memory, some of the m hash buckets 310 may reach saturation first. Accordingly, the deduplication DRAM memory module 130 includes an overflow provision that uses the buffer memory 230 to store data blocks that cannot enter the hash table memory 220. The Original Lookup Address can then be retrieved and ALUTM 210 updated according to the lookup address calculated from hashing the incoming data.

During an attempted write process, when all of the hash waddles 320 are determined to be full, the buffer memory 230 may be used. That is, when the hash table 380 is full, data may be placed in a non-deduplicated overflow region of the buffer memory 230, thereby reducing the level of deduplication. Accordingly, buffer memory 230 is essentially a reserved, standard, simple excess memory area and serves as a SOC memory buffer/cache to implement virtual density over-provision management overflow. . Once data is placed in buffer memory 230, it can no longer be hashed and can no longer be deduplicated.

When a computer application attempts to store the same sequence of values in memory multiple times, multiple items in the translation array stored in ALUTM 210 reference the same address of a block of data stored in hash table memory 220, where: The items of ALUTM 210 are smaller than the original unique data blocks, thereby allowing efficient compression to be achieved.

Each of the m hash buckets 310 has a reference/frequency count line 340 containing a unique identifier to indicate the corresponding hash way 320 of the hash bucket 310, and a signature line 330. It may further include. For each hash bucket 310, the corresponding signature line 330 includes either 0 to indicate a free line or a non-zero secondary hash value for content search optimization. Therefore, for content retrieval, typically either there is no signature match requiring that an empty line be assigned based on the zero occurrence of the signature line, or one signature match such that subsequent data line reads and content comparisons confirm the presence of a duplicate. exists. Each of the m hash buckets 310 may further include a hopword line described in FIGS. 5A, 5B, and 5C below.

The physical line ID (PLID, 350) can be used to index data into the hash table (380). PLID 350 may be used to identify distinct memory lines in either ALUTM 210, hash table memory 220, or buffer memory 230. Each memory line is a data line for storing unique content in the hash table 380, or stores each PLID 350 and deduplicated data of the hash table 380 from the processor bus address. It may be related to any one of the translation lines to provide mapping to blocks. That is, the bus address identifies the conversion line, which in turn further identifies the item within the conversion line containing the associated PLID 350 that specifies the specific data line. Accordingly, the PLID 350 may be implemented to include an overflow tag, a specific hash table 350, corresponding hash bucket bits, and a corresponding tag indicating the location of the data block corresponding to the PLID 350. It may contain data representing way bits.

Each hash bucket 310 has an associated hash function/hash algorithm “h(x)”, which is an algorithm that generates a log2(m)-bit hash used to index data into hash buckets 310 ( For example, if hash table 380 has eight physical hash buckets 310, the hash function of hash table 380 will generate a 3-bit hash.) In other words, the hash function h(x) makes it possible to input a relatively large amount of input data (e.g., an input data file stored in memory) into the hash function h(x), and output data (e.g., Significantly different small quantities of hash values) are generated and output by the hash function h(x) to be stored in the hash table 380. Therefore, the hash function h(x) enables compression because different data sets can sometimes hash to the same hash value.

In writing to deduplicated memory, after receiving a write request corresponding to a data file, the deduplicated memory first determines whether an identical/duplicate data block is already stored in the hash table 380. Perform a duplicate search for Then, the deduplicated memory updates the items of the ALUTM (210) and the hash table memory (220). For example, reference/frequency count line 340 may be updated by updating the frequency count of the original search address (i.e., decremented by 1) in hash table memory 220, and when the reference count reaches 0, The corresponding data block is deleted. In addition, a new PLID 350 is created in ALUTM 210.

During duplicate search, which may be referred to as content search, deduplication DRAM memory module 130 finds pre-existing instances of a data file or portion thereof that is intended to be written. If there is an existing instance of data stored in hash table memory 220, duplicate search returns a PLID 350 pointing to that data line. If an existing instance of the data is not found, a new data line for that data block is created by allocating space in the hash table 380, writing the content there, and returning a new PLID 350. Content can be written by storing PLID 350 in ALUTM 210 at an offset determined by the bus address.

To insert data line “C” into hash table 380, the corresponding hash function “h(C)” of C can be calculated with a mathematical operation. When a hash function is computed for data line C, the rows of the hash table T(h(C)) are This can be checked by a content search operation (to see if the corresponding data line C already exists).

As described above, each hash bucket 310 of hash table 380 additionally includes a signature line 330 and a reference/frequency count line 340, and the signatures 332 of the signature line 330 and Due to the fact that the reference counts 342 of the reference/frequency count line 340 can be designed to be small enough to fill each hash bucket 310 with multiple quantities, the signature line 330 and the reference/frequency count line ( 340) Each can occupy only one hash way 320. That is, in the hash table 380, one entire row of the hash table 380 may be assigned to signature lines 330 belonging to each hash bucket 310, and one entire row may be assigned to each hash bucket 310. Reference/frequency count lines 340 belonging to 310 may be assigned.

As actual data blocks such as data line “C” are added to the hash table 380, the hash table 380 stores the corresponding PLID 350 stored in the ALUTM 210 in the hash table 380 of each individual deduplicated data line. ) By matching my address, data that can be accessed later begins to fill in. The address within the hash table 380 is determined by identifying a specific hash bucket 310 and a specific hash way 320 where the data is located (e.g., identifying the rows and columns of the hash table 380). can be identified. Accordingly, for each data block stored in hash table 380, there are one or more corresponding addresses identified by corresponding PLIDs 380 stored in ALUTM 210 that point to the location of the data block. Once the hash table 380 is filled with data, newly introduced data is placed in the non-deduplicated excess area/buffer memory 230, thereby reducing the level of deduplication.

On a read from deduplicated memory, the deduplicated memory returns a copy of either the data line from hash table memory 220 or the excess lines from buffer memory 230. For example, when stored data is read, after receiving a read request, corresponding addresses in hash table 380 are searched using PLIDs 350 stored in ALUTM 210. Then, blocks corresponding to each address are searched and reassembled.

Figure 4 is a block diagram of a plurality of hash table arrays according to an embodiment of the present invention.

Referring to FIG. 4, the deduplication DRAM system architecture according to an embodiment of the present invention uses a hash table array 400 composed of multiple hash tables (MHT, 480), each of a plurality of Hash tables 480 include m hash buckets 410, and each hash bucket 410 includes n hash ways 420. Although embodiments of the invention describe hash tables 480 and hash buckets 410 as being constant in size (e.g., with m and n described as integers), in other embodiments , different hash tables within the same plurality of hash table arrays may have different numbers of hash buckets, and similarly, within the plurality of hash table arrays, different hash buckets, or even within the same hash table, different numbers of hash buckets. You can have hash waddles. Furthermore, even if a plurality of hash tables 480 are used collectively, in some respects, different hash tables 480 are independent of each other (e.g., different hash tables 480 are independent of each other). You can have different hash functions, or you can have a common hash function).

The hash table array 400 includes k parallel hash tables (T ₁ , T ₂ , …, T _k , k is an integer), and each hash table 480 has separate, independent hash functions ( When using h ₁ (x), h ₂ (x), …, h _k (x)), each hash table (T ₁ , T ₂ , …, T _k ) has m hash buckets ( 410), the hash functions h ₁ (x), h ₂ (x), ..., h _k (x)) still produce hashes of log2(m)-bits, and each physical bucket Since 410 includes n hash ways 420, the capacity of a three-dimensional (3D) hash table array (eg, an array of multiple hash tables) is mxnxk.

Each hash table 480 may correspond to one hash function that determines how the data is indexed. By hashing the incoming data to be written, the resulting computation (e.g., a hash value containing the search address and key) can be compared, and if the values match, the corresponding hash bucket ( The reference/frequency count line 340 of 410 is incremented, thereby indicating that the additional PLID 350 of ALUTM 210 points to that particular line.

Unlike conventional hash tables, the plurality of hash tables 480 according to an embodiment of the present invention includes a plurality of virtual hash buckets/virtual buckets (VB, 460), and virtual buckets 460 ) consists of a plurality of physical hash buckets/physical buckets 410. Hereinafter, “physical bucket” will refer to the above-described hash buckets 310 and will be used to distinguish the above-described hash buckets 310 from virtual buckets 460.

Each virtual bucket 460 may include H of m physical buckets 410 of the hash table 480, and H is an integer smaller than m. However, it should be noted that other of the virtual buckets 460 in the same hash table 480 may share one or more physical buckets 410. As will be described later, by using virtual buckets 460 according to an embodiment of the present invention, a fourth dimension is added to the three-dimensional multiple hash table array. Accordingly, great flexibility can be provided in placing and sorting data, thereby increasing the efficiency and compression ratio of the deduplication DRAM system architecture.

An embodiment of the present invention uses virtual buckets 460 to increase another level of data placement flexibility, to free up other physical buckets 410 from being shared by other virtual buckets 460: A data block stored in one of the hash tables 480 may be moved within the corresponding virtual bucket 460 or to another physical bucket 410. By freeing up space within hash table 480, deduplication can be achieved by removing useless/duplicate data. That is, by using the virtual buckets 460 according to an embodiment of the present invention, there are no strict restrictions caused by hashing a data line using a hash function to a limited location, and the data is stored in a “near location.” It is possible to place a hash bucket 410 in a hash bucket 410 , which represents a physical bucket 410 that exists within the same virtual bucket 460 that contains the initially intended (but occupied) physical hash bucket 410 .

In one example, the content (e.g., data line C) is stored in k hash tables T ₁ (h ₁ (C)), T ₂ (h ₂ (C)), ..., T _k (h _k ( C))) may be placed in any one of the physical buckets 410. If data line C wishes to be placed at T1(h1(C)), instead of requesting that data line C be placed into the physical bucket 410 indicated by T1(h1(C)), the method of the present invention An embodiment is a virtual bucket ( 460) is allowed. That is, the virtual bucket 460 has T ₁ (h ₁ (C)), T ₁ (h ₁ (C)+1), T ₁ (h ₁ (C)+2), … , T ₁ (h ₁ (C)+H-1).

Accordingly, virtual buckets 460 allow blocks of data to be moved within hash table 480 or free up space for future write operations. The operation of an embodiment of the invention to enable movement of data blocks previously entered into hash table 480 (within virtual buckets 460 including physical buckets 410 of hash table 480) includes It can be expressed as Hopscotch. The Hapscotch operation using a plurality of hash tables 480 for memory deduplication can be improved as described later.

First, deduplication DRAM module 130 may attempt to insert data line C into hash table 480 as a result of the hash function of hash table 480. However, often other data lines may have previously entered hash table 480 as a result of the same hash function. That is, different data lines, despite being different, may be sent to the same location in hash table 480 as a result of the hash function. To determine where data line C is inserted, the operation may first find the first available physical bucket 410 in (or subsequent to) the physical bucket 410, represented as T(h(C)). there is.

Therefore, in determining where to spend data line C, the initially intended physical bucket 410, represented as T(h(C)), may be occupied, and thus the first available physical bucket 410, i.e. , the first empty space into which a data line can be inserted) can be expressed as T(h(C)+f), where f is greater than or equal to 0. Assuming that the physical bucket 410 expressed as T(h(C)) is the first physical bucket 410 of the H physical buckets 410 of the corresponding virtual bucket 460, if f is greater than H If it is small (i.e., if there is an unoccupied physical bucket 410 within the same virtual bucket 460), data line C may be placed into the corresponding virtual bucket 460. Similarly, if physical bucket 410, expressed as T(h(C)), is the second physical bucket of the corresponding virtual bucket 460, then if f is less than H-1, then data line C is in the corresponding virtual bucket 460. It can be placed at (460).

However, assuming that the first physical bucket 410 of the virtual bucket 460 is the intended physical bucket 410, if f is greater than or equal to H (i.e., the first physical bucket 410 of the virtual bucket 460 (410), even if data line C does not fit into its virtual bucket 460, the operation is to create empty space in the virtual bucket 460 in the following way. You can try. For example, the deduplication DRAM memory module 130 according to an embodiment of the present invention determines whether the physical bucket 410 represented by T(h(C)+f-1) has data contained therein. You can find the physical buckets 410 starting from the physical bucket 410, expressed as T(h(C)+f-H), and then T(h(C)+f-H+ You can find the physical bucket 410 represented by 1), and so on (for example, you can scan the virtual bucket 460 from start to finish). Then, the deduplication DRAM memory module 130 selects any data object contained within the physical buckets 410 from T(h(C)+f-H) to T(h(C)+f-1). It can be determined whether this empty space T(h(C)+f) can be placed. That is, the deduplication DRAM memory module 130 determines which of the physical buckets from T(h(C)+f-H) to T(h(C)+f-1) is the physical bucket T(h(C) +f) can be determined whether it is within a common virtual bucket 460, thereby allowing the data contained therein to be moved. Deduplication DRAM memory module 130 may then place the earliest such data object found into the empty space, thereby expressing it as T(h(C)+e, where e is an integer less than f). New empty space within the physical buckets 410 can be created. This process can be repeated until e becomes smaller than H (e.g., data can be moved within a hash table in a cascading fashion), thereby allowing The space required to place data line C can be secured.

For example, referring to FIG. 5B, in an embodiment of the present invention, physical bucket PB2 may be allocated as the intended physical bucket 410. Since the intended physical bucket PB2 is occupied in association with virtual bucket VB1, virtual bucket VB2 can be scanned from beginning to end (e.g., from physical bucket PB2 to physical bucket PB5). Since physical buckets PB3, PB4, and PB5 are also occupied, the first available physical bucket 410 is physical bucket PB6 (i.e., f is equal to 4, and f is therefore equal to H or greater than H, and the first available physical bucket 410 is not in that virtual bucket VB2.) Therefore, data in physical bucket PB5 can be moved to physical bucket PB6, thereby creating a virtual bucket. To reserve space in VB2, data line C can be placed in the corresponding virtual bucket VB2 (in the physical bucket PB5). However, if the intended physical bucket was PB1 (i.e., the corresponding virtual bucket 460 is VB1) , the process can be repeated so that the data in physical bucket PB4 can be moved from virtual bucket VB1 to adjacent virtual bucket VB2, i.e., into the newly freed space in physical bucket PB5. Afterwards, data line C is transferred to the intended It can be written to physical bucket PB4 of virtual bucket VB1, which corresponds to physical bucket PB1.

Accordingly, due to the common ownership of certain physical buckets 410 by other virtual buckets 460, which may be considered duplicates of other virtual buckets 460, data is transferred from one virtual bucket 460 to another virtual bucket 460. Bucket 460 may be moved, thereby creating space for the initial hash bucket 410.

In another embodiment, during a write process, after receiving a request to write a data block to the array of hash tables 400, the deduplication DRAM memory module 130 determines whether the existing entry is already in one of the hash tables 480. You can search the entire virtual bucket 460 of each hash table for data that is meaningful to you. If the first intended hash table 480 is full, and a data block cannot be found in the first intended hash table 480 (i.e., each hash way ( 420 is occupied by another data block), deduplication DRAM memory module 130 may seek to input data into another hash table 480 of hash table array 400. However, when all hash tables 480 of the plurality of hash table arrays 400 are full, data blocks may spill over into the buffer memory 230. In this embodiment, data movement within the hash table array 400 may not be permitted by the deduplication DRAM memory module 130.

5A, 5B, and 5C depict two-dimensional arrays for generating Hopwords for associating virtual buckets with specific physical buckets according to embodiments of the present invention.

5A, 5B, and 5C, according to an embodiment of the present invention, various virtual buckets 460 use either a sumword value 591 or a sumword vector 592, and Data movement can be associated with their corresponding physical buckets 410 by using virtual bucket usage values to effectively track data movement. Since each occupied physical bucket 410 may correspond to only one virtual bucket 460, the sumword value 591 or sumword vector 492 determines which virtual bucket 460 corresponds to each occupied physical bucket. It can be used to track whether it corresponds to (410).

In the example of the present invention, each of the four virtual buckets VB0, VB1, VB2, and VB3 has four adjacent physical buckets from the group of physical buckets PB0, PB1, PB2, PB3, PB4, PB5, and PB6. has a set (i.e. H is 4).

For example, referring to FIGS. 5A and 5B, sumword vector 592 is a two-dimensional array consisting of physical bucket locations and virtual bucket locations (e.g., quasi-addresses). creating a 1 in each physical bucket 410 containing data for each virtual bucket 460, indicating that there is only one 1 in any column corresponding to that physical bucket 410. It can be determined by placing (e.g. a binary indicator). Accordingly, sumword vector 592 may include an array of 1s or 0s that can be used to track physical bucket usage for each virtual bucket 460. In an example of the invention, physical buckets PB0, PB1, and PB3 are occupied for the first virtual bucket VB0, physical buckets PB2 and PB4 are occupied for the second virtual bucket VB1, and only the physical bucket PB5 is occupied for the third virtual bucket VB2, and the fourth virtual bucket VB3 is unoccupied.

Similarly, referring to Figure 5C, sumword value 591 is generated based on occupied physical buckets 410 by knowing which virtual buckets 460 correspond to occupied physical buckets. It can be. The sumword value 591 can be log2(H) length bits (H is the number of physical hash buckets 410 per virtual bucket 460).

Information on the sumword vector 592 or sumword value 591 may be stored in the sumword line of each hash bucket 410, and the relationship between the physical buckets 410 and the virtual buckets 460 is Can be indexed into memory.

Figure 6 is a block diagram of a physical line ID (PLID) for addressing data blocks of a hash table memory according to an embodiment of the present invention.

Referring to Figure 6, according to an embodiment of the present invention, a modified PLID 650 is provided. PLID 650 according to an embodiment of the present invention includes addresses, offsets, table index, hash, and slot/way, and a specific virtual bucket (460) for tracking moving items between virtual buckets 460. It may include a plurality of bits indicating each key 651 paired with 460). Accordingly, if key 651 matches a particular virtual bucket 460, then that particular virtual bucket 460 can contain the data item written there.

However, in another embodiment, PLID 650 replaces key 651 with a virtual bucket usage value field 652 (e.g., virtual bucket index) consisting of log2(H) bits (e.g. , a virtual bucket with a height of 16 physical buckets corresponds to the 4-bit virtual bucket usage value field of the PLID (650). The virtual bucket usage value field 652 indicates which virtual bucket 460 corresponds to each occupied physical bucket 410. Accordingly, when writing a data item to virtual bucket 460, the number of items already existing in virtual bucket 460 can be calculated, and a P value equal to the number of items already existing in the virtual bucket plus 1. may be written as the virtual bucket usage value 652. By using the virtual bucket usage value 652 of the PLID 650, the storage overload of the PLID 650 can be reduced.

Figure 7 is a flowchart showing a process of writing data to a plurality of hash table arrays of a memory module using the Hapscotch method according to an embodiment of the present invention.

Referring to Figure 7, in operation S701, a plurality of hash tables can be identified, each of the hash tables corresponds to a hash function, and each includes physical hash buckets, and each physical hash bucket has a hash way. and configured to store data (e.g., the deduplication DRAM memory module 130 may identify k hash tables 480, each corresponding to a hash function h(x), each includes m physical hash buckets 410, and each physical hash bucket includes n hash ways 420.

At step S702, a plurality of virtual buckets may be identified, each of the virtual buckets including a portion of physical hash buckets, and each sharing at least one physical hash bucket with another virtual bucket (e.g. , the deduplication DRAM memory module 130 may identify a plurality of virtual buckets 460, each of the virtual buckets 460 including H of m physical hash buckets 410, and As shown in Figure 4, each virtual bucket 460 shares at least one physical hash bucket 410 with other virtual buckets 460.). In step S702a, the plurality of virtual buckets have a virtual bucket usage value field of log2(h) bits, and a physical line ID (PLID) containing a value equal to the number of data blocks in the corresponding one of the virtual buckets. ), and incrementing the virtual bucket usage value field by 1 when an entry is written to the corresponding one of the virtual buckets (e.g., shown in Figure 6 As described above, the virtual buckets 460 have a virtual bucket usage value field 652, and a physical line ID containing a value equal to the number of data blocks in the corresponding virtual bucket of the virtual buckets 460. It can be identified by indexing the hash table 480 with (PLID, 650), and the virtual bucket usage value field 652 indicates that an item or data block will be written to the corresponding one of the virtual buckets 460. When 1 can be increased).

In step S703, each of the physical hash buckets having data stored in the physical hash buckets may be identified as being assigned to a corresponding one of the virtual buckets (e.g., a deduplication DRAM memory module ( 130) is a physical hash bucket 410 with data stored in PB0, PB1, PB2, PB3, PB4, and PB5, as shown in FIGS. 55B and 5C, and virtual buckets 460, VB0, VB1, And it can be identified by assigning it to a corresponding virtual bucket in VB2). In step S703a, it may be identified by generating a sumword vector or sumword value to indicate which of the physical hash buckets containing data corresponds to which of the virtual buckets (e.g., Figure As shown in FIGS. 5B and 5C, the deduplication DRAM memory module 130 uses a sum to indicate which of the physical hash buckets 410 containing data corresponds to which of the virtual buckets 460. A word vector (592) or a sumword value (591) can be generated).

In step S704, the data line may be hashed according to a corresponding one of the hash functions to generate a hash value (e.g., the deduplication DRAM memory module 130 may be connected to the memory controller 120. A write request corresponding to data line C may be received from , and the incoming data may be hashed according to a corresponding one of the hash functions h(x) to generate a hash value. ).

In step S705, it may be determined whether a corresponding one virtual bucket among the virtual buckets of the hash table has available space for a data block according to the hash value (e.g., as shown in FIGS. 5B and 5C) As shown, the deduplication DRAM memory module 130 may determine whether the virtual bucket 460 (VB3) has space in the physical bucket (PB6) for a data block.

In step S706, the data is stored in the corresponding one of the virtual buckets if the corresponding one of the virtual buckets does not have space available until the corresponding one of the virtual buckets has space for a data block. It may be sequentially moved from one virtual bucket to an adjacent one of the virtual buckets (e.g., as shown in FIGS. 5B and 5C, the deduplication DRAM memory module 130 is connected to the virtual bucket VB2. ) to transfer data from the physical bucket (PB5) of the virtual bucket (VB2) to the virtual bucket (VB3) if the virtual bucket (VB2) does not have any other available physical buckets until ) has space for the data block. It can be moved sequentially, and the above-described process is, if the virtual bucket (VB1) is a corresponding virtual bucket among the virtual buckets 460, the virtual bucket (VB2) is moved from the physical bucket (PB4) of the virtual bucket (VB1). can be repeated to move data into the physical bucket (PB5). In step S706a, the address lookup table memory may be updated to change one or more lookup addresses corresponding to the moved data block (e.g., the deduplication DRAM memory module 130 may use the hash table memory 220 ALUTM 210 may be updated to change one or more address pointers corresponding to the moved data block so that the new address of the moved data block can be retrieved.).

In step S707, the data block may be stored in a corresponding one of the virtual buckets (e.g., as shown in FIGS. 5B and 5C, the deduplication DRAM memory module 130 is If (VB1) is the intended virtual bucket 460, the data block can be stored in the physical bucket (PB4) of the virtual bucket (VB1). ). If the hash table 480 including the virtual bucket VB1 is determined to be full, the data block may be stored in the buffer memory 230.

Figure 8 is a flowchart showing a process of reading data from a plurality of hash table arrays of a memory module according to an embodiment of the present invention.

In step S801, a read request corresponding to a plurality of data blocks stored in the hash table array may be received (for example, the deduplication DRAM memory module 130 is configured with a data line C from the memory controller 120). A read request corresponding to a plurality of data blocks may be received, and the plurality of data blocks are stored in the hash table array 400 of the hash table memory 220.

In step S802, the corresponding pointers among the pointers corresponding to the plurality of data blocks may be retrieved from the ALUTM 210 (for example, the deduplication DRAM memory module 130 may retrieve data line C from the ALUTM 210). Address pointers corresponding to a plurality of data blocks composed of can be searched.).

In step S803, a plurality of data blocks based on corresponding pointers among the pointers may be accessed from the hash table memory (for example, the deduplication DRAM memory module 130 may access the hash table of the hash table memory 220. Data blocks can be accessed and retrieved from different addresses corresponding to the retrieved address pointers in array 400.).

In step S804, the plurality of data blocks may be reassembled to generate reassembled data (e.g., the deduplication DRAM memory module 130 may be equivalent to the data line C corresponding to the received read request. Data blocks retrieved from the hash table memory 220 can be reassembled to generate reassembled data.

In step S805, the reassembled data may be transmitted from the memory module to the memory controller (for example, the deduplication DRAM memory module 130 may transmit data line C to the memory controller 120).

As described above, data deduplication can be performed using a deduplication DRAM memory module according to an embodiment of the present invention. Accordingly, memory accesses can be reduced and the lifespan of the DRAM system can be extended.

The foregoing illustrates exemplary embodiments and is not limited thereto. Although several exemplary embodiments have been described, those skilled in the art will readily understand that various modifications may be made in the exemplary embodiments without departing from the novel teachings and advantages of the exemplary embodiments. Accordingly, all such modifications are intended to be included within the scope of the exemplary embodiments recited in the claims. In the claims, functional clauses are intended to include equivalent structures as well as structures that perform the recited function and structural equivalents. Therefore, it is to be understood that the foregoing is not limited to the specific embodiments disclosed, and that variations of the disclosed exemplary embodiments, as well as other exemplary embodiments, are intended to be included within the scope of the appended claims. The invention is defined by the following claims, within which equivalents of the claims are included.

100: Deduplication DRAM system architecture
110: processor
120: Memory/PCIe/KIT controller
130: Deduplication DRAM memory module
140: operating system
210: Address search table memory
220: Hash table memory
230: buffer memory

Claims (20)

Each of the plurality of hash tables corresponds to a hash function, each of the plurality of hash tables consists of physical hash buckets, and each physical bucket consists of Ways. identifying the plurality of hash tables to store data;
Each of the plurality of virtual buckets consists of a portion of the physical hash buckets and shares at least one of the physical hash buckets with another virtual bucket among the plurality of virtual buckets. identifying;
identifying each of the plurality of physical buckets having data allocated and stored in a corresponding one of the virtual buckets;
Hashing a data line according to a hash function among hash functions to generate a hash value;
determining whether a corresponding virtual bucket among the virtual buckets of the hash table has available space for a data block according to the hash value;
When the corresponding virtual bucket among the virtual buckets does not have available space until the corresponding virtual bucket among the virtual buckets has space for the data block, the virtual bucket from the corresponding virtual bucket among the virtual buckets Among them, sequentially moving data to adjacent virtual buckets; and
A memory deduplication operation method comprising storing the data block in a corresponding virtual bucket among the virtual buckets. According to claim 1,
A method of deduplicating memory, further comprising updating an address lookup table memory to change one or more lookup addresses corresponding to the moved data block. According to claim 1,
Each of the plurality of hash tables further includes a reference count line, a signature line, and a hopword line. According to claim 3,
A memory deduplication operation method further comprising generating a Hopword vector to represent the physical hash buckets containing data corresponding to the virtual bucket. According to claim 4,
The steps for generating the sumword vector are:
For each of the plurality of virtual buckets, whether each of the plurality of physical hash buckets of the relative virtual bucket among the plurality of virtual buckets includes a data block linked to the relative virtual bucket among the plurality of virtual buckets. A method of operating deduplication in memory comprising using a binary indicator to indicate. According to claim 3,
Consisting of log2(H) bits for each physical hash bucket to indicate a physical hash bucket among the plurality of physical hash buckets containing data corresponding to a certain virtual bucket among the plurality of virtual buckets. A memory deduplication operation method further comprising the step of generating a sumword value. According to claim 6,
The step of generating the sumword value consists of pseudo-addresses for each of the plurality of physical hash buckets containing data in a location expressed in association with pairs of the plurality of physical hash buckets and virtual buckets. A memory deduplication operation method including the step of creating a two-dimensional array. According to claim 1,
A memory deduplication operation method in which the plurality of hash tables are stored in volatile memory. According to claim 8,
A method of deduplicating a memory in which the volatile memory consists of dynamic random access memory (DRAM). According to claim 1,
Further comprising indexing the hash table with a physical line ID (PLID) consisting of a virtual bucket usage value field of log2(H) bits, wherein the value is a corresponding virtual bucket among the plurality of virtual buckets. A method of deduplicating memory with the same number of data blocks. According to claim 10,
A memory deduplication operation method further comprising: increasing the virtual bucket usage value field by 1 when writing an item to the corresponding virtual bucket among the plurality of virtual buckets. According to claim 1,
When the corresponding hash table is full, a memory deduplication operation method further comprising storing the data block in a buffer memory. In a deduplication dynamic random access memory (DRAM) memory module that reduces duplication of data blocks in memory for deduplication memory,
Hash table memory for storing deduplicated data blocks;
an address lookup table memory for storing addresses corresponding to the deduplicated data blocks; and
The deduplication DRAM memory module receives read requests that cause the deduplication DRAM memory module to retrieve the data blocks from the hash table memory and transmit the data blocks, and the deduplication DRAM memory module stores the data blocks in the hash table memory. A deduplicated DRAM memory module including a processor that receives write requests to perform. According to claim 13,
The hash table memory consists of a three-dimensional array of hash tables stored in the hash table memory, each of the hash tables consists of physical hash buckets, and each physical hash bucket is divided into Ways. A deduplicated DRAM memory module configured to store the deduplicated data blocks. According to claim 14,
Each of the hash tables further includes a plurality of virtual buckets, and each virtual bucket is comprised of two or more of the physical hash buckets. According to claim 15,
A deduplication DRAM memory module that moves the deduplicated data blocks between adjacent virtual buckets among the plurality of virtual buckets within the hash table. According to claim 13,
A deduplication DRAM memory module that performs data deduplication without externally provided commands. According to claim 13,
A deduplication DRAM memory module further comprising a buffer memory for storing data when the hash table memory is full. In a deduplication DRAM (Dynamic random access memory) memory module to reduce duplication of data blocks in memory,
Each of the plurality of hash tables is composed of physical hash buckets, each physical hash bucket is composed of Ways and stores the data blocks, and 3 of the hash tables are stored in the deduplication DRAM memory module. dimensional array;
processor; and
Includes a memory, wherein the memory stores instructions, and when the instructions are executed by the processor, the memory is configured to allow the deduplication DRAM memory module to select a dictionary between adjacent virtual buckets of the hash table among the plurality of hash tables. A deduplicated DRAM memory module that causes a deduplicated data block stored in to move, and each of the virtual buckets consists of two or more physical hash buckets. According to claim 19,
The memory further stores instructions, and when the instructions are executed by the processor, the memory causes a deduplication DRAM memory module to store instructions in one of the virtual buckets from which the prestored deduplicated data block is moved. A deduplicated DRAM memory module that causes the incoming data to be stored in the.

Images