CN116453565A - Efficient automatic duplicate removal storage circuit based on content addressing - Google Patents
Efficient automatic duplicate removal storage circuit based on content addressing Download PDFInfo
- Publication number
- CN116453565A CN116453565A CN202310312164.7A CN202310312164A CN116453565A CN 116453565 A CN116453565 A CN 116453565A CN 202310312164 A CN202310312164 A CN 202310312164A CN 116453565 A CN116453565 A CN 116453565A
- Authority
- CN
- China
- Prior art keywords
- data
- input data
- logic
- result
- write
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004364 calculation method Methods 0.000 claims abstract description 28
- 238000013500 data storage Methods 0.000 claims abstract description 27
- 238000000034 method Methods 0.000 claims abstract description 6
- 230000008569 process Effects 0.000 claims abstract description 4
- 238000013479 data entry Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 3
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000007274 generation of a signal involved in cell-cell signaling Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11C—STATIC STORES
- G11C15/00—Digital stores in which information comprising one or more characteristic parts is written into the store and in which information is read-out by searching for one or more of these characteristic parts, i.e. associative or content-addressed stores
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/023—Free address space management
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of data processing digital circuits, in particular to a high-efficiency automatic deduplication storage circuit based on content addressing, which comprises a bus access port, a data storage array, an input data path, matching calculation logic, writing control logic and result return logic; the bus access port is connected with the data storage array, the data storage array is respectively connected with the write-in control logic, the matching calculation logic and the input data path control, the matching calculation logic is respectively connected with the input data path, the result return logic and the write-in control logic control, the input data path is connected with the write-in control logic control, and the write-in control logic is connected with the result return logic control. The circuit can automatically process continuous input data without waiting for a matching result, realizes high-efficiency duplicate removal storage of the input data, and saves system resources.
Description
Technical Field
The invention relates to the technical field of data processing digital circuits, in particular to a high-efficiency automatic deduplication storage circuit based on content addressing.
Background
The content addressable memory (ContentAddressableMemory, CAM) acts as a special RAM designed specifically for lookup, matches the input data with all stored data and returns the results, primarily for mass data retrieval. The CAM which needs to be controlled separately for data retrieval and data storage cannot meet the requirements of automatic and efficient data deduplication storage processing. The invention designs for the efficient deduplication storage of data, realizes the rapid matching based on the content addressing technology, automatically performs the deduplication storage according to the matching result in the same period, and supports the continuous input of the data to be processed.
Disclosure of Invention
The invention aims to provide a high-efficiency automatic deduplication storage circuit based on content addressing, which can automatically process continuous input data without waiting for a matching result, realize high-efficiency deduplication storage of the input data and save system resources.
In order to solve the technical problems, the invention provides the following technical scheme:
a high-efficiency automatic deduplication storage circuit based on content addressing comprises a bus access port, a data storage array, an input data path, matching calculation logic, writing control logic and result return logic; the bus access port is connected with the data storage array, the data storage array is respectively connected with the write-in control logic, the matching calculation logic and the input data path control, the matching calculation logic is respectively connected with the input data path, the result return logic and the write-in control logic control, the input data path is connected with the write-in control logic control, and the write-in control logic is connected with the result return logic control.
Preferably, the input data path is used for performing data de-duplication storage processing, and comprises an input data bus and an input data valid signal.
Preferably, the matching calculation logic compares the input data with all effective data items in the register array simultaneously in a content addressing mode, and gathers each comparison result to obtain a matching result and a matching address of the current input data.
Preferably, the write control logic writes the input data which is not successfully matched into the data register pointed by the null pointer according to the matching result of the input data matching logic, updates the corresponding data valid flag bit, and discards the input data which is successfully matched with the existing data.
Preferably, the data storage array includes a plurality of data register units, each data register unit includes a data register and a corresponding data valid flag bit, where the data valid flag bit indicates whether data in the corresponding data register is valid or not, and is used for null pointer calculation.
Preferably, the bus access port is used for data reading and writing and valid bit configuration of the data storage array.
Preferably, the result return logic feeds back a matching result of each input data, and a matching address or a writing address corresponding to the matching result.
Preferably, the method further comprises the following steps: the deduplication storage circuit provides a special path for input data to be subjected to deduplication storage processing and corresponding effective signals, a bus port is provided for system access data storage, the effective input data is subjected to matching calculation logic to obtain a matching result, the writing control logic generates a writing enabling signal of current input data according to the matching result, generates a null pointer according to data effective bits of a register array, controls the current input data to be written into a free data register or to be directly discarded, and the processing result of the effective input data is stored into a register of a result return logic.
The invention has the following beneficial effects:
by adopting the technical scheme, the circuit can perform matching calculation and writing control of input data in real time, the input data cannot be interrupted by the behavior of the circuit for processing data or bus operation, the whole data processing process does not occupy system resources, and efficient automatic duplicate removal storage is realized.
Drawings
Fig. 1 is a circuit block diagram of the present invention.
FIG. 2 is a block diagram of the match calculation logic of the present invention.
FIG. 3 is a block diagram of write control logic in the present invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and the specific examples. Advantages and features of the invention will become more apparent from the following description and from the claims. It should be noted that the drawings are in a very simplified form and are all to a non-precise scale, merely for convenience and clarity in aiding in the description of embodiments of the invention.
Referring to fig. 1-3, a content addressing based efficient automatic deduplication storage circuit is provided, which can be integrated into a digital circuit, and the circuit efficiently matches input data with stored data based on a content addressing scheme, discards input data that is repeated with stored data according to the result, writes the input data into an idle data register without repetition, realizes real-time deduplication storage processing of the input data, and returns the processing result of the input data. The data processing system comprises a bus access port, a data storage array, an input data path, matching calculation logic, writing control logic and result return logic; the bus access port is connected with the data storage array, the data storage array is respectively connected with the write-in control logic, the matching calculation logic and the input data path control, the matching calculation logic is respectively connected with the input data path, the result return logic and the write-in control logic control, the input data path is connected with the write-in control logic control, and the write-in control logic is connected with the result return logic control.
In an embodiment of the present invention, further comprising: and the input data path is used for carrying out data processing needing to be subjected to duplicate removal storage and comprises an input data bus and an input data valid signal. And the matching calculation logic is used for simultaneously comparing the input data with all effective data items in the register array in a content addressing mode, and summarizing each comparison result to obtain a matching result and a matching address of the current input data. In the same period of data input, the matching calculation logic uses a content addressing mode to respectively and simultaneously compare the input data with all stored data, and encodes all comparison results to obtain a matching result and a matching address of the input data and the stored data. And the writing control logic generates a writing control signal according to the matching result of the input data matching logic in the same period of data input, writes the input data which is not successfully matched into a data register of the idle storage space pointed by the null pointer, updates the corresponding data valid flag bit, and discards the input data which is successfully matched with the existing data. The data storage array comprises a plurality of data register units, wherein each data register unit comprises a data register and a corresponding data valid flag bit, and the data valid flag bit indicates whether the data in the corresponding data register is valid or not and is used for null pointer calculation. And the bus access port is used for data reading and writing and valid bit configuration of the data storage array. The result returns the logic, the matching result of each piece of input data and the matching address or writing address corresponding to the matching result are fed back; namely, the return logic stores the matching result of the current period, stores the matching address or the input data write address pointed by the null pointer according to the matching result, and returns the stored result in the next period of the input data, so that the existence of a combinational logic ring in a digital circuit using the circuit is avoided.
As shown in fig. 1, the method further comprises the following steps: the deduplication storage circuit provides a special path for input data to be subjected to deduplication storage processing and corresponding effective signals, a bus port is provided for system access data storage, the effective input data is subjected to matching calculation logic to obtain a matching result, the writing control logic generates a writing enabling signal of current input data according to the matching result, generates a null pointer according to data effective bits of the data storage array, controls the current input data to be written into a free data register or to be directly discarded, and the processing result of the effective input data is stored in a register of a result return logic.
As shown in fig. 2, the matching calculation logic is implemented based on a content addressing mode, and valid input data and all valid register data are compared simultaneously to obtain a plurality of comparison results, and the comparison results are summarized to obtain a matching result of the current input data and the stored data. And encoding the comparison result to obtain the address of the data register matched with the input data.
As shown in fig. 3, the write control logic includes write signal generation and null pointer computation. And when the input data is valid and the matching result is that the matching is unsuccessful, enabling the writing signal, and writing the current input data into a data register pointed by the null pointer by the data storage array. The data valid bit of the data storage array is subjected to priority calculation to obtain a null pointer pointing to an idle data register.
Each data storage unit in the data storage array comprises a data register and a data valid bit, and when a write signal written into the control logic is valid, the data register pointed by the null pointer stores input data, and the corresponding data valid bit is automatically updated to be in a valid state. After the circuit is integrated into a digital circuit, software can read, write and access any data register and data valid bit through a bus port so as to realize functions of data backup, addition, rewriting, erasure and the like. To prevent blocking of incoming data processing, the register array is preferentially responsive to operation of the write control logic.
And when the input data is valid, the return logic stores the matching result of the matching calculation logic into a matching result bit. Meanwhile, if the matching is successful, the data register address matched with the input data, which is obtained by the matching calculation logic, is stored into a matching address register; otherwise, the data register address pointed by the null pointer of the write control logic is stored into the write address register.
The above description is only illustrative of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention, and any alterations and modifications made by those skilled in the art based on the above disclosure shall fall within the scope of the appended claims.
Claims (8)
1. The high-efficiency automatic deduplication storage circuit based on content addressing is characterized by comprising a bus access port, a data storage array, an input data path, matching calculation logic, writing control logic and result return logic; the bus access port is connected with the data storage array, the data storage array is respectively connected with the write-in control logic, the matching calculation logic and the input data path control, the matching calculation logic is respectively connected with the input data path, the result return logic and the write-in control logic control, the input data path is connected with the write-in control logic control, and the write-in control logic is connected with the result return logic control.
2. The content-addressable, high-efficiency, automatic deduplication storage circuit of claim 1, wherein the input data path is configured to store data for deduplication processing, and comprises an input data bus and an input data valid signal.
3. The content-addressable efficient automatic deduplication storage circuit of claim 1, wherein the match calculation logic compares the input data with all valid data entries in the register array simultaneously in a content-addressable manner, and sums each comparison result to obtain a match result and a match address for the current input data.
4. The content addressing based efficient automatic deduplication storage circuit as claimed in claim 1, wherein the write control logic writes the input data that is not successfully matched into the data register pointed by the null pointer and updates the corresponding data valid flag bit according to the matching result of the input data matching logic, and discards the input data that is successfully matched with the existing data.
5. The content-addressable high-efficiency automatic deduplication storage circuit of claim 1, wherein the data storage array comprises a plurality of data register units, each data register unit comprising a data register and a corresponding data valid flag bit, the data valid flag bit indicating whether data in the corresponding data register is valid, and the data register unit being used for null pointer calculation.
6. The content-addressable, high-efficiency, automatic deduplication storage circuit of claim 1, wherein the bus access port is configured for data read-write and valid bit configuration of the data storage array.
7. The content-addressable, high-efficiency, automatic deduplication storage circuit of claim 1, wherein the result return logic is configured to feed back a match result for each piece of input data, and a match address or write address corresponding to the match result.
8. The content-addressing-based efficient automatic deduplication storage circuit of claim 1, further comprising the process of: the deduplication storage circuit provides a special path for input data to be subjected to deduplication storage processing and corresponding effective signals, a bus port is provided for system access data storage, the effective input data is subjected to matching calculation logic to obtain a matching result, the writing control logic generates a writing enabling signal of current input data according to the matching result, generates a null pointer according to data effective bits of a register array, controls the current input data to be written into a free data register or to be directly discarded, and the processing result of the effective input data is stored into a register of a result return logic.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310312164.7A CN116453565A (en) | 2023-03-28 | 2023-03-28 | Efficient automatic duplicate removal storage circuit based on content addressing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310312164.7A CN116453565A (en) | 2023-03-28 | 2023-03-28 | Efficient automatic duplicate removal storage circuit based on content addressing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116453565A true CN116453565A (en) | 2023-07-18 |
Family
ID=87128066
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310312164.7A Pending CN116453565A (en) | 2023-03-28 | 2023-03-28 | Efficient automatic duplicate removal storage circuit based on content addressing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116453565A (en) |
-
2023
- 2023-03-28 CN CN202310312164.7A patent/CN116453565A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11307769B2 (en) | Data storage method, apparatus and storage medium | |
| US10628319B2 (en) | Methods for caching and reading data to be programmed into a storage unit and apparatuses using the same | |
| US10140215B1 (en) | Low overhead mapping for highly sequential data | |
| CN105117351A (en) | Method and apparatus for writing data into cache | |
| US10223037B2 (en) | Memory device including controller for controlling data writing using writing order confirmation request | |
| US10168951B2 (en) | Methods for accessing data in a circular block mode and apparatuses using the same | |
| US11210226B2 (en) | Data storage device and method for first processing core to determine that second processing core has completed loading portion of logical-to-physical mapping table thereof | |
| US6272595B1 (en) | N-way set-associative cache memory which includes a store hit buffer for improved data access | |
| CN108519858B (en) | Memory chip hardware hit method | |
| CN100492323C (en) | Method and apparatus for protecting caching content and caching controller thereof | |
| CN116453565A (en) | Efficient automatic duplicate removal storage circuit based on content addressing | |
| CN116149554B (en) | RISC-V and extended instruction based data storage processing system and method thereof | |
| US20040078544A1 (en) | Memory address remapping method | |
| US20210373887A1 (en) | Command delay | |
| CN116339613A (en) | Storage device, method of operating the same, and method of operating storage system including the same | |
| CN113377684B (en) | Data writing system and method | |
| US20040128414A1 (en) | Using system memory as a write buffer for a non-volatile memory | |
| WO1991007754A1 (en) | Read-while-write-memory | |
| KR20220062399A (en) | Managing data dependence for out-of-order processing on hybrid DIMMs | |
| CN105701060A (en) | FPGA-based high-speed real-time data recording system | |
| CN112306379A (en) | Data movement recovery method and device, electronic equipment and storage medium | |
| KR20200113991A (en) | Controller and memory system | |
| US7080222B1 (en) | Cellular telephone memory with backup memory interface | |
| US20240119017A1 (en) | Bridge control chip and associated signal processing method | |
| US11989431B2 (en) | Write updates sorting during BKOPS idle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |