CN110109873B - File management method for message queue - Google Patents
File management method for message queue Download PDFInfo
- Publication number
- CN110109873B CN110109873B CN201910381124.1A CN201910381124A CN110109873B CN 110109873 B CN110109873 B CN 110109873B CN 201910381124 A CN201910381124 A CN 201910381124A CN 110109873 B CN110109873 B CN 110109873B
- Authority
- CN
- China
- Prior art keywords
- message
- file
- index
- files
- metadata
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a file management method for a message queue, which is characterized by comprising the following steps: the novel nonvolatile storage device is adopted, message files and metadata files are arranged in a storage module, two key value storage structures are established and used for recording the number of messages and the size of the message files contained in each message file, the message files and the metadata files are in one-to-one correspondence and are stored separately, the message files are used for storing message entity data, and the metadata files are used for storing description information of each message, including the size of the message, offset in the message files and message generation time. The effect is as follows: the NVM is used as a persistent storage medium of the message, the message data can be accessed through a virtual address of a process, a complex I/O software stack is avoided, the message data can be randomly accessed through accessing a message number, a multi-precision message hierarchical index structure based on the message production time is established, and the message can be quickly traced.
Description
Technical Field
The invention relates to the computer data storage technology, in particular to a file management method for a message queue.
Background
The message queue, also called as message middleware, is a container for storing messages in the message transmission process, is an important component in a distributed system, and mainly solves the problems of application decoupling, asynchronous consumption and the like. Under a general condition, a message queue adopts a publishing mode and a subscribing mode, provides a uniform interface to help decoupling of a client and a server, provides a message persistence mechanism, and ensures that messages can be correctly received and cannot be lost under an emergency condition, so that the server can deal with a message processing scene with high throughput. The message queue has wide application in the fields of mobile internet, electronic commerce and the like.
Most of the existing message queues store messages in a memory or a file system based on block devices such as a disk. The data is stored in the memory, so that the real-time operation and speed block of the message can be realized, but the data cannot be stored persistently, and the data can be lost after power failure. Saving the message in a file system based on a disk, SSD, or other block device may persist the message. But accessing a block device based file system requires going through a complex I/O software stack, which significantly degrades the performance of the message queue.
The novel non-volatile memory (NVM) has a read-write speed close to that of a DRAM (dynamic random access memory), can be directly mounted on a memory bus, and has high density, low energy consumption, no loss in power failure and byte addressing. The new nonvolatile memory mainly includes a Phase Change Memory (PCM), a resistance change memory (RRAM), a Memristor (Memristor), and the like. The NVM is simulated into a magnetic disk device, and the advantage of the NVM cannot be exerted by using the initialization of the existing magnetic disk file system. Because the operating system connects the block devices via the I/O bus, accessing the block device-based file system requires going through a complex I/O software stack. As shown in fig. 1, the I/O software stack of the Linux File System is a Virtual File System (VFS) first. Internal operations of the file system, such as data block lookups, follow. An I/O request to access the block device is then issued, followed by a device-oriented software layer such as a generic block layer, an I/O scheduling layer, and a block device driver layer. Accessing data using a conventional I/O software stack incurs significant overhead, taking reading file data as an example, the file data is first copied from a block device to a kernel I/O buffer by DMA, typically also to a page cache, and finally to a user mode buffer. In the process, process context switching is also needed, and CPU resources are occupied.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a file management method for a message queue, which accesses file data in a mode of accessing a virtual address of a process based on novel nonvolatile storage equipment and avoids complex context switching of an I/O (input/output) software stack and the process.
In order to realize the purpose, the invention adopts the following specific technical scheme:
a file management method for a message queue is characterized in that: the novel nonvolatile storage device is adopted, message files and metadata files are arranged in a storage module, two key value storage structures are established and used for recording the number of messages and the size of the message files contained in each message file, the message files and the metadata files are in one-to-one correspondence and are stored separately, the message files are used for storing message entity data, and the metadata files are used for storing description information of each message, including the size of the message, offset in the message files and message generation time.
Optionally, the system stores the message by using a shared memory, and the message is written and read by using a virtual address of the process.
Optionally, in the process of reading the message, the metadata file is read first, and the size of the message and the offset of the message in the message file are obtained from the metadata file, and then the corresponding message file is read.
Optionally, in the message writing process, the method includes three steps of writing a message file, writing a metadata file, and calculating and updating the number of messages and the key value storage structure of the message file size.
Optionally, the same subject partition only runs one producer process for writing, and if there is a message that both producer processes generate a certain subject, different partitions are allocated to the producer process.
Optionally, the size of the message file in the key value storage structure and the number of messages in the message file are updated by the background thread detecting the metadata item in the metadata file through the sliding detection point.
Optionally, in the message file, the messages are stored sequentially in the order of generation time.
Optionally, a message fast index structure based on generation time is adopted during message query, a system establishes a tree-shaped index structure with three layers of time, minute and second, each index node of each layer of index is a continuous space, each index node is composed of a plurality of index items with fixed sizes, and each index item can be randomly accessed, wherein:
the first layer is based on a whole-time index and is provided with an index node, each index node consists of 24 index items, and each index item consists of a file index number, a message number in a message file and an initial address of the index node of the next layer;
the second layer is based on minute indexes and comprises 24 index nodes, each index node comprises 60 index items, and each index item comprises a file index number, a message number in a message file and an initial address of the index node of the next layer;
the third layer is based on second-level index, and has 24 × 60 index nodes, each index node is composed of 60 index entries, and each index entry is composed of a file index number and a message number in a message file.
The invention has the following remarkable effects:
the NVM is used as a persistent storage medium of the message, the message data can be accessed through a virtual address of a process, a complex I/O software stack is avoided, the message data can be randomly accessed through accessing a message number, a multi-precision message hierarchical index structure based on the message production time is established, and the message can be quickly traced.
Drawings
The invention will be further described with reference to the accompanying drawings and examples, in which:
FIG. 1 is a diagram of an I/O software stack in a conventional Linux system;
FIG. 2 is a logical structure of a message storage module in an embodiment of the present invention;
FIG. 3 is a schematic block diagram of a shared memory according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a relationship between a message file and a metadata file in an embodiment of the present invention;
FIG. 5 is a schematic block diagram of a message write operation in an embodiment of the invention;
FIG. 6 is a schematic diagram of slip detection in an embodiment of the present invention;
FIG. 7 is a diagram of a production time based message index architecture in an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages of the present invention more apparent, the following detailed description is given with reference to the accompanying drawings and specific embodiments, it being understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.
As shown in fig. 2, the present invention provides a file management method for a message queue, which is used for persistent storage of messages in the message queue based on Topic, where Topic is an abstract concept, that is, a category of a certain type of messages, and can be expanded horizontally into Partition partitions (hereinafter, referred to as "P"), and each Partition includes a plurality of files for storing messages. The invention adopts a novel nonvolatile storage device, persistently stores messages in the form of files, separately stores message data and message description information (hereinafter referred to as message metadata), and searches the message data by looking up the message metadata information. The storage module has two files of message file and metadata file, the message file stores the entity data of the message, and the metadata file stores the description information of each message, including the size of the message, the offset of the message in the message file, the production time of the message, etc. Each message file and metadata file is fixed in size and can be configured by itself, and the metadata file is smaller than the message file under normal conditions. In specific implementation, the naming mode of the message file is Topic _ P-date-index.log, wherein Topic _ P is the name of Topic Partition, date is the current date, index is the file id, the file names are increased progressively from 0 to represent the several files on the Partition day, and the message file is stored in a log/Topic directory. Corresponding to the message file, there is a metadata file with the same name, and the suffix of the metadata file is meta, which is stored in the meta/topic directory. Two key value storage structures are established in the storage module to record the number of messages and the size of the message files contained in each message file, the name of the message file, topic _ P-date-index, is taken as a key, and the number of messages in the message file, counts, is taken as a value; take Topic _ P-date-index as key and message file size as value.
In the present system, the message file needs to be written by the write process and read by the read process, which are also typically working together. Linux systems provide a mechanism for sharing memory (shm), which is to allow the memory in the same block to be mapped to address spaces of different processes, as shown in fig. 3, process 1 can see the update of process 2 to the data in the shared memory immediately, and vice versa. The system takes the NVM physical space as a shared NVM memory pool, and each message file and metadata file is essentially a segment of shared NVM memory.
The implementation of the shared memory mainly includes the following 4 system call functions:
(1) creating a shared memory int shmvet (key _ t key, size _ t size, int shmflg);
the first parameter is the name of the shared memory segment;
the second parameter is the memory capacity to be created;
the third is a permission sign;
when the shmlet function succeeds, a shared memory identifier shm _ id associated with the key is returned.
(2) Mapping the shared memory in an address space void (int shm _ id, const void (stm) addr, int shmflg) of the process;
the first parameter is a shared memory identifier returned by the shmget function;
the second parameter is that the designated shared memory is mapped to the address in the current process, and is usually empty, which indicates that the system is allowed to select the address of the shared memory;
the third parameter, shm _ flg, is a set of flag bits, typically 0;
and returning the first address of the mapping space after the calling is successful.
(3) Separating int shmdt (constvoid shmaddr) from the total current process of the shared memory;
this function is used to separate the shared memory from the current process. Note that detaching the shared memory is not deleting it, but simply making the shared memory no longer available to the current process.
(4) Controlling a shared memory int shmctl (int shm _ id, int command, struct shmid _ ds _ buf); such as changing the associated value of the shared memory, deleting the shared memory segment, etc.
The sizes of the message file and the metadata file are fixed, when a new file needs to be created after the file is completely written, hash values are respectively taken for the new message file name and the new metadata name, linux shared memory functions shmlet are used as parameters to obtain corresponding shared memory identifiers shm _ id, when a process needs to access the message file and the metadata file, a shmat function is called to be mapped in a virtual address space of the process, and data access can be completed through accessing a returned address.
In the process of reading the message, the metadata file is read first, the size of the message and the offset of the message in the message file are obtained from the metadata file, and then the corresponding message file is read.
Specifically, the query and the reading can be performed quickly according to the message id, the message id is composed of the file name of the file where the message is located and the sequence number of the message in the file, and the message id is increased from 0 in each message file. The message file and the metadata file are in one-to-one correspondence, as shown in fig. 4, the metadata file stores metadata information of each message, and the metadata item of each message is composed of the production time of the message, the offset of the message in the message file, and the length of the message. The size of the metadata item is fixed, and after the metadata file is mapped in the virtual address space of the process, any metadata item can be randomly accessed in a user mode. When reading the message, reading the corresponding metadata item, and acquiring the offset and the message size of the message in the message file, so that the real message data can be accessed.
In the process of writing the message, the method comprises three steps of writing a message file, writing a metadata file and calculating and updating the number of the message and the key value storage structure of the size of the message file.
During specific implementation, the same Partition only allows one producer process to perform write operation at the same time, and if two producer processes produce a message of a certain Topic, different partitions are allocated to the two producer processes. The message write operation is now described by taking as an example message 3 where the producer process produces the Topic1_ P0- # - # file, as shown in FIG. 5.
1) And the generating process calls the API to generate a message, and writes the message into a message file in an apend mode after encapsulation processing.
2) And after the message file is successfully written, updating the metadata file corresponding to the message file, and writing the metadata item of the message 3 in the corresponding metadata file in an apend mode.
3) And the background thread calculates the total number of the messages in the message file and the size of the message file by detecting the metadata item, and updates the related key value storage structure. If any step fails in the middle, the message writing fails, and the step returns to the first step for re-execution.
Updating the size of the message file and the number of messages in the message file in the key value storage structure is calculated by the background thread through detecting the metadata item at the sliding detection point, and the detection flow is shown in fig. 6.
After the metadata file is created, all fields of the metadata item are initialized to 0. After the metadata file is mapped in the virtual address space of the process, any metadata item can be accessed because the size of each metadata item is fixed. The field size representing the message length is placed at the end of the message metadata item, and is represented by shaping, and the CPU performs a write operation as an atomic operation, so as long as the size field is not 0, the write operation of the message is completed, the metadata item identifies a new message, and then the number of messages in the message file and the size of the message file are changed, and the detection point is moved to the next metadata item.
The persistent message queue system plays a role of storing messages at the same time, and it is necessary to provide a quick message query function for the message queue, and a user may need to trace back messages in a past time period, and it is obviously unreasonable to search for messages one by one. Locating a message file to a given Topic in a message storage module is not difficult, the difficulty being how to quickly locate a message that a given Topic produces at a certain time period.
In the system, the file name of the message contains the current date, but if the time is divided into finer granularity, such as hour, minute and second, a data structure needs to be established to assist in quickly positioning the message, and a three-layer index is established according to the hour, minute and second. The metadata information of each message records the production time of the message, so that an index structure can be established by detecting metadata items in the running process, and the establishment of the index structure is implemented according to Partition, as shown in fig. 7.
In the message file, the messages are stored according to the production time sequence, so that only the id of the production message at the whole time point needs to be recorded, and the messages after the id are all produced after the time point. For the requirements of different time granularities, three layers of tree indexes are established according to time, minutes and seconds. Each index node of each layer of index is a section of continuous space, each index node is composed of a plurality of index items with fixed size, and each index item can be randomly accessed. The first-layer index is an index based on the whole time, and has only one index node, and since one day is divided into 24 hours, the index node is composed of 24 index items, and each index item is composed of a file index, a message id in a message file, and a start address addr of the next-layer index node. Since there may be more than one file per day generated by each Topic Partition, different files are denoted herein by incremental indices, the id identifying the message id within the current file. Index entry 0 identifies that all messages are produced after 0's rounding from the message file's first message, and so on. Since an hour is divided into 60 minutes, the addr field of each index entry points to the next level of inode representing these 60 minutes. Similarly, in the minute-level index entry, index entry 0 identifies that all messages are produced after 0 minutes from the message number of a certain message file. One minute again includes 60 seconds, so the addr field of each index entry in the second level index points to the third level second level inode. The index entries of the second-level inode are only index and id, and all messages after the id are identified to be produced after the second.
Through the three-layer index structure, the messages produced every second can be quickly located. For example, to search for a message after 8 points, 07 minutes and 56 seconds, an index entry with a subscript of 8 at the first layer may be searched first, and a second-layer index node to which the index entry points may be found, then an index entry with a subscript of 7 at the node may be searched, and a start address to which the index entry points at the third-layer index node may be found, and then an index entry with a subscript of 56 at the node may be searched, so that a message id meeting requirements may be found, and then a corresponding metadata entry may be read, and finally, message data may be obtained.
In summary, the NVM is used as a persistent storage medium for messages, and the invention can access message data through a virtual address of a process, avoid a complex I/O software stack, and randomly access message data through an access message id. A multi-precision message hierarchical index structure based on message production time is established, and messages can be quickly traced.
Finally, it is noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
The above sequence numbers of the embodiments of the present invention are merely for description, and do not represent the advantages and disadvantages of the embodiments, and it is clear to those skilled in the art from the above description of the embodiments that the method of the embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation manner in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (6)
1. A file management method for a message queue, characterized by: the method comprises the steps that a phase change memory, a resistive random access memory or a memristor is used as a storage module, message files and metadata files are arranged in the storage module, two key value storage structures are established and used for recording the number of messages and the size of the message files contained in each message file, the message files and the metadata files are in one-to-one correspondence and are stored separately, the message files are used for storing message entity data, the metadata files are used for storing description information of each message, and the description information comprises the size of the message, offset in the message files and message generation time;
in the message file, messages are sequentially stored according to the generation time sequence;
the system establishes a tree index structure with three layers of time, minute and second by adopting a quick message index structure based on generation time during message query, each index node of each layer of index is a section of continuous space, each index node consists of a plurality of index items with fixed size, and each index item can be randomly accessed, wherein:
the first layer is based on the whole-time index and is provided with an index node, each index node consists of 24 index items, and each index item consists of a file index number, a message number in a message file and an initial address of the index node of the next layer;
the second layer is based on minute indexes and comprises 24 index nodes, each index node comprises 60 index items, and each index item comprises a file index number, a message number in a message file and an initial address of the index node of the next layer;
the third layer is based on second-level index, and has 60 × 24 index nodes, each index node is composed of 60 index entries, and each index entry is composed of a file index number and a message number in a message file.
2. The file management method for a message queue according to claim 1, characterized in that: and storing the message in a shared memory mode, and writing and reading the message by using the virtual address of the process.
3. The file management method for a message queue according to claim 1 or 2, characterized in that: in the process of reading the message, the metadata file is read first, the size of the message and the offset of the message in the message file are obtained from the metadata file, and then the corresponding message file is read.
4. The file management method for a message queue according to claim 1 or 2, characterized in that: in the process of writing the message, the method comprises three steps of writing a message file, writing a metadata file and calculating and updating the number of the message and the key value storage structure of the size of the message file.
5. The file management method for a message queue of claim 4, wherein: and simultaneously, only one producer process is operated to write in the same theme partition, and if two producer processes produce the message of a certain theme, different partitions are allocated to the producer processes.
6. The file management method for a message queue according to claim 4, characterized in that: and the size of the message file in the key value storage structure and the number of messages in the message file are updated by the background thread through detecting the metadata items in the metadata file by the sliding detection point.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910381124.1A CN110109873B (en) | 2019-05-08 | 2019-05-08 | File management method for message queue |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910381124.1A CN110109873B (en) | 2019-05-08 | 2019-05-08 | File management method for message queue |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110109873A CN110109873A (en) | 2019-08-09 |
| CN110109873B true CN110109873B (en) | 2023-04-07 |
Family
ID=67488894
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910381124.1A Active CN110109873B (en) | 2019-05-08 | 2019-05-08 | File management method for message queue |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110109873B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110704212B (en) * | 2019-09-29 | 2022-04-22 | 广州荔支网络技术有限公司 | Message processing method and device |
| CN111125019A (en) * | 2019-12-20 | 2020-05-08 | 北京无线电测量研究所 | File retrieval method, writing method, system, FPGA chip and device |
| CN111338834B (en) * | 2020-02-21 | 2023-06-30 | 北京百度网讯科技有限公司 | Data storage method and device |
| CN111597124B (en) * | 2020-04-21 | 2023-05-05 | 重庆大学 | Method, system and storage medium for organizing data of persistent memory file system |
| CN113239307A (en) * | 2021-05-17 | 2021-08-10 | 北京百度网讯科技有限公司 | Method and device for storing message theme |
| CN116455956B (en) * | 2023-06-16 | 2023-08-15 | 中国人民解放军国防科技大学 | Method and system for data acquisition and data playback based on message middleware |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101055591A (en) * | 2007-05-25 | 2007-10-17 | 中兴通讯股份有限公司 | Data access method for all-memory database |
| CN105005510A (en) * | 2015-07-02 | 2015-10-28 | 西安交通大学 | Error correction protection architecture and method applied to resistive random access memory cache of solid state disk |
| JP2015191361A (en) * | 2014-03-27 | 2015-11-02 | キヤノン株式会社 | Information processor, information processing system, control method of information processor and computer program |
| CN106612306A (en) * | 2015-10-22 | 2017-05-03 | 中兴通讯股份有限公司 | Data sharing method and device of virtual machine |
| CN107704604A (en) * | 2017-10-16 | 2018-02-16 | 中汇信息技术（上海）有限公司 | A kind of information persistence method, server and computer-readable recording medium |
| CN108132845A (en) * | 2016-12-01 | 2018-06-08 | 阿里巴巴集团控股有限公司 | Message storage, delivering method and device and electronic equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9996403B2 (en) * | 2011-09-30 | 2018-06-12 | Oracle International Corporation | System and method for providing message queues for multinode applications in a middleware machine environment |
-
2019
- 2019-05-08 CN CN201910381124.1A patent/CN110109873B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101055591A (en) * | 2007-05-25 | 2007-10-17 | 中兴通讯股份有限公司 | Data access method for all-memory database |
| JP2015191361A (en) * | 2014-03-27 | 2015-11-02 | キヤノン株式会社 | Information processor, information processing system, control method of information processor and computer program |
| CN105005510A (en) * | 2015-07-02 | 2015-10-28 | 西安交通大学 | Error correction protection architecture and method applied to resistive random access memory cache of solid state disk |
| CN106612306A (en) * | 2015-10-22 | 2017-05-03 | 中兴通讯股份有限公司 | Data sharing method and device of virtual machine |
| CN108132845A (en) * | 2016-12-01 | 2018-06-08 | 阿里巴巴集团控股有限公司 | Message storage, delivering method and device and electronic equipment |
| CN107704604A (en) * | 2017-10-16 | 2018-02-16 | 中汇信息技术（上海）有限公司 | A kind of information persistence method, server and computer-readable recording medium |
Non-Patent Citations (5)
| Title |
|---|
| "Redis数据库在非易失性内存上的交互技术研究与实现";秦杰杰;《中国优秀硕士学位论文全文数据库》;20190415;全文 * |
| "UMFS: an efficient user-space file system for non-volatile memory";xianzhang chen等;《 Journal of Systems Architecture》;20180930;全文 * |
| "基于异构存储设备的分布式对象存储系统优化研究";吴林;《中国博士学位论文全文数据库》;20190415;正文第2.4.5节，第3.3.4节，第4.3.5节 * |
| "面向非易失性内存的文件系统与页面交换机制研究";陈咸彰;《中国博士学位论文全文数据库》;20180615;全文 * |
| "面向非易失性存储器的多表连接写操作的优化研究";马竹琳，李心池，诸葛晴凤，吴林，陈咸彰;《计算机学报》;20180925;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110109873A (en) | 2019-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110109873B (en) | File management method for message queue | |
| US20210056074A1 (en) | File System Data Access Method and File System | |
| JP6205650B2 (en) | Method and apparatus utilizing non-uniform hash function to place records in non-uniform access memory | |
| EP2863310B1 (en) | Data processing method and apparatus, and shared storage device | |
| CN107066498B (en) | Key value KV storage method and device | |
| CN114860163B (en) | Storage system, memory management method and management node | |
| CN110555001B (en) | Data processing method, device, terminal and medium | |
| JP2015512604A (en) | Cryptographic hash database | |
| CN101567003A (en) | Method for managing and allocating resource in parallel file system | |
| CN112463753B (en) | Block chain data storage method, system, equipment and readable storage medium | |
| CN111177143B (en) | Key value data storage method and device, storage medium and electronic equipment | |
| CN106570113B (en) | Mass vector slice data cloud storage method and system | |
| CN114281762B (en) | Log storage acceleration method, device, equipment and medium | |
| GB2439576A (en) | Storing related small data fragments in the same block | |
| CN109407985B (en) | Data management method and related device | |
| RU2525752C2 (en) | Method and apparatus for storing, reading and writing compound document | |
| CN115470156A (en) | RDMA-based memory use method, system, electronic device and storage medium | |
| JP2018513454A (en) | Efficient performance of insert and point query operations in the column store | |
| KR100907477B1 (en) | Apparatus and method for managing index of data stored in flash memory | |
| CN112463880B (en) | Block chain data storage method and related device | |
| US7509461B1 (en) | Method and apparatus for intelligent buffer cache pre-emption | |
| GB2440357A (en) | Data storage method | |
| CN108804571B (en) | Data storage method, device and equipment | |
| CN113204520B (en) | Remote sensing data rapid concurrent read-write method based on distributed file system | |
| CN111399753B (en) | Method and device for writing pictures |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |