CN112148229B - Method for accelerating read-write efficiency of nonvolatile memory under Feiteng server platform - Google Patents
Method for accelerating read-write efficiency of nonvolatile memory under Feiteng server platform Download PDFInfo
- Publication number
- CN112148229B CN112148229B CN202011083307.4A CN202011083307A CN112148229B CN 112148229 B CN112148229 B CN 112148229B CN 202011083307 A CN202011083307 A CN 202011083307A CN 112148229 B CN112148229 B CN 112148229B
- Authority
- CN
- China
- Prior art keywords
- data
- read
- nonvolatile memory
- address space
- write
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000006870 function Effects 0.000 claims abstract description 23
- 230000005540 biological transmission Effects 0.000 claims abstract description 11
- 238000013507 mapping Methods 0.000 claims abstract description 5
- 239000000523 sample Substances 0.000 claims description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0613—Improving I/O performance in relation to throughput
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0662—Virtualisation aspects
- G06F3/0667—Virtualisation aspects at data level, e.g. file, record or object virtualisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Techniques For Improving Reliability Of Storages (AREA)
- Memory System (AREA)
Abstract
The invention relates to a method for accelerating the read-write efficiency of a nonvolatile memory under a Feiteng server platform, which comprises the following steps: applying for a virtual address space in a linux kernel virtual address space, and mapping the virtual address space with a nonvolatile memory physical address space by establishing a page table; if the nonvolatile memory needs to read and write a large amount of data, the data read and write are performed in parallel through a plurality of established device file nodes, the application layer performs block division on the data to be read and written, a plurality of processes are established, and each process corresponds to a block of data object; copying the data of the source address space into the destination address space by the read and write functions of the device driving operation function of the newly established device node; and judging whether the result fed back by the driving program is correct or not, if so, continuing copying the next data block, and if not, returning to the exception, and ending the transmission.
Description
Technical Field
The invention relates to a Feiteng server platform technology, in particular to a method for accelerating the read-write efficiency of a nonvolatile memory under a Feiteng server platform.
Background
In the initialization stage, the drive of the nonvolatile memory under the kernel applies a virtual address space to the kernel according to the physical address and the memory space of the nonvolatile memory, and the applied virtual address space is configured with a page table interface function through the kernel to realize the mapping relation between the virtual address and the physical address. When the nonvolatile memory is read and written, the nonvolatile memory driver can address and read and write the memory data according to the virtual address base address plus the address offset required to be read and written. And the nonvolatile memory can judge whether the nonvolatile memory is in a sector mode, an fsdax mode or an fsdax mode, then judge the validity of data under different modes, and finally transmit a block of data. The non-volatile memory will execute the above process in a loop while transmitting multiple blocks of data until all data operations are completed.
The nonvolatile memory driver is a single data path, and on the Feiteng2000+ processor, the nonvolatile memory can support multi-path parallelization access, so that the read-write performance under the native drive of the linux kernel is far lower than the theoretical data transmission bandwidth of hardware. And in the nonvolatile memory drive, a lot of redundancy judgment is performed, so that a lot of processor resources are occupied, and the data transmission performance is greatly influenced.
Disclosure of Invention
The invention aims to provide a method for accelerating the read-write efficiency of a nonvolatile memory under a Feiteng server platform, which is used for solving the problems in the prior art.
The invention discloses a method for accelerating the read-write efficiency of a nonvolatile memory under a Feiteng server platform, which comprises the following steps: applying for a virtual address space in a linux kernel virtual address space, and mapping the virtual address space with a nonvolatile memory physical address space by establishing a page table; if the nonvolatile memory needs to read and write a large amount of data, the data read and write are performed in parallel through a plurality of established device file nodes, the application layer performs block division on the data to be read and written, a plurality of processes are established, and each process corresponds to a block of data object; copying the data of the source address space into the destination address space by the read and write functions of the device driving operation function of the newly established device node; and judging whether the result fed back by the driving program is correct or not, if so, continuing copying the next data block, and if not, returning to the exception, and ending the transmission.
According to an embodiment of the method for accelerating the read-write efficiency of the nonvolatile memory under the Feiteng server platform, in the nonvolatile memory drive, a function is added to be called in a probe stage, namely an initialization stage, of the nonvolatile memory drive, and the function functions comprise: applying for the numbers of the primary device and the secondary device from the linux kernel, establishing a plurality of device file nodes, and registering a device driving operation interface function.
According to the embodiment of the method for accelerating the read-write efficiency of the nonvolatile memory under the Feiteng server platform, a plurality of processes are operated to work, the maximum data volume transmitted by each process is 2GB, and the processes divide the data to be transmitted into a plurality of data blocks aligned in the size of 2GB.
According to one embodiment of the method for accelerating the read-write efficiency of the nonvolatile memory under the Feiteng server platform, after one copy of the 2GB data block is completed, the driving program informs the application program.
According to the embodiment of the method for accelerating the read-write efficiency of the nonvolatile memory under the Feiteng server platform, a plurality of processes simultaneously utilize a plurality of equipment nodes to perform parallelization transmission of a plurality of data paths.
The invention supports multiple simultaneous accesses in the nonvolatile memory on Feiteng2000+, and establishes a plurality of equipment nodes on the nonvolatile memory to support the simultaneous operation of a plurality of data paths, thereby increasing the throughput efficiency of data. Meanwhile, the mode is not judged, and the data area is directly operated in the form of binary data.
Drawings
Fig. 1 is a flow chart of a method for accelerating the read-write efficiency of a nonvolatile memory under a Feiteng server platform.
Detailed Description
For the purposes of clarity, content, and advantages of the present invention, a detailed description of the embodiments of the present invention will be described in detail below with reference to the drawings and examples.
Referring to fig. 1, the read-write process of the nonvolatile memory of the present invention includes the following steps:
step one: in the nonvolatile memory drive, a function is added, the function of the function mainly comprises the application of a main device number and a secondary device number from a linux kernel, the effect of the main device number and the secondary device number in the linux kernel is to number an external device, and thus an operating system can manage and access the external device through the main device number and the secondary device number. The functions of the added function further include creating a plurality of device file nodes, registering the device driver operation interface function. Functions added in the nonvolatile memory drive are called in the probe phase, i.e., initialization phase, of the nonvolatile memory drive.
Step two: applying for a block of virtual address space in the linux kernel virtual address space, and mapping the virtual address space to a nonvolatile memory physical address space by establishing a page table. In the linux kernel, virtual addresses are used, and the management of the memory is realized by the linux kernel through the management of the virtual addresses. The virtual address does not have a physical entity memory, so the virtual address needs to access the virtual address through the corresponding relation between the virtual address and the physical address stored in the page table, and the virtual address is actually accessed by the physical address.
Step three: if the nonvolatile memory needs to read and write a large amount of data, the data can be read and written in parallel through a plurality of established device file nodes. The application program cuts and divides the data to be read and written at the application layer, and establishes a plurality of processes, wherein each process corresponds to a block of data object. The application program runs a plurality of processes to work simultaneously. The maximum amount of data per transmission by a process is 2GB. The process splits the data to be transmitted into a plurality of data blocks aligned in a size of 2GB.
Step four: the newly established device node's device driver functions read and write function copy the data of the source address space into the destination address space. This step operation achieves copying of the 2GB data blocks. After completing one copy of the 2GB block, the driver will inform the application.
Step five: and the application program judges whether the data block is correct according to the feedback result of the driving program, and if the data block is correct, the copying of the next data block is continued. The number of bytes returned by the driver to be successfully transmitted is equal to the number of bytes sent or requested by the application program, and the result is correct; the number of bytes returned by the driver to be successfully transmitted is not equal to the number of bytes sent or requested by the application program, and the result is wrong. If the judgment is incorrect, the exception is returned, and the transmission is terminated.
Step six: and thirdly, a plurality of processes established in the step three simultaneously utilize a plurality of equipment nodes to realize parallelization transmission of a plurality of data paths. Finally, the purpose of accelerating the transmission of large-block data is achieved.
According to the invention, a plurality of device nodes are established in the drive of the nonvolatile memory, each device node correspondingly operates a data area, the device nodes do not influence each other, and the device nodes can work in parallel, so that the data copying speed is increased. In addition, by adopting the additionally established equipment nodes, a series of redundancy judgment before copying the nonvolatile memory can be avoided, so that the performance loss of the processor is avoided from being up to the redundancy judgment.
Compared with the prior art, the technical method provided by the invention has the advantages that the multichannel data paths are used for parallel access, and the binary read-write operation is directly carried out, so that the read-write speed of the nonvolatile memory is greatly increased. The increased performance is almost twice as high as the read-write performance of the prior art solutions.
The foregoing is merely a preferred embodiment of the present invention, and it should be noted that modifications and variations could be made by those skilled in the art without departing from the technical principles of the present invention, and such modifications and variations should also be regarded as being within the scope of the invention.
Claims (5)
1. A method for accelerating the read-write efficiency of a nonvolatile memory under a Feiteng server platform is characterized by comprising the following steps:
applying for a virtual address space in a linux kernel virtual address space, and mapping the virtual address space with a nonvolatile memory physical address space by establishing a page table;
if the nonvolatile memory needs to read and write a large amount of data, the data read and write are performed in parallel through a plurality of established device file nodes, the application layer performs block division on the data to be read and written, a plurality of processes are established, and each process corresponds to a block of data object;
copying the data of the source address space into the destination address space by the read and write functions of the device driving operation function of the newly established device node;
and judging whether the result fed back by the driving program is correct or not, if so, continuing copying the next data block, and if not, returning to the exception, and ending the transmission.
2. The method for accelerating the read-write efficiency of a nonvolatile memory under a Feiteng server platform according to claim 1, wherein in the nonvolatile memory driver, a function is added to be called in a probe stage, namely an initialization stage, of the nonvolatile memory driver, and the function comprises: applying for the numbers of the primary device and the secondary device from the linux kernel, establishing a plurality of device file nodes, and registering a device driving operation interface function.
3. The method for accelerating the read-write efficiency of the nonvolatile memory under the Feiteng server platform according to claim 1, wherein a plurality of processes are operated to work, the maximum data quantity transmitted by each process is 2GB, and the processes divide the data to be transmitted into a plurality of data blocks aligned in the size of 2GB.
4. A method for accelerating the read-write efficiency of a nonvolatile memory under a Feiteng server platform according to claim 3, wherein the driver program notifies the application program after completing one copy of the 2GB data block.
5. The method for accelerating the read-write efficiency of a nonvolatile memory under a Feiteng server platform according to claim 1, wherein a plurality of processes simultaneously use a plurality of device nodes to perform parallelized transmission of a plurality of data paths.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011083307.4A CN112148229B (en) | 2020-10-12 | 2020-10-12 | Method for accelerating read-write efficiency of nonvolatile memory under Feiteng server platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011083307.4A CN112148229B (en) | 2020-10-12 | 2020-10-12 | Method for accelerating read-write efficiency of nonvolatile memory under Feiteng server platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112148229A CN112148229A (en) | 2020-12-29 |
| CN112148229B true CN112148229B (en) | 2024-01-30 |
Family
ID=73951525
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011083307.4A Active CN112148229B (en) | 2020-10-12 | 2020-10-12 | Method for accelerating read-write efficiency of nonvolatile memory under Feiteng server platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112148229B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112947863B (en) * | 2021-03-25 | 2024-01-30 | 北京计算机技术及应用研究所 | Method for combining storage spaces under Feiteng server platform |
| CN113326213B (en) * | 2021-05-24 | 2023-07-28 | 北京计算机技术及应用研究所 | Method for realizing address mapping in driver under Feiteng server platform |
| CN113326072B (en) * | 2021-05-24 | 2023-11-24 | 北京计算机技术及应用研究所 | Real-time monitoring method based on nonvolatile memory under Feiteng server platform |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102664923A (en) * | 2012-03-30 | 2012-09-12 | 浪潮电子信息产业股份有限公司 | Method for realizing shared storage pool by utilizing Linux global file system |
| CN104298541A (en) * | 2014-10-22 | 2015-01-21 | 浪潮(北京)电子信息产业有限公司 | Data distribution algorithm and data distribution device for cloud storage system |
| CN105050164A (en) * | 2015-01-16 | 2015-11-11 | 中国矿业大学 | Method for lowering wifi power consumption based on data importance |
| CN105095094A (en) * | 2014-05-06 | 2015-11-25 | 华为技术有限公司 | Memory management method and equipment |
| CN105378687A (en) * | 2014-03-07 | 2016-03-02 | 华为技术有限公司 | File accessing method and related device |
| CN105573809A (en) * | 2012-11-15 | 2016-05-11 | 青岛海信宽带多媒体技术有限公司 | Method and device for establishing equipment nodes |
| CN105975407A (en) * | 2016-03-22 | 2016-09-28 | 华为技术有限公司 | Mapping method of memory address, and equipment |
| CN110109778A (en) * | 2019-07-02 | 2019-08-09 | 南京云信达科技有限公司 | A kind of a large amount of small data file backup methods and restoration methods |
| CN111414248A (en) * | 2019-01-04 | 2020-07-14 | 阿里巴巴集团控股有限公司 | Memory management method and device and computing equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9696692B2 (en) * | 2012-04-13 | 2017-07-04 | Rockwell Automation Technologies, Inc. | Industrial automation control system |
-
2020
- 2020-10-12 CN CN202011083307.4A patent/CN112148229B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102664923A (en) * | 2012-03-30 | 2012-09-12 | 浪潮电子信息产业股份有限公司 | Method for realizing shared storage pool by utilizing Linux global file system |
| CN105573809A (en) * | 2012-11-15 | 2016-05-11 | 青岛海信宽带多媒体技术有限公司 | Method and device for establishing equipment nodes |
| CN105378687A (en) * | 2014-03-07 | 2016-03-02 | 华为技术有限公司 | File accessing method and related device |
| CN105095094A (en) * | 2014-05-06 | 2015-11-25 | 华为技术有限公司 | Memory management method and equipment |
| CN104298541A (en) * | 2014-10-22 | 2015-01-21 | 浪潮(北京)电子信息产业有限公司 | Data distribution algorithm and data distribution device for cloud storage system |
| CN105050164A (en) * | 2015-01-16 | 2015-11-11 | 中国矿业大学 | Method for lowering wifi power consumption based on data importance |
| CN105975407A (en) * | 2016-03-22 | 2016-09-28 | 华为技术有限公司 | Mapping method of memory address, and equipment |
| CN111414248A (en) * | 2019-01-04 | 2020-07-14 | 阿里巴巴集团控股有限公司 | Memory management method and device and computing equipment |
| CN110109778A (en) * | 2019-07-02 | 2019-08-09 | 南京云信达科技有限公司 | A kind of a large amount of small data file backup methods and restoration methods |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112148229A (en) | 2020-12-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112148229B (en) | Method for accelerating read-write efficiency of nonvolatile memory under Feiteng server platform | |
| US11892957B2 (en) | SSD architecture for FPGA based acceleration | |
| US10810135B2 (en) | Data transmission method, apparatus, device, and system | |
| US7912051B1 (en) | Techniques for port virtualization | |
| US8639898B2 (en) | Storage apparatus and data copy method | |
| US20100023724A1 (en) | Network Based Virtualization Performance | |
| CN109791522B (en) | Data migration method and system and intelligent network card | |
| US11665046B2 (en) | Failover port forwarding between peer storage nodes | |
| US20240119014A1 (en) | Novel ssd architecture for fpga based acceleration | |
| CN115344197A (en) | Data access method, network card and server | |
| US20180239649A1 (en) | Multi Root I/O Virtualization System | |
| US20220222016A1 (en) | Method for accessing solid state disk and storage device | |
| US10229085B2 (en) | Fibre channel hardware card port assignment and management method for port names | |
| US20190114076A1 (en) | Method and Apparatus for Storing Data in Distributed Block Storage System, and Computer Readable Storage Medium | |
| CN116032930A (en) | Network storage method, storage system, data processing unit and computer system | |
| US11494257B2 (en) | Efficient networking for a distributed storage system | |
| US11379128B2 (en) | Application-based storage device configuration settings | |
| US20060182110A1 (en) | Apparatus, system, and method for fibre channel device addressing | |
| US20130031570A1 (en) | Sas virtual tape drive | |
| US11640311B2 (en) | Providing non-volatile storage for permanent data to virtual machines | |
| US11226756B2 (en) | Indirect storage data transfer | |
| US7434022B1 (en) | Distributed workflow techniques | |
| CN116932451A (en) | Data processing method, host and related equipment | |
| CN114328434A (en) | Data processing system, method, device and storage medium | |
| US10437497B1 (en) | Active-active host environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |