CN107179929B - WRITE _ SAME function optimization implementation method and device - Google Patents
WRITE _ SAME function optimization implementation method and device Download PDFInfo
- Publication number
- CN107179929B CN107179929B CN201710338288.7A CN201710338288A CN107179929B CN 107179929 B CN107179929 B CN 107179929B CN 201710338288 A CN201710338288 A CN 201710338288A CN 107179929 B CN107179929 B CN 107179929B
- Authority
- CN
- China
- Prior art keywords
- written
- data
- write
- byte number
- same function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44568—Immediately runnable code
- G06F9/44578—Preparing or optimising for loading
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
- G06F8/71—Version control; Configuration management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The invention discloses a method and a device for realizing a SCSI instruction WRITE _ SAME function in a ceph + tgt architecture, wherein the method comprises the following steps: allocating a predetermined buffer to a WRITE _ SAME function; repeatedly copying data to be written to the preset cache to fill the preset cache with the data to be written; writing the filled predetermined memory into a back-end memory through an rbd _ write () interface; therefore, compared with the mode that the data writing is realized by frequently calling the rbd _ write () interface in the existing scheme, in the scheme, the data to be written is written into the preset cache in advance, and the data to be written can be written into the back end for storage by calling the rbd _ write () interface once, so that the realization mode of the write _ same is greatly optimized, and the problem of compatibility of a Ceph + tgt architecture to a vmware system is solved.
Description
Technical Field
The invention relates to the technical field of function optimization, in particular to a method and a device for realizing a SCSI instruction WRITE _ SAME function in a ceph + tgt architecture.
Background
The architecture of the Ceph + tgt can externally realize a standard iscsi protocol interface for distributed storage realized by the Ceph through the tgt, so that a client can be accessed to the distributed storage realized by the Ceph through a standard iscsi initiator terminal without other complex modes. The compatibility problem of the architecture of the Ceph + tgt to vmware is a problem that most storage products have to face, and when the architecture of the Ceph + tgt is applied to a vmware system as a client, the speed of loading a virtual machine in a mapped lun is found to be very slow, and after testing, the speed of copying data to the mapped lun is also found to be very slow, and the speed of a back end is about 70K/S, so that the speed of loading the virtual machine in the mapped lun is very slow.
Therefore, how to solve the above problems and improve the compatibility of the storage architecture of ceph + tgt and the vmware system are problems to be solved by those skilled in the art.
Disclosure of Invention
The invention aims to provide a method and a device for realizing a WRITE _ SAME function of an SCSI instruction in a ceph + tgt architecture, so as to improve the compatibility of the storage architecture of the ceph + tgt and a vmware system.
In order to achieve the above purpose, the embodiment of the present invention provides the following technical solutions:
a method for implementing a SCSI instruction WRITE _ SAME function in a ceph + tgt architecture comprises the following steps:
allocating a predetermined buffer to a WRITE _ SAME function;
repeatedly copying data to be written to the preset cache to fill the preset cache with the data to be written;
writing the filled predetermined memory to a back-end storage through an rbd _ write () interface.
Wherein the allocating a predetermined buffer to the WRITE _ SAME function includes:
determining a first byte number of data to be written;
determining a second byte number of a preset cache according to the first byte data;
allocating a predetermined buffer of the second number of bytes to a WRITE _ SAME function.
Wherein the allocating the predetermined buffer of the second byte number to the WRITE _ SAME function includes: and allocating a predetermined buffer of the second byte number to the WRITE _ SAME function by using a malloc () function.
Wherein the repeatedly copying the data to be written to the predetermined cache to fill the predetermined cache with the data to be written includes:
acquiring data to be written by utilizing a scsi _ get _ out _ buffer (cmd) function;
and repeatedly copying the data to be written in the first byte number to the preset cache of the second byte number so as to fill the preset cache of the second byte number with the data to be written in the first byte number.
Wherein the repeatedly copying the data to be written in the first byte number to the predetermined cache in the second byte number includes:
and repeatedly copying the data to be written of the first byte number to a preset cache of the second byte number through a memcpy function interface or a strncpy function interface.
An apparatus for implementing a WRITE _ SAME function in a SCSI command in a ceph + tgt architecture, comprising:
the allocation module is used for allocating a preset buffer to the WRITE _ SAME function;
the copying module is used for repeatedly copying data to be written to the preset cache so as to fill the preset cache with the data to be written;
and the data writing module is used for writing the filled predetermined memory into the back-end storage through an rbd _ write () interface.
Wherein the allocation module comprises:
the first determining unit is used for determining a first byte number of data to be written;
a second determining unit, configured to determine a second byte number of a predetermined cache according to the first byte data;
and the allocation unit is used for allocating the preset buffer memory of the second byte number to the WRITE _ SAME function.
Wherein the allocation unit allocates the predetermined buffer of the second byte number to the WRITE _ SAME function using a malloc () function.
Wherein the replication module comprises:
an obtaining unit, configured to obtain data to be written by using a scsi _ get _ out _ buffer (cmd) function;
and the copying unit is used for repeatedly copying the data to be written in the first byte number to the preset cache of the second byte number so as to fill the preset cache of the second byte number with the data to be written in the first byte number.
And the copying unit is used for repeatedly copying the data to be written of the first byte number to a preset buffer memory of the second byte number through a memcpy function interface or a strncpy function interface.
As can be seen from the above solutions, the method for implementing the SCSI command WRITE _ SAME function in the ceph + tgt architecture provided in the embodiment of the present invention includes: allocating a predetermined buffer to a WRITE _ SAME function; repeatedly copying data to be written to the preset cache to fill the preset cache with the data to be written; writing the filled predetermined memory into a back-end memory through an rbd _ write () interface;
therefore, compared with the mode that the data writing is realized by frequently calling the rbd _ write () interface in the existing scheme, in the scheme, the data to be written is written into the preset cache in advance, and the data to be written can be written into the rear end for storage by calling the rbd _ write () interface once, so that the realization mode of write _ same is greatly optimized, and the problem of compatibility of a Ceph + tgt architecture to a vmware system is solved; the invention also discloses a device for realizing the SCSI instruction WRITE _ SAME function in the ceph + tgt architecture, which can also realize the technical effect.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flowchart of a method for implementing an SCSI command WRITE _ SAME function in a ceph + tgt architecture according to an embodiment of the present invention;
fig. 2 is a schematic diagram illustrating that data to be written is copied in a predetermined cache according to an embodiment of the disclosure;
fig. 3 is a schematic structural diagram of an implementation apparatus of a SCSI command WRITE _ SAME function in a ceph + tgt architecture according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that when the architecture of Ceph + tgt is applied to a vmware system as a client, it is found that the speed of copying data and loading a virtual machine in a mapped lun is very slow, thereby causing a compatibility problem of the architecture with the vmware system. The implementation manner of the vmware system is different from that of a common system (Linux, windows, or the like) when copying data, the common system only needs a write instruction to copy a file, and the vmware system needs a write _ same instruction besides the common write instruction to copy data, and the write _ same instruction is briefly described as follows:
write _ same mainly implements two functions:
1. unmap function: the realization of unmap function directly calls the function interface of rbd _ discard () in ceph;
2. data of one LBA is read from inside the write _ buf and then written to consecutive LBAs from a certain address.
It should be noted that, in function 2, the rbd _ write () function interface of ceph is mainly encapsulated, the main related source code file is line 271 and line 308 in the Bs _ rbd.c file under the user directory in tgt, and the specific source code is:
by analyzing the source code, it can be found that the main process of the way of implementing write _ same is as follows:
1. obtaining the size of the blocksize as 512B through 1< < cmd- > dev- > blk _ shift;
2. obtaining data to be written by tmpbuf _ scsi _ get _ out _ buffer (cmd); where tmpbuf is 512B in size;
3. through a while loop, the loop calls the rbd _ write () interface tl/blocksize times.
In the process, the implementation manner of the rbd _ write () interface needs to pass through a plurality of processes at the back end, the implementation process is complex and time-consuming, and the process for implementing the write _ same frequently writes small blocks of data and frequently calls the rbd _ write () interface for a plurality of times, which is time-consuming.
Therefore, the embodiment of the invention discloses a method and a device for realizing a WRITE _ SAME function of an SCSI instruction in a ceph + tgt architecture, which can greatly reduce the calling frequency of an rbd _ WRITE () function interface and save time resources by realizing adjustment on a logic architecture in a WRITE _ SAME, thereby improving the performance of the WRITE _ SAME and improving the compatibility of the storage architecture of the ceph + tgt and a vmware system.
Referring to fig. 1, an implementation method of a WRITE _ SAME function in an SCSI command in a ceph + tgt architecture provided by the embodiment of the present invention includes:
s101, distributing a preset buffer to a WRITE _ SAME function;
wherein the allocating a predetermined buffer to the WRITE _ SAME function includes:
determining a first byte number of data to be written;
determining a second byte number of a preset cache according to the first byte data;
allocating a predetermined buffer of the second number of bytes to a WRITE _ SAME function.
Specifically, the WRITE _ SAME function may be allocated with a predetermined buffer of the second number of bytes by using a malloc () function.
In this embodiment, a buffer of the second byte number tl may be allocated in the memory, specifically, may be allocated using a malloc () function, and the second byte number t1 may be determined according to the first byte number blocksize of the data to be written, for example: the second byte count may be set to be an integer multiple of the first byte count so as to repeatedly write the data to be written into the predetermined buffer.
S102, repeatedly copying data to be written to the preset cache to fill the preset cache with the data to be written;
wherein the repeatedly copying the data to be written to the predetermined cache to fill the predetermined cache with the data to be written includes:
acquiring data to be written by utilizing a scsi _ get _ out _ buffer (cmd) function;
and repeatedly copying the data to be written in the first byte number to the preset cache of the second byte number so as to fill the preset cache of the second byte number with the data to be written in the first byte number.
Referring to fig. 2, in the present scheme, read-write data obtained from a command, that is, tmsi _ get _ out _ buffer (cmd), to-be-written data tmpbuf is repeatedly copied into the allocated memory until the buffer with the number of tl bytes is filled, which may be specifically implemented by a function interface such as memcpy or strcpy.
S103, writing the filled predetermined memory into the back-end storage through an rbd _ write () interface.
It can be understood that, in the present solution, the above filled buffer is written into the backend storage at one time through the rbd _ write () interface, and specifically, the rbd _ write (rbd- > rbd _ image, offset, tl, buffer) can be called to implement. Therefore, through the implementation mode of the write _ same, only one-time rbd _ write () write interface is needed to write data into the back-end storage, so that the implementation mode of the write _ same is greatly optimized, and the problem of compatibility of a Ceph + tgt architecture to a vmware system is solved.
The following describes an implementation apparatus provided in an embodiment of the present invention, and the implementation apparatus described below and the implementation method described above may be referred to each other.
Referring to fig. 3, an apparatus for implementing a WRITE _ SAME function in an SCSI command in a ceph + tgt architecture according to an embodiment of the present invention includes:
an allocating module 100, configured to allocate a predetermined buffer to a WRITE _ SAME function;
the copying module 200 is configured to repeatedly copy data to be written to the predetermined cache, so as to fill the predetermined cache with the data to be written;
a data writing module 300, configured to write the filled predetermined memory into the backend storage through the rbd _ write () interface.
Based on the above embodiment, the allocation module includes:
the first determining unit is used for determining a first byte number of data to be written;
a second determining unit, configured to determine a second byte number of a predetermined cache according to the first byte data;
and the allocation unit is used for allocating the preset buffer memory of the second byte number to the WRITE _ SAME function.
Based on the above embodiment, the allocating unit allocates the predetermined buffer of the second byte number to the WRITE _ SAME function using a malloc () function.
Based on the above embodiment, the copy module includes:
an obtaining unit, configured to obtain data to be written by using a scsi _ get _ out _ buffer (cmd) function;
and the copying unit is used for repeatedly copying the data to be written in the first byte number to the preset cache of the second byte number so as to fill the preset cache of the second byte number with the data to be written in the first byte number.
Based on the above embodiment, the copy unit repeatedly copies the data to be written in the first byte number to the predetermined buffer in the second byte number through a memcpy function interface or a strncpy function interface.
In summary, under the architecture of Ceph + tgt, there is a certain problem in the implementation of write _ same in tgt, which mainly lies in frequently calling the rbd _ write () write interface, and the implementation of this rbd _ write () write interface is very complex and consumes time resources.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method for implementing a SCSI instruction WRITE _ SAME function in a ceph + tgt architecture is characterized by comprising the following steps:
allocating a predetermined buffer to a WRITE _ SAME function;
repeatedly copying data to be written to the preset cache to fill the preset cache with the data to be written;
and writing the data to be written in the filled predetermined cache into a back-end memory through an rbd _ write () interface.
2. The method of claim 1, wherein said assigning a predetermined buffer to a WRITE _ SAME function comprises:
determining a first byte number of data to be written;
determining a second byte number of a preset cache according to the first byte number;
allocating a predetermined buffer of the second number of bytes to a WRITE _ SAME function.
3. The method of claim 2, wherein said assigning a predetermined buffer of said second number of bytes to a WRITE _ SAME function comprises:
and allocating a predetermined buffer of the second byte number to the WRITE _ SAME function by using a malloc () function.
4. The method according to claim 3, wherein the repeatedly copying the data to be written to the predetermined buffer to fill the predetermined buffer with the data to be written comprises:
acquiring data to be written by utilizing a scsi _ get _ out _ buffer (cmd) function;
and repeatedly copying the data to be written in the first byte number to the preset cache of the second byte number so as to fill the preset cache of the second byte number with the data to be written in the first byte number.
5. The method of claim 4, wherein the repeatedly copying the first byte number of data to be written to the predetermined buffer of the second byte number comprises:
and repeatedly copying the data to be written of the first byte number to a preset cache of the second byte number through a memcpy function interface or a strncpy function interface.
6. An apparatus for implementing a WRITE _ SAME function in a SCSI command in a ceph + tgt architecture, comprising:
the allocation module is used for allocating a preset buffer to the WRITE _ SAME function;
the copying module is used for repeatedly copying data to be written to the preset cache so as to fill the preset cache with the data to be written;
and the data writing module is used for writing the filled data to be written in the preset cache into the back-end storage through an rbd _ write () interface.
7. The apparatus of claim 6, wherein the allocation module comprises:
the first determining unit is used for determining a first byte number of data to be written;
a second determining unit, configured to determine a second byte number of the predetermined cache according to the first byte number;
and the allocation unit is used for allocating the preset buffer memory of the second byte number to the WRITE _ SAME function.
8. The implementation device of claim 7,
the allocation unit allocates a predetermined buffer of the second byte number to the WRITE _ SAME function using a malloc () function.
9. The apparatus of claim 8, wherein the replication module comprises:
an obtaining unit, configured to obtain data to be written by using a scsi _ get _ out _ buffer (cmd) function;
and the copying unit is used for repeatedly copying the data to be written in the first byte number to the preset cache of the second byte number so as to fill the preset cache of the second byte number with the data to be written in the first byte number.
10. The implementation device of claim 9,
and the copying unit is used for repeatedly copying the data to be written of the first byte number to a preset buffer memory of the second byte number through a memcpy function interface or a strncpy function interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710338288.7A CN107179929B (en) | 2017-05-15 | 2017-05-15 | WRITE _ SAME function optimization implementation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710338288.7A CN107179929B (en) | 2017-05-15 | 2017-05-15 | WRITE _ SAME function optimization implementation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107179929A CN107179929A (en) | 2017-09-19 |
| CN107179929B true CN107179929B (en) | 2020-02-07 |
Family
ID=59831105
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710338288.7A Active CN107179929B (en) | 2017-05-15 | 2017-05-15 | WRITE _ SAME function optimization implementation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107179929B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110795043B (en) * | 2019-10-29 | 2023-06-06 | 北京浪潮数据技术有限公司 | Distributed storage block zeroing method and device, electronic equipment and storage medium |
| CN114579062B (en) * | 2022-04-29 | 2022-08-05 | 深圳市杉岩数据技术有限公司 | Disk optimization method and device based on distributed storage system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102541878A (en) * | 2010-12-17 | 2012-07-04 | 新奥特（北京）视频技术有限公司 | Method and device for storing project file in non-linear editing |
| CN105450734A (en) * | 2015-11-09 | 2016-03-30 | 上海爱数信息技术股份有限公司 | Distributed storage CEPH data distribution optimization method |
| CN105471989A (en) * | 2015-11-23 | 2016-04-06 | 上海爱数信息技术股份有限公司 | Data storage method |
| CN106527985A (en) * | 2016-11-02 | 2017-03-22 | 郑州云海信息技术有限公司 | Storage interaction device and storage system based on ceph |
-
2017
- 2017-05-15 CN CN201710338288.7A patent/CN107179929B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102541878A (en) * | 2010-12-17 | 2012-07-04 | 新奥特（北京）视频技术有限公司 | Method and device for storing project file in non-linear editing |
| CN105450734A (en) * | 2015-11-09 | 2016-03-30 | 上海爱数信息技术股份有限公司 | Distributed storage CEPH data distribution optimization method |
| CN105471989A (en) * | 2015-11-23 | 2016-04-06 | 上海爱数信息技术股份有限公司 | Data storage method |
| CN106527985A (en) * | 2016-11-02 | 2017-03-22 | 郑州云海信息技术有限公司 | Storage interaction device and storage system based on ceph |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107179929A (en) | 2017-09-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10452562B2 (en) | File access method and related device | |
| US10078453B1 (en) | Storage system, computer program product, and method for managing a hybrid memory device system | |
| US10572379B2 (en) | Data accessing method and data accessing apparatus | |
| US20190278507A1 (en) | Data Migration Method, Host, and Solid State Disk | |
| US10635356B2 (en) | Data management method and storage controller using the same | |
| EP3506075A1 (en) | Mass storage device capable of fine grained read and/or write operations | |
| TWI715248B (en) | Apparatus and method and computer program product for executing host input-output commands | |
| KR20140146458A (en) | Method for managing memory and apparatus thereof | |
| CN109799959B (en) | Method for improving parallelism of open channel solid-state disk writing | |
| US10049042B2 (en) | Storage device, semiconductor memory device, and method for controlling same | |
| US9229891B2 (en) | Determining a direct memory access data transfer mode | |
| CN107179929B (en) | WRITE _ SAME function optimization implementation method and device | |
| US20240086113A1 (en) | Synchronous write method and device, storage system and electronic device | |
| US9645946B2 (en) | Encryption for solid state drives (SSDs) | |
| US20150261436A1 (en) | Flexible storage block for a solid state drive (ssd)-based file system | |
| CN104360956A (en) | Method and device for copying data | |
| KR20150096177A (en) | Method for performing garbage collection and flash memory apparatus using the method | |
| US9195595B2 (en) | Portable apparatus using multi-core storage mechanism and data access method therefor | |
| US10922003B1 (en) | Realizing host-assisted device-level data deduplication on solid-state data storage devices with embedded non-volatile memory | |
| CN111444113B (en) | Nonvolatile storage medium sharing method and device, electronic equipment and storage equipment | |
| CN109032965B (en) | Data reading method, host and storage device | |
| US11797183B1 (en) | Host assisted application grouping for efficient utilization of device resources | |
| US11853203B1 (en) | Systems and methods with variable size super blocks in zoned namespace devices | |
| US10579539B2 (en) | Storage infrastructure and method for exploiting in-storage transparent compression using a dummy file that consumes LBA storage without consuming PBA storage | |
| US11409664B2 (en) | Logical memory allocation and provisioning using timestamps |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20191203 Address after: 215100 No. 1 Guanpu Road, Guoxiang Street, Wuzhong Economic Development Zone, Suzhou City, Jiangsu Province Applicant after: Suzhou Wave Intelligent Technology Co., Ltd. Address before: 450018 Henan province Zheng Dong New District of Zhengzhou City Xinyi Road No. 278 16 floor room 1601 Applicant before: Zhengzhou Yunhai Information Technology Co. Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |