CN109358820B - Data access method and device, electronic equipment and computer readable storage medium - Google Patents
Data access method and device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN109358820B CN109358820B CN201811476095.9A CN201811476095A CN109358820B CN 109358820 B CN109358820 B CN 109358820B CN 201811476095 A CN201811476095 A CN 201811476095A CN 109358820 B CN109358820 B CN 109358820B
- Authority
- CN
- China
- Prior art keywords
- data
- slice
- field
- hard disk
- access
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0661—Format or protocol conversion arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
- G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application discloses a data access method, a data access device, electronic equipment and a computer readable storage medium. The data access method comprises the following steps: acquiring access data on a target hard disk; and encapsulating the access data by using a general encapsulation protocol, and sending the access data to the target storage array at the far end through an optical fiber. According to the data access method, the SATA FIS frame can be sent to the remote target storage array through the optical fiber after being packaged by the aid of the general packaging protocol, protocol conversion is not needed, various transmission protocols such as data slicing and non-slicing are supported, remote data interaction between the server and the storage array is achieved, distance limitation between the server and the storage array is broken, and the large-scale storage system is more flexibly and conveniently deployed.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data access method, an apparatus, an electronic device, and a computer-readable storage medium.
Background
With the rapid development of the internet and the internet of things, the amount of data generated and consumed increases rapidly, and the contradiction between the amount of data and the storage space is increasingly prominent. The capacity expansion mode of a traditional large-scale storage system is generally realized by adopting a server stacking mode, servers are connected through a high-speed switch, meanwhile, data exchange between the servers needs participation of software components such as an operating system and a file system, the cost is high, and large waste of computing resources and power resources and performance reduction are caused. Another capacity expansion mode can use an FC optical fiber storage network, the FC optical fiber storage network has two topological structures of ring topology and mesh topology, a plurality of storage nodes of the ring topology are connected end to end, the bandwidth is shared, and the method is suitable for small-scale storage; the mesh topology has higher bandwidth and data exchange flexibility, but a dedicated FC switch is needed to connect each storage node, so the cost is high, and at the same time, the MTU of the FC protocol transmits 2112 bytes of data payload at maximum, so when the FC protocol is used to carry a standard SATA FIS structure, the FIS needs to be sliced and encapsulated in many cases, and the processing complexity is also increased.
In addition, in the related art, when the remote storage node is connected by the optical fiber, protocol conversion is required, transmission data is converted into an FC protocol and transmitted in the optical fiber, and then converted back into an SATA protocol at a remote end to operate the disk. The related art cannot directly transmit the SATA packet through the optical fiber because of problems of the protocol structure and the data structure.
Disclosure of Invention
The embodiment of the application provides a data access method, a data access device, electronic equipment and a computer-readable storage medium.
In a first aspect, an embodiment of the present application provides a data access method.
Specifically, the data access method includes:
acquiring access data on a target hard disk; wherein the target hard disk is located on a target storage array in a cascaded storage array system;
packaging the access data by using a general packaging protocol, and sending the access data to the target storage array at a far end through an optical fiber; the general encapsulation protocol at least comprises a service type field, a slice information field, a destination address field and a data payload field.
Further, the access data comprises SATA FIS frames and/or user-defined data.
Further, in the process of writing data to the target hard disk, acquiring access data on the target hard disk includes:
acquiring a data writing command from a server memory; and/or the presence of a gas in the gas,
and acquiring data to be written from the server memory.
Further, the slice information field includes whether a last slice field and a slice number field.
Further, when a write command for writing data to the target hard disk is acquired from the memory of the server, in the encapsulated access data, the service type field is an SATA FIS frame identifier, the slice information field is a non-slice identifier, the address field is a physical address of the target hard disk, and the data payload field is a write command.
Further, when the data to be written is acquired from the memory of the server and slice packaging is performed, in the packaged access data, the service type field is SATA FIS frame identification, the slice information field includes a slice number and an identification of whether the last slice is included, the address field is a physical address of a target hard disk, and the data payload field is the data to be written; and/or the presence of a gas in the gas,
when data to be written is acquired from a server memory and non-slicing packaging is carried out, in the packaged access data, the service type field is a written data identifier, the slicing information field comprises a non-slicing identifier, the address field is a physical address of a target hard disk, and the data payload field is the data to be written.
Further, when reading data to the target storage array, after encapsulating the access data by using a general encapsulation protocol and sending the encapsulated access data to a target hard disk, the method further includes:
receiving packed data from the target storage array;
and analyzing the packaging data, and determining the data to be read according to the slice information field in the packaging data.
Further, determining data to be read according to a slice information field in the encapsulated data includes:
when the slice information field comprises an identifier of no slice, determining the data in the data payload field as the data to be read; and/or the presence of a gas in the gas,
and when the slice information field comprises the slice identifier, determining the data in the data payload field as the corresponding part in the data to be read according to the slice number in the slice information field and whether the identifier of the last slice is included.
In a second aspect, an embodiment of the present application provides a data access apparatus.
Specifically, the data access device includes:
the first acquisition module is configured to acquire access data on a target storage array hard disk; wherein the target hard disk is located on a target storage array in a cascaded storage array system;
the packaging module is configured to package the access data by using a general packaging protocol and send the access data to the target storage array at a remote end through an optical fiber; the generic encapsulation protocol at least comprises a service type field, a slice information field, an address field and a destination address field of the target storage array and a data payload field.
Further, the access data comprises SATA FIS frame access commands and/or write data user-defined data.
Further, in the process of writing data to the target hard disk, the first obtaining module includes:
the first obtaining submodule is configured to obtain a write data command from a server memory; and/or the presence of a gas in the gas,
and the second obtaining submodule is configured to obtain the data to be written from the server memory.
Further, the slice information field includes whether a last slice field and a slice number field.
Further, when a write command for writing data to the target hard disk is acquired from the memory of the server, in the encapsulated access data, the service type field is an SATA FIS frame identifier, the slice information field is a non-slice identifier, the address field is a physical address of the target hard disk, and the data payload field is a write command.
Further, when the data to be written is acquired from the memory of the server and slice packaging is performed, in the packaged access data, the service type field is SATA FIS frame identification, the slice information field includes a slice number and an identification of whether the last slice is included, the address field is a physical address of a target hard disk, and the data payload field is the data to be written; and/or the presence of a gas in the gas,
when data to be written is acquired from a server memory and non-slicing packaging is carried out, in the packaged access data, the service type field is a written data identifier, the slicing information field comprises a non-slicing identifier, the address field is a physical address of a target hard disk, and the data payload field is the data to be written.
Further, when reading data to the target storage array, after encapsulating the access data by using a general encapsulation protocol and sending the encapsulated access data to a target hard disk, the method further includes:
a receiving module configured to receive packed data from the target storage array;
and the determining module is configured to analyze the packaging data and determine the data to be read according to the slice information field in the packaging data.
Further, the determining module includes:
a first determining submodule configured to determine data in the data payload field as data to be read when the slice information field includes an identifier of no slice; and/or the presence of a gas in the gas,
and the second determining submodule is configured to determine the data in the data payload field as a corresponding part in the data to be read according to the slice number in the slice information field and whether the identifier of the last slice is included in the slice information field when the identifier of the slice is included in the slice information field.
The functions can be realized by hardware, and the functions can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the above-described functions.
In one possible design, the data access apparatus includes a memory and a processor, the memory is used for storing one or more computer instructions for supporting the data access apparatus to execute the data access method in the first aspect, and the processor is configured to execute the computer instructions stored in the memory. The data access device may further comprise a communication interface for the data access device to communicate with other devices or a communication network.
In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor; wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps of the first aspect.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing computer instructions for a data access apparatus, which contains computer instructions for executing the data access method in the first aspect.
The technical scheme provided by the embodiment of the application can have the following beneficial effects:
in the embodiment of the application, when a server of a cascade storage array system accesses data of a remote storage array, access data is firstly obtained, then the access data is packaged by using a general packaging protocol and then is sent to a target storage array through light; the general encapsulation protocol at least comprises a service type field, a slice information field, an address field of a target storage array and a data payload field. According to the data access method, the SATA FIS frame can be sent to the remote target storage array through the optical fiber after being packaged by the aid of the general packaging protocol, protocol conversion is not needed, various transmission protocols such as data slicing and data non-slicing are supported, remote data interaction between the server and the storage array is achieved, distance limitation between the server and the storage array is broken, and the large-scale storage system is more flexibly and conveniently deployed.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
Other features, objects, and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments when taken in conjunction with the accompanying drawings. In the drawings:
FIG. 1 illustrates a flow diagram of a data access method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a cascaded memory array system according to an embodiment of the present application;
FIG. 3 illustrates a flow chart of a data access method after step S102 according to an embodiment of the present application;
FIG. 4 is a schematic diagram illustrating a process of a server interacting with a hard disk via optical fiber according to an embodiment of the present application;
FIG. 5 shows a block diagram of a data access device according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device suitable for implementing a data access method according to an embodiment of the present application.
Detailed Description
Hereinafter, exemplary embodiments of the present application will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. Also, for the sake of clarity, parts not relevant to the description of the exemplary embodiments are omitted in the drawings.
In the present application, it is to be understood that terms such as "including" or "having," etc., are intended to indicate the presence of the disclosed features, numbers, steps, actions, components, parts, or combinations thereof, and are not intended to preclude the possibility that one or more other features, numbers, steps, actions, components, parts, or combinations thereof may be present or added.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 shows a flow chart of a data access method according to an embodiment of the present application. As shown in fig. 1, the data access method includes the following steps S101 to S102:
in step S101, access data to the hard disk is acquired; the hard disk is positioned on a target storage array in the cascade storage array system;
in step S102, the access data is encapsulated by using a general encapsulation protocol, and is sent to a remote target storage array through an optical fiber; the general encapsulation protocol at least comprises a service type field, a slice information field, a destination address field and a data payload field.
In this embodiment, the target storage array may be any storage array in a cascaded storage array system. The cascade storage array system can comprise a plurality of cascade branches, and each cascade branch is connected with a plurality of storage arrays at the upper level; the plurality of cascaded branches are connected to the server through a plurality of optical ports on the server. In an embodiment, a PCIE card is disposed inside the server, and a plurality of optical ports may be provided to the outside, and the first storage array in each cascade branch is connected to one of the optical ports on the PCIE card through the first optical port.
Fig. 2 is a schematic structural diagram of a cascade memory array system according to an embodiment of the present invention. As shown in fig. 2, the server 1 includes M (M > ═ 1) optical ports, and M cascade branches 2-1 to 2-M connected to the storage arrays respectively, where in the view of the figure, the M cascade branches are formed by cascading N (N > ═ 1) storage arrays 3-x-y, and the physical addresses of the storage arrays can be represented as 3-x-y, where 3 is a device role identifier (1 represents the server, 2 represents the cascade branches, and 3 represents the storage arrays), x represents a cascade branch identifier (including 1 to M), and y represents a cascade depth identifier (including 1 to N).
In the embodiment of the application, in the topology structure of the cascaded storage array system, the optical fiber connection mode is adopted between the server and the storage arrays and between the storage arrays in each cascaded branch, so that the cascaded storage array system has the characteristics of large bandwidth, high efficiency, interference resistance and the like, and meanwhile, the optical fiber transmission breaks through the distance limitation of cable transmission, so that the server can directly and remotely access and manage the hard disks on the storage arrays.
The embodiment of the application also provides a general encapsulation protocol so that after a standard SATA FIS frame can be encapsulated, the SATA FIS frame can be directly transmitted on optical fibers between the server and the storage array and between the cascaded storage arrays, and the problem that the SATA FIS frame cannot be directly transmitted by the optical fibers in the related technology is solved. The general encapsulation protocol proposed in the embodiment of the present application at least includes a service type field, a slice information field, a destination address field, and a data payload field. The service type field can be used for identifying whether the current access data is SATA data; the slice information field may be used to identify information related to the SATA data slice in the current encapsulation packet, such as whether the SATA data currently being transmitted is sliced, whether the SATA data is the last slice, the serial number of the slice, and the like; the destination address field is used for identifying a destination address to be accessed by the server, such as a target storage array address and/or a hard disk address; the data payload field is used to record the data payload of the current transmission, such as a SATA FIS frame. Of course, it should be noted that, in addition to the above fields, the general encapsulation protocol may also include other fields, which may be set according to actual situations, and are not limited herein.
In the embodiment of the application, when a server of a cascade storage array system accesses data of a remote storage array, access data is firstly obtained, then the access data is packaged by using a general packaging protocol and then is sent to a target storage array through light; the general encapsulation protocol at least comprises a service type field, a slice information field, an address field of a target storage array and a data payload field. According to the data access method, the SATA FIS frame can be sent to the remote target storage array through the optical fiber after being packaged by the aid of the general packaging protocol, protocol conversion is not needed, various transmission protocols such as data slicing and data non-slicing are supported, remote data interaction between the server and the storage array is achieved, distance limitation between the server and the storage array is broken, and the large-scale storage system is more flexibly and conveniently deployed.
The following is an exemplary structure field of the generic encapsulation protocol in an exemplary embodiment, and it should be noted that the generic encapsulation protocol structure in the embodiment of the present application is not limited to the following specific structure, and may be modified accordingly according to actual situations. The generic encapsulation protocol structure includes:
ver: the version number of the 4-bit protocol can be 1;
type: the 4-bit service type can be 0(SATA data) or 1(OAM management data);
flag: an 8-bit flag bit, which is bit0, called L flag, associated with the encapsulation, indicating whether it is the last encapsulation slice;
status: 8bit status bits;
frg: 8bit slice number, Frg fixed to 0 when data is not sliced, Frg incremented when sliced;
tag: a 16bit service number;
HD Addr: 16bit hard disk address;
payload: the data payload area with variable length can be filled with SATA FIS or user-defined data.
In an optional implementation manner of this embodiment, the access data includes a SATA FIS frame and/or user-defined data.
In this optional implementation, the generic encapsulation protocol proposed in the present application may be used to encapsulate both standard SATA FIS frames and user-defined data. The type of service field included in the generic encapsulation protocol may be used to identify whether the currently encapsulated data is a SATA FIS frame or user-defined data.
In an optional implementation manner of this embodiment, in the process of writing data to the target hard disk, the step S101, that is, the step of acquiring access data to the target hard disk, further includes the following steps:
acquiring a write command from a server memory; and/or the presence of a gas in the gas,
and acquiring data to be written from the server memory.
In the optional implementation manner, in the process of writing data to the target hard disk by the server, the unit executing the write command on the server acquires the write command from the memory, where the write command may be a standard SATA FIS frame, and after acquiring the write command, the write command is encapsulated by using a general encapsulation protocol, and then is sent to the target storage array where the target hard disk is located, and then is sent to the target hard disk by the target storage array; and after the encapsulated write command is sent, acquiring data to be written into the target hard disk from the internal memory of the server, generating a standard SATA FIS frame, encapsulating by using a general encapsulation protocol, sending to the target storage array after encapsulation, and then sending to the target hard disk by the target storage array.
In an alternative implementation of this embodiment, the slice information field includes whether the last slice field and a slice number field.
In this optional implementation manner, because the maximum data payload bytes that can be carried by different transport protocols are different, some transport protocols need to divide the SATA FIS frame into multiple pieces for transmission, and some transport protocols do not need to transmit. Therefore, a slice information field is set in the generic encapsulation protocol to identify whether the currently transmitted packet is a complete SATA FIS frame, the last slice, the slice number (the second slice), etc. Through the slicing information field, the hard disk can restore the data packet after receiving the data packet to obtain a complete SATA FIS frame.
In an optional implementation manner of this embodiment, when a write command for writing data to the target hard disk is acquired from the memory of the server, in the encapsulated access data, the service type field is an SATA FIS frame identifier, the slice information field is a non-slice identifier, the address field is a physical address of the target hard disk, and the data payload field is the write command.
In this optional implementation manner, when the server writes data to the target hard disk, the CPU on the server puts the write command into the command queue, and executes a component (e.g., an FPGA on a PCIE card) accessing the hard disk data to acquire the write command from the command queue, and generates a standard SATA FIS frame, and then encapsulates the frame by using a general encapsulation protocol and sends the frame to the target hard disk on the target storage array. At the time of packaging, the service type identifier in the packaging structure is set as a SATA FIS frame identifier, the slice information field is set as a non-slice identifier, the address field is set as a physical address of a target hard disk (including an address of a target storage array), and the data payload field is a write command. The physical address of the target hard disk may include the physical address of the target storage array and the physical address of the target hard disk in the target storage array.
In an optional implementation manner of this embodiment, when data to be written is acquired from a memory of a server and slice encapsulation is performed, in the encapsulated access data, the service type field is an SATA FIS frame identifier, the slice information field includes a slice number and an identifier of whether a last slice is included, the address field is a physical address of a target hard disk, and the data payload field is the data to be written. The physical address of the target hard disk may include the physical address of the target storage array and the physical address of the target hard disk in the target storage array.
In this alternative implementation, after the write command is sent, the data to be written needs to be sent to the target hard disk. The CPU on the server puts the data to be written into the memory, and the component (e.g., FPGA on the PCIE card) that accesses the hard disk data acquires the data to be written from the data memory, generates a standard SATA FIS frame, encapsulates the frame using the general encapsulation protocol, and sends the frame to the target hard disk. The encapsulation of data to be written can be divided into two types, one is slicing encapsulation and one is non-slicing encapsulation, depending on the transmission protocol.
When the slice is packaged, the service type identifier in the packaging structure is set as a SATA FIS frame identifier, the slice number field in the slice information field is set as a specific second slice identifier, whether the field of the last slice is set as a yes or no identifier according to the actual situation or not, the address field is set as the physical address of the target hard disk (including the address of the target storage array), and the data payload field is the current slice of the data to be written.
When the data is not sliced, the service type identifier in the packaging structure is set as a SATA FIS frame identifier, the slice information field is set as no slice, the address field is set as a physical address of a target hard disk (including an address of a target storage array), and the data payload field is a current slice of data to be written.
In an optional implementation manner of this embodiment, as shown in fig. 3, when reading data to the target storage array, after step S102, that is, after the step of encapsulating the access data by using a general encapsulation protocol and sending the encapsulated access data to the target hard disk, the method further includes the following steps S301 to S302:
in step S301, receiving packed data from the target storage array;
in step S302, the package data is analyzed, and the data to be read is determined according to the slice information field in the package data.
In this optional implementation manner, when the server reads data from the target hard disk, the server sends a read command encapsulated by using a general encapsulation protocol to the target storage array, and the control component on the target storage array analyzes the read command, acquires data to be read from the target hard disk according to the read command, encapsulates the data by using the general encapsulation protocol, and reports the encapsulated data to the server. Because the used transmission protocols are different, the data to be read returned by the target storage array may be packaged after being sliced, so the server can determine whether the received data is complete data to be read or a slice of the data to be read according to the slice information field in the received data packet, and if the received data is a slice of the data to be read and the slice is not the last slice, the server can continue to wait for receiving other slices to finally form the data to be read.
In an optional implementation manner of this embodiment, the step S302, namely, the step of analyzing the encapsulated data and determining data to be read according to a slice information field in the encapsulated data, further includes the following steps:
when the slice information field comprises an identifier of no slice, determining the data in the data payload field as the data to be read; and/or the presence of a gas in the gas,
and when the slice information field comprises the slice identifier, determining the data in the data payload field as the corresponding part in the data to be read according to the slice number in the slice information field and whether the identifier of the last slice is included.
In this alternative implementation, the data to be read may be further determined according to a slice information field in the encapsulation data. Because the used transmission protocols are different, the data to be read returned by the target storage array may be packaged after being sliced, so the server can determine whether the received data is complete data to be read or a slice of the data to be read according to the slice information field in the received data packet, and if the received data is a slice of the data to be read and the slice is not the last slice, the server can continue to wait for receiving other slices to finally form the data to be read.
The foregoing data access method proposed in this embodiment of the present application is described below by a specific implementation example, and as shown in fig. 4, a process of a server performing remote data interaction with a hard disk through an optical fiber is as follows:
the method comprises the steps that a PCIE card is initialized at the starting stage of a server system, and a command queue storage space address and a service state response storage space address are configured;
when a hard disk access service request exists, a CPU on a server stores service data in a DDR RAM, generates command data and puts the command data into a command queue, and then informs an FPGA1 on a PCIE card to start executing a command through a PCIE interface;
after receiving the notification, the FPGA1 first reads a service command from the command queue in the DDR RAM and analyzes the command type, generates a command FIS frame of a corresponding type, and encapsulates the command frame using an encapsulation protocol, and then sends the command frame to the base storage array through an optical fiber;
for access of hard disk write-in type, after the FPGA1 sends out the command FIS, the data to be written is read out from the DDR RAM to generate a standard data FIS frame (FIS _46), and the standard data FIS frame is encapsulated and then sent to the target storage array through the optical fiber;
for access of hard disk reading, the FPGA1 needs to parse an SATA FIS frame from an encapsulation packet received by a target storage array, extract valid data from the FIS frame, and write the data into the DDR RAM through a PCIE interface;
the FPGA2 on the control card of the target storage array receives the encapsulated frame from the optical fiber, then the SATA FIS frame is extracted by decapsulation and sent to the hard disk in the target storage array through the SATA port; when the SATA FIS frame sent by the hard disk is received, the SATA FIS frame is packaged and transmitted to the FPGA1 on the server through the optical fiber;
after the hard disk access command is executed, the FPGA1 generates an access normal/abnormal state response, and writes the access state response into an access state response storage space in the DDR RAM;
through the steps, the remote access operation of the server to the hard disk can be realized.
The process of slicing and non-slicing encapsulated data using a generic encapsulation protocol is described in detail below by two illustrative embodiments:
example 1: process for unsliced encapsulation of SATA FIS frames using a generic encapsulation protocol
By using the general encapsulation protocol provided in the embodiment of the application, the standard SATA FIS frame can be encapsulated without slicing, and the encapsulation mode can be applied to a transmission protocol without limitation on the size of a link MTU, such as an AURORA transmission protocol provided by XILINX.
When non-slicing encapsulation is performed, it is sufficient to add two DWORD (32bit) encapsulation headers to each SATA FIS frame, where head1 is 32h10010000, that is:
Ver＝1；
Type＝0；
Flag_L＝1；
Status＝0；
Frg＝0；
example 2: process for encapsulating SATA FIS frames using generic encapsulation protocol slices
The general encapsulation protocol provided by the embodiment of the application can be used for slicing and encapsulating the standard SATA FIS frame, and the encapsulation mode can be applied to a transmission protocol with limited link MTU size, for example, the Ethernet transmission protocol limits the MTU to 1500 bytes, and the maximum length of the SATA FIS can reach 8200 bytes, so that the SATA FIS needs to be sliced and encapsulated when being transmitted in the Ethernet.
When slice packaging is performed, the information in the head1 is used to complete the slice packaging, and the following method for slice packaging is described by taking the example of transmitting a SATA FIS frame with a maximum length of 2050DWORD in ethernet as an example:
(1) since the slice sequence number field frg in the head1 is 8 bits, a complete FIS frame can be accommodated in 256 slices at most for transmission, the size of the slice can be defined by the user according to the transmission protocol, and a slice size is defined as 256DWORD, i.e. 1024 bytes, so that high transmission efficiency can be obtained for the ethernet MTU of 1500 bytes. The 2050DWORD requires 9 slices to accommodate, with the first 8 slices all being 256 DWORDs in length and the 9 th slice being 2DWORD in length;
(2) when the first 8 slices are packaged, the contents of head1 are as follows:
Ver＝1；
Type＝0；
flag _ L ═ 0, meaning not the last slice;
Status＝0；
frg is x, and x is increased by 0-7;
(3) when packaging the 9 th slice, the contents of head1 are as follows:
Ver＝1；
Type＝0；
flag _ L ═ 1, indicating that it is the last slice;
Status＝0；
Frg＝8；
core(s)
The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application.
Fig. 5 shows a block diagram of a data access apparatus according to an embodiment of the present application, which may be implemented as part or all of an electronic device by software, hardware, or a combination of the two. As shown in fig. 5, the data access device includes:
a first obtaining module 501 configured to obtain access data on a hard disk; the hard disk is positioned on a target storage array in the cascade storage array system;
an encapsulation module 502 configured to encapsulate the access data by using a general encapsulation protocol, and send the encapsulated access data to a remote target storage array through an optical fiber; the general encapsulation protocol at least comprises a service type field, a slice information field, a destination address field and a data payload field.
In an optional implementation manner of this embodiment, the access data includes a SATA FIS frame and/or user-defined data.
In an optional implementation manner of this embodiment, in the process of writing data to the target hard disk, the first obtaining module 501 includes:
the first obtaining submodule is configured to obtain a write command from a server memory; and/or the presence of a gas in the gas,
and the second obtaining submodule is configured to obtain the data to be written from the server memory.
In an alternative implementation of this embodiment, the slice information field includes whether the last slice field and a slice number field.
In an optional implementation manner of this embodiment, when a write command for writing data to the target hard disk is acquired from the memory of the server, in the encapsulated access data, the service type field is an SATA FIS frame identifier, the slice information field is a non-slice identifier, the address field is a physical address of the target hard disk, and the data payload field is the write command.
In an optional implementation manner of this embodiment, when data to be written is acquired from a memory of a server and slice encapsulation is performed, in the encapsulated access data, the service type field is an SATA FIS frame identifier, the slice information field includes a slice number and an identifier of whether a last slice is included, the address field is a physical address of a target hard disk, and the data payload field is the data to be written; and/or the physical address of the target hard disk may include the physical address of the target storage array and the physical address of the target hard disk in the target storage array.
In an optional implementation manner of this embodiment, when reading data to the target storage array, after the packaging module 502, the apparatus further includes:
a receiving module configured to receive packed data from the target storage array;
and the determining module is configured to analyze the packaging data and determine the data to be read according to the slice information field in the packaging data.
In an optional implementation manner of this embodiment, the determining module includes:
a first determining submodule configured to determine data in the data payload field as data to be read when the slice information field includes an identifier of no slice; and/or the presence of a gas in the gas,
and the second determining submodule is configured to determine the data in the data payload field as a corresponding part in the data to be read according to the slice number in the slice information field and whether the identifier of the last slice is included in the slice information field when the identifier of the slice is included in the slice information field.
The data access device provided in this embodiment corresponds to the data access method provided in the embodiment and the related embodiment shown in fig. 1, and specific details can be referred to the above description of the data access method, which is not described herein again.
Fig. 6 is a schematic structural diagram of an electronic device suitable for implementing the data access method according to the embodiment of the present application.
As shown in fig. 6, the electronic apparatus 600 includes a Central Processing Unit (CPU)601 that can execute various processes in the embodiment shown in fig. 1 described above according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The CPU601, ROM602, and RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
In particular, according to embodiments of the present application, the method described above with reference to fig. 1 may be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program tangibly embodied on a medium readable thereby, the computer program comprising program code for performing the method illustrated in FIG. 1. In such embodiments, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams may represent a module, a program segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present application may be implemented by software or hardware. The units or modules described may also be provided in a processor, and the names of the units or modules do not in some cases constitute a limitation of the units or modules themselves.
As another aspect, the present application also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus in the above-described embodiment; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the methods described herein.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by a person skilled in the art that the scope of the invention as referred to in the present application is not limited to the embodiments with a specific combination of the above-mentioned features, but also covers other embodiments with any combination of the above-mentioned features or their equivalents without departing from the inventive concept. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (18)
1. A method of data access, comprising:
acquiring access data on a target hard disk; wherein the target hard disk is located on a target storage array in a cascaded storage array system, and the access data comprises SATA FIS frames;
packaging the access data by using a general packaging protocol, and sending the access data to the target storage array at a far end through an optical fiber; the general encapsulation protocol at least comprises a service type field, a slice information field, a destination address field and a data payload field.
2. The method of claim 1, wherein the access data comprises SATA FIS frames and/or user-defined data.
3. The method of claim 2, wherein obtaining access data on the target hard disk during writing data to the target hard disk comprises:
acquiring a data writing command from a server memory; and/or the presence of a gas in the gas,
and acquiring data to be written from the server memory.
4. The method according to any of claims 1-3, wherein the slice information field comprises whether the last slice field and a slice number field.
5. The method according to any one of claims 1 to 3, wherein when the write command for writing data to the target hard disk is obtained from the server memory, in the encapsulated access data, the service type field is a SATA FIS frame identifier, the slice information field is an un-slice identifier, the address field is a physical address of the target hard disk, and the data payload field is a write command.
6. The method according to any one of claims 1 to 3, wherein when the data to be written is obtained from the server memory and the slice is encapsulated, in the encapsulated access data, the service type field is SATA FIS frame identification, the slice information field includes a slice number and an identification of whether the last slice is present, the address field is a physical address of a target hard disk, and the data payload field is the data to be written; and/or the presence of a gas in the gas,
when data to be written is acquired from a server memory and non-slicing packaging is carried out, in the packaged access data, the service type field is a written data identifier, the slicing information field comprises a non-slicing identifier, the address field is a physical address of a target hard disk, and the data payload field is the data to be written.
7. The method of any of claims 1-3, wherein when reading data from the target storage array, after encapsulating the access data using a generic encapsulation protocol and sending the encapsulated access data to a target hard disk, further comprising:
receiving packed data from the target storage array;
and analyzing the packaging data, and determining the data to be read according to the slice information field in the packaging data.
8. The method of claim 7, wherein determining data to be read according to a slice information field in the encapsulated data comprises:
when the slice information field comprises an identifier of no slice, determining the data in the data payload field as the data to be read; and/or the presence of a gas in the gas,
and when the slice information field comprises the slice identifier, determining the data in the data payload field as the corresponding part in the data to be read according to the slice number in the slice information field and whether the identifier of the last slice is included.
9. A data access device, comprising:
the first acquisition module is configured to acquire access data on a target storage array hard disk; wherein the target hard disk is located on a target storage array in a cascaded storage array system;
the packaging module is configured to package the access data by using a general packaging protocol and send the access data to the target storage array at a remote end through an optical fiber; the generic encapsulation protocol at least comprises a service type field, a slice information field, an address field and a destination address field of the target storage array and a data payload field.
10. The apparatus of claim 9, wherein the access data comprises SATA FIS frame access commands and/or write data user-defined data.
11. The apparatus of claim 10, wherein during the process of writing data to the target hard disk, the first obtaining module comprises:
the first obtaining submodule is configured to obtain a write data command from a server memory; and/or the presence of a gas in the gas,
and the second obtaining submodule is configured to obtain the data to be written from the server memory.
12. The apparatus of any of claims 9-11, wherein the slice information field comprises whether a last slice field and a slice number field.
13. The apparatus according to any one of claims 9 to 11, wherein when a write command for writing data to the target hard disk is obtained from the server memory, in the encapsulated access data, the service type field is a SATA FIS frame identifier, the slice information field is an un-slice identifier, the address field is a physical address of the target hard disk, and the data payload field is a write command.
14. The apparatus according to any one of claims 9 to 11, wherein when the data to be written is obtained from the server memory and slice-packaged, in the packaged access data, the service type field is SATA FIS frame identification, the slice information field includes a slice number and an identification of whether a last slice is included, the address field is a physical address of a target hard disk, and the data payload field is the data to be written; and/or the presence of a gas in the gas,
when data to be written is acquired from a server memory and non-slicing packaging is carried out, in the packaged access data, the service type field is a written data identifier, the slicing information field comprises a non-slicing identifier, the address field is a physical address of a target hard disk, and the data payload field is the data to be written.
15. The apparatus according to any one of claims 9-11, wherein when reading data from the target storage array, after encapsulating the access data using a generic encapsulation protocol and sending the encapsulated access data to the target hard disk, the apparatus further comprises:
a receiving module configured to receive packed data from the target storage array;
and the determining module is configured to analyze the packaging data and determine the data to be read according to the slice information field in the packaging data.
16. The apparatus of claim 15, wherein the determining module comprises:
a first determining submodule configured to determine data in the data payload field as data to be read when the slice information field includes an identifier of no slice; and/or the presence of a gas in the gas,
and the second determining submodule is configured to determine the data in the data payload field as a corresponding part in the data to be read according to the slice number in the slice information field and whether the identifier of the last slice is included in the slice information field when the identifier of the slice is included in the slice information field.
17. An electronic device comprising a memory and a processor; wherein the content of the first and second substances,
the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps of any of claims 1-8.
18. A computer-readable storage medium having stored thereon computer instructions, characterized in that the computer instructions, when executed by a processor, carry out the method steps of any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811476095.9A CN109358820B (en) | 2018-12-04 | 2018-12-04 | Data access method and device, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811476095.9A CN109358820B (en) | 2018-12-04 | 2018-12-04 | Data access method and device, electronic equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109358820A CN109358820A (en) | 2019-02-19 |
| CN109358820B true CN109358820B (en) | 2022-04-19 |
Family
ID=65331046
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811476095.9A Active CN109358820B (en) | 2018-12-04 | 2018-12-04 | Data access method and device, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109358820B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111796876A (en) * | 2020-07-08 | 2020-10-20 | 深圳忆联信息系统有限公司 | Method, device, equipment and medium for defining requirements on storage equipment based on user |
| CN114721984B (en) * | 2022-03-30 | 2024-03-26 | 湖南长城银河科技有限公司 | SATA interface data transmission method and system for low-delay application |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1606290A (en) * | 2003-10-09 | 2005-04-13 | 英特尔公司 | Method, system, and program for managing memory for data transmission through a network |
| CN101431349A (en) * | 2008-11-29 | 2009-05-13 | 成都市华为赛门铁克科技有限公司 | Wireless memory device, system and method |
| CN101478462A (en) * | 2008-12-17 | 2009-07-08 | 成都市华为赛门铁克科技有限公司 | Apparatus and method for storage data reading and writing, solid hard disk |
| CN101753528A (en) * | 2008-12-22 | 2010-06-23 | 成都市华为赛门铁克科技有限公司 | A kind of methods, devices and systems of communication of solid-state disk |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7853741B2 (en) * | 2005-04-11 | 2010-12-14 | Emulex Design & Manufacturing Corporation | Tunneling SATA targets through fibre channel |
-
2018
- 2018-12-04 CN CN201811476095.9A patent/CN109358820B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1606290A (en) * | 2003-10-09 | 2005-04-13 | 英特尔公司 | Method, system, and program for managing memory for data transmission through a network |
| CN101431349A (en) * | 2008-11-29 | 2009-05-13 | 成都市华为赛门铁克科技有限公司 | Wireless memory device, system and method |
| CN101478462A (en) * | 2008-12-17 | 2009-07-08 | 成都市华为赛门铁克科技有限公司 | Apparatus and method for storage data reading and writing, solid hard disk |
| CN101753528A (en) * | 2008-12-22 | 2010-06-23 | 成都市华为赛门铁克科技有限公司 | A kind of methods, devices and systems of communication of solid-state disk |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109358820A (en) | 2019-02-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11726938B2 (en) | Communications for field programmable gate array device | |
| CN109358820B (en) | Data access method and device, electronic equipment and computer readable storage medium | |
| CN111294235A (en) | Data processing method, device, gateway and readable storage medium | |
| CN113596017B (en) | Protocol analysis method and device, soft gateway and storage medium | |
| CN109104368B (en) | Connection request method, device, server and computer readable storage medium | |
| CN111885174A (en) | Method and system for processing nodes of different network segments | |
| CN112291259B (en) | Protocol conversion method, gateway, equipment and readable storage medium | |
| CN113300984B (en) | Communication method, device, system and storage medium | |
| CN106059927B (en) | A kind of automatic reconstructed network routing device of the limited cascade of hub-and-spoke configuration and its network | |
| CN109582242B (en) | Address determination method and device for cascade memory array system and electronic equipment | |
| CN105491082A (en) | Remote resource access method and switch equipment | |
| WO2023104054A1 (en) | Network processing module, data processing method, network node and storage medium | |
| CN108365976B (en) | Network service optimization method and device | |
| CN114666289B (en) | Data transmission method and system based on electromagnetic shielding body | |
| CN115499432A (en) | Family terminal computing resource management system and computing resource scheduling method | |
| CN114615355A (en) | Message processing method and message analysis module | |
| JP7294577B2 (en) | Methods, devices and computer programs for managing remote devices by managed devices | |
| CN113037812A (en) | Data packet scheduling method and device, electronic equipment, medium and intelligent network card | |
| CN110784368A (en) | Memcached-based data acquisition method and system | |
| CN111092817A (en) | Data transmission method and device | |
| CN116489536B (en) | Efficient point location acquisition method and device based on modbusRTU protocol communication | |
| CN114978950B (en) | Network algorithm calling method and system based on FPGA and CPU cooperation | |
| CN113542412B (en) | Data transmission method, device, electronic equipment and storage medium | |
| WO2023097903A1 (en) | Message transmission method and apparatus, electronic device, and storage medium | |
| CN114500302B (en) | ICE service arrangement method, ICE service arrangement device, terminal equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |