CN117971135A - Storage device access method and device, storage medium and electronic device - Google Patents

Storage device access method and device, storage medium and electronic device Download PDF

Info

Publication number
CN117971135A
CN117971135A CN202410376341.2A CN202410376341A CN117971135A CN 117971135 A CN117971135 A CN 117971135A CN 202410376341 A CN202410376341 A CN 202410376341A CN 117971135 A CN117971135 A CN 117971135A
Authority
CN
China
Prior art keywords
storage
virtual
storage device
data access
channel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202410376341.2A
Other languages
Chinese (zh)
Inventor
刘铁军
李仁刚
公维锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Metabrain Intelligent Technology Co Ltd
Original Assignee
Suzhou Metabrain Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Metabrain Intelligent Technology Co Ltd filed Critical Suzhou Metabrain Intelligent Technology Co Ltd
Priority to CN202410376341.2A priority Critical patent/CN117971135A/en
Publication of CN117971135A publication Critical patent/CN117971135A/en
Pending legal-status Critical Current

Links

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the application provides a storage device access method, a storage device access device, a storage medium and an electronic device, wherein the method is applied to a storage expansion device, and the storage expansion device is connected between a first storage device and a server and comprises the following steps: receiving a data access request sent by a reference virtual machine; converting the first data access to a second data access by referring to the storage channel in response to the data access request; and controlling the reference storage space of the first storage device to execute the second data access. According to the application, the problem of low storage device expansion efficiency when the storage device connected with the server is expanded in the related technology is solved, and the effect of improving the storage device expansion efficiency when the storage device connected with the server is expanded is achieved.

Description

Storage device access method and device, storage medium and electronic device
Technical Field
The embodiment of the application relates to the field of computers, in particular to a storage device access method, a storage device access device, a storage medium and electronic equipment.
Background
In a cloud service scene of a data center, applications with strong instantaneity and high IO access density in some virtual machines or containers have high performance requirements on bandwidth delay and the like of storage equipment, the performance of a remotely deployed centralized storage system cannot meet the requirements, and the support is required to be provided by the storage equipment of a server. Because the number of PCIe ports that the server CPU can provide is limited, the number of storage devices supported is limited, so how to use limited local storage device resources to support local access requirements of a plurality of virtual machines is one of the problems faced by the storage performance improvement in the current cloud scenario. In order to solve the above problem, a PCIE SWITCH expansion scheme is adopted in the related art, in which a dedicated Switch chip is adopted, and the PCIe port of one CPU is expanded to a plurality of downstream ports to increase the number of storage devices carried by the server, and each downstream port is connected to one storage device. The method has the advantages that the local storage capacity of the server is increased on hardware, no additional software cooperation is needed, time delay is low, and universality is strong. However, the expansion flexibility of the scheme is insufficient, the model PCIE SWITCH needs to be selected at the beginning of the design of the server main board, and the expansion capacity is fixed. If more channels need to be expanded later, only PCIE SWITCH chips with more ports can be replaced, and main board hardware needs to be redesigned; if the number of the requests is reduced, the additional PCIe channels can only be idle, so that waste is caused. In addition, the expansion is realized by connecting a storage device to the downstream port of each PCIE SWITCH, so that the overall cost is high.
Disclosure of Invention
The embodiment of the application provides a storage device access method, a storage device access device, a storage medium and electronic equipment, which are used for at least solving the problem that storage device expansion efficiency is low when storage devices connected with a server are expanded in the related technology.
According to an embodiment of the present application, there is provided a storage device access method applied to a storage expansion apparatus, the storage expansion apparatus being connected between a first storage device and a server on which one or more virtual machines are deployed, one or more storage channels that allow the storage expansion apparatus to be identified by the server as a second storage device being allocated to the virtual machine deployed on the server, the storage channels being allocated corresponding storage space on the first storage device, the method including: receiving a data access request sent by a reference virtual machine, wherein the one or more virtual machines comprise the reference virtual machine, and the data access request is used for requesting first data access to a reference storage device allocated to the reference virtual machine in the second storage device; responding to the data access request, and converting the first data access into second data access through a reference storage channel, wherein the reference storage channel is a storage channel which is identified as the reference storage device in a storage channel established in the storage expansion device, and the second data access is data access to a reference storage space corresponding to the reference storage channel in the first storage device; controlling the reference storage space of the first storage device to execute the second data access.
The converting the first data access to the second data access by referring to the storage channel comprises:
determining a reference storage channel identified as the reference storage device from storage channels established in the storage expansion means;
forwarding the first data access to the reference storage channel;
And converting the first data access into the second data access through the reference storage channel.
Optionally, a first virtual bridge device is further disposed in the storage expansion apparatus, where the first virtual bridge device is connected to a storage channel established in the storage expansion apparatus, and the first virtual bridge device is further used to connect to the server and is identified by the server as being connected to the second storage device, and a second storage device and a storage channel with a corresponding relationship are recorded on the first virtual bridge device,
The determining a reference storage channel identified as the reference storage device from the storage channels established in the storage expansion apparatus includes: searching the reference storage channel corresponding to the reference storage device from the second storage device and the storage channel with corresponding relation through the first virtual bridge device;
the forwarding the first data access to the reference storage channel includes: forwarding, by the first virtual bridge device, the first data access to the reference storage channel.
Optionally, the searching, by the first virtual bridge device, the reference storage channel corresponding to the reference storage device from the second storage device and the storage channel having a corresponding relationship includes:
Extracting, by the first virtual bridge device, a reference device identifier of the reference storage device from the data access request;
And searching the reference storage channel corresponding to the reference device identifier from the storage device identifiers and the storage channels which have a corresponding relationship through the first virtual bridge device, wherein the storage device identifier is a device identifier which is allocated to the corresponding storage channel and is allowed to be identified as a storage device.
Optionally, the method further comprises:
Creating an initial virtual bridge device in the storage expansion device;
Performing first attribute configuration of a transmission protocol on the initial virtual bridging device in a protocol configuration space of the initial virtual bridging device to obtain the first virtual bridging device, wherein the server and the storage expansion device are connected through a protocol bus conforming to the transmission protocol, and the first attribute is a protocol attribute of the bridging device which is allowed to be identified by the server as conforming to the transmission protocol;
and establishing connection between the first virtual bridge equipment and the storage channel established in the storage expansion device, and recording a second storage equipment and the storage channel with corresponding relations.
Optionally, the storage channel includes a second virtual bridge device and a virtual storage device, the first virtual bridge device is connected with the second virtual bridge device, the second virtual bridge device is connected with the virtual storage device, the virtual storage device is used for connecting the first storage device, wherein,
The converting the first data access to the second data access through the reference storage channel includes:
receiving, by the second virtual bridge device in the reference storage channel, the first data access;
Transmitting, by the second virtual bridge device in the reference storage channel, the first data access to the virtual storage device in the reference storage channel;
the first data access is converted to the second data access by the virtual storage device in the reference storage channel.
Optionally, the converting, by the virtual storage device in the reference storage channel, the first data access to the second data access includes:
Acquiring a first access address of the access indicated by the first data access through the virtual storage device in the reference storage channel, wherein the first access address is a virtual address of the reference storage device indicated by the reference virtual machine;
querying, by the virtual storage device in the reference storage channel, a first storage address corresponding to the first access address from a virtual address range and a storage address range having a corresponding relationship, where the virtual address range is an address range of the reference storage device identified by the server, and the storage address range is an address range of a storage space allocated on the first storage device by the virtual storage device in the reference storage channel;
And replacing the first access address in the first data access with the first storage address to obtain the second data access.
Optionally, the querying, by the virtual storage device in the reference storage channel, the first storage address corresponding to the first access address from a virtual address range and a storage address range having a corresponding relationship includes:
Querying, by the virtual storage device in the reference storage channel, the first storage address mapped by the first access address from an address mapping between an addressing space of a transport protocol of the reference storage device and a storage space of a storage protocol of the first storage device, wherein the reference storage device is identified by the server as a device conforming to the transport protocol, the virtual address range includes the addressing space of the transport protocol, the first storage device is a storage device of the storage protocol, and the storage address range includes the storage space of the storage protocol.
Optionally, the method further comprises:
Creating a plurality of groups of initial virtual bridging devices and initial virtual storage devices in the storage expansion device;
Configuring a second attribute of a transmission protocol of the initial virtual bridge device in a protocol configuration space of the initial virtual bridge device to obtain the second virtual bridge device, wherein the server and the storage expansion device are connected through a protocol bus conforming to the transmission protocol, and the second attribute is a protocol attribute of the bridge device which is allowed to be identified by the server as conforming to the transmission protocol;
And performing third attribute configuration of a transmission protocol on the initial virtual storage device in a transmission protocol configuration space of the initial virtual storage device, and performing fourth attribute configuration of a storage protocol on the initial virtual storage device in a storage protocol configuration space of the initial virtual storage device to obtain the virtual storage device, wherein the first storage device is a storage device conforming to the storage protocol, the third attribute is a protocol attribute of a terminal device which is allowed to be identified by the server as conforming to the transmission protocol, and the fourth attribute is a protocol attribute of a storage device which is allowed to be identified by the server as conforming to the storage protocol.
Optionally, a protocol parsing unit is deployed in the virtual storage device, and the performing, in a transport protocol configuration space of the initial virtual storage device, third attribute configuration of a transport protocol on the initial virtual storage device includes:
and performing configuration of data link layer attribute and transaction layer attribute of the transmission protocol on the initial virtual storage device in a transmission protocol configuration space of the initial virtual storage device, wherein the third attribute configuration comprises: the protocol analysis unit is used for carrying out protocol analysis of a data link layer on the received data access by using the data link layer attribute and/or carrying out protocol analysis of a transaction layer on the received data access by using the transaction layer attribute.
Optionally, the performing, in the storage protocol configuration space of the initial virtual storage device, a fourth attribute configuration of a storage protocol on the initial virtual storage device includes:
And performing configuration of storage capacity attribute and queue number attribute of the storage protocol on the initial virtual storage device in a storage protocol configuration space of the initial virtual storage device, wherein the fourth attribute configuration comprises: the storage capacity attribute and the queue number attribute are for use by the server to identify storage devices that conform to the storage protocol.
Optionally, a target bus and a bus interface are further disposed in the storage expansion device, where the bus interface is used to connect the plurality of storage channels connected to the target bus to the first storage device, and the bus interface has a mapping relationship between a bus address space of the target bus and a storage address space of the first storage device,
The controlling the reference storage space of the first storage device to perform the second data access includes:
Receiving the second data access via the target bus;
And forwarding the second data access to the reference storage space according to the mapping relation through the bus interface.
Optionally, the bus interface includes: a bus arbitration unit and a memory device driving unit having therein a mapping relationship of a bus address space of the target bus and a memory address space of the first memory device,
The forwarding the second data access to the reference storage space through the bus interface according to the mapping relationship includes:
determining whether to process bus access of the reference memory channel according to the priority of the reference memory channel through the bus arbitration unit;
And forwarding the second data access to the reference memory space by the memory device driving unit according to the mapping relation under the condition of determining to process the bus access of the reference memory channel.
According to another embodiment of the present application, there is provided a storage expansion apparatus including: the server port is connected with the processing chip, the processing chip is connected with the storage device port, one or more storage channels are established in the processing chip, the server port is used for being connected with a server, one or more virtual machines are deployed on the server, the one or more storage channels allow the server to recognize the one or more virtual machines as one or more second storage devices, and the one or more second storage devices are distributed to the virtual machines deployed on the server; the storage device port is used for connecting a first storage device, wherein the storage channel is allocated with a corresponding storage space on the first storage device; the processing chip is used for realizing the step of accessing the storage device.
Optionally, a first virtual bridge device is disposed in the processing chip, where the first virtual bridge device is connected to the one or more storage channels, and the first virtual bridge device is further configured to connect to the server port and is recognized by the server as being connected to the second storage device, where the first virtual bridge device records the second storage device and the storage channel that have a corresponding relationship,
The first virtual bridge device is configured to search, when a first data access is received, a reference storage channel corresponding to a reference storage device from the second storage devices and storage channels having a corresponding relationship, where the reference storage device is a storage device allocated to a reference virtual machine that initiates a data access request in the one or more second storage devices, and the reference storage channel is a storage channel identified as the reference storage device in the one or more storage channels; forwarding the first data access to the reference storage channel.
Optionally, the storage channel includes a second virtual bridge device and a virtual storage device, the second virtual bridge device being connected to the first virtual bridge device, the second virtual bridge device being connected to the virtual storage device, the virtual storage device being connected to the first storage device, wherein,
The second virtual bridge device is configured to forward the first data access to the virtual storage device connected to the second virtual bridge device when the first data access is received;
The virtual storage device is used for converting the first data access into a second data access for accessing the storage space corresponding to the reference storage channel in the first storage device.
Optionally, an external storage device is further disposed in the storage expansion device, and the external storage device is connected with the virtual storage device, where,
The external storage device is used for caching the data to be accessed by the virtual storage device in the first storage device.
Optionally, a target processor is further disposed in the processing chip, the target processor is connected to the one or more storage channels through an internal high-speed bus, the target processor is further connected to the storage device port, wherein,
And the target processor is used for responding to the data access request transmitted by the storage channel and performing storage control and data access on the first storage device.
According to another embodiment of the present application, there is provided an access apparatus for a storage device, applied to a storage expansion apparatus, the storage expansion apparatus being connected between a first storage device and a server on which one or more virtual machines are deployed, one or more storage channels that allow the storage expansion apparatus to be identified by the server as a second storage device being allocated to the virtual machine deployed on the server, the storage channels being allocated corresponding storage space on the first storage device, the apparatus including: the receiving module is used for receiving a data access request sent by a reference virtual machine, wherein the one or more virtual machines comprise the reference virtual machine, and the data access request is used for requesting to perform first data access on a reference storage device allocated to the reference virtual machine in the second storage device; the conversion module is used for responding to the data access request and converting the first data access into second data access through a reference storage channel, wherein the reference storage channel is a storage channel which is identified as the reference storage device in a storage channel established in the storage expansion device, and the second data access is data access to a reference storage space corresponding to the reference storage channel in the first storage device; and the control module is used for controlling the reference storage space of the first storage device to execute the second data access.
According to a further embodiment of the application, there is also provided a computer readable storage medium having stored therein a computer program, wherein the computer program is arranged to perform the steps of any of the method embodiments described above when run.
According to a further embodiment of the application there is also provided an electronic device comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.
According to a further embodiment of the application, there is also provided a computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the steps of any of the method embodiments described above.
According to the application, the storage expansion device is deployed between the server and the first storage equipment deployed by the server, one or more storage channels established by the storage expansion device allow the server to identify the storage channels as one or more second storage equipment and allocate the storage channels to the virtual machines deployed on the server for use, so that each virtual machine can be considered to be connected with one second storage equipment on the virtual machine side, further, after the storage expansion device receives a data access request sent by the reference virtual machine on the server, the storage expansion device converts the data access of the reference storage equipment into the data access of the storage space corresponding to the reference storage channel in the first storage equipment through the reference storage channel identified as the reference storage equipment in the one or more storage channels, thereby realizing that the service can be provided for a plurality of virtual machines on the server only by connecting one storage equipment, realizing the expansion of the storage equipment connected with the service on the premise that the service logic of the storage equipment is not changed and the storage equipment is not required to be newly increased, and further providing each virtual machine for the server with higher flexibility and better expansion performance, and more general storage. The problem that the expansion efficiency of the storage equipment is lower when the storage equipment connected with the server is expanded in the related technology can be solved, and the effect of improving the expansion efficiency of the storage equipment when the storage equipment connected with the server is expanded is achieved.
Drawings
FIG. 1 is a schematic diagram of a method of accessing a storage device according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a storage expansion device according to an embodiment of the present application;
FIG. 3 is a schematic diagram of an alternative storage device space allocation according to an embodiment of the present application;
FIG. 4 is a schematic diagram of an alternative bus interface according to an embodiment of the application;
FIG. 5 is an alternative storage device connection system architecture diagram according to an embodiment of the present application;
FIG. 6 is a flowchart of an alternative storage expansion device operation in accordance with an embodiment of the present application;
FIG. 7 is an alternative storage device access flow diagram according to an embodiment of the application;
FIG. 8 is a block diagram of an alternative memory expansion device according to an embodiment of the present application;
FIG. 9 is a block diagram of an alternative storage expansion device according to an embodiment of the present application;
fig. 10 is a block diagram of a structure of an access apparatus of a storage device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application will be described in detail below with reference to the accompanying drawings in conjunction with the embodiments.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order.
In this embodiment, there is provided a method for accessing a storage device, fig. 1 is a schematic diagram of a method for accessing a storage device according to an embodiment of the present application, and as shown in fig. 1, the method is applied to a storage expansion device, where the storage expansion device is connected between a first storage device and a server, and one or more virtual machines are deployed on the server, and one or more storage channels that allow the server to identify as a second storage device are established in the storage expansion device, where the second storage device is allocated to a virtual machine deployed on the server, and where a corresponding storage space is allocated on the first storage device, and the method includes the following steps:
Step S102, receiving a data access request sent by a reference virtual machine, wherein the one or more virtual machines comprise the reference virtual machine, and the data access request is used for requesting first data access to a reference storage device allocated to the reference virtual machine in the second storage device;
step S104, responding to the data access request, and converting the first data access into second data access through a reference storage channel, wherein the reference storage channel is a storage channel which is identified as the reference storage device in a storage channel established in the storage expansion device, and the second data access is data access to a reference storage space corresponding to the reference storage channel in the first storage device;
step S106, controlling the reference storage space of the first storage device to perform the second data access.
Through the steps, the storage expansion device is deployed between the server and the first storage equipment deployed by the server, one or more storage channels established by the storage expansion device allow the server to identify the storage channels as one or more second storage equipment and allocate the storage channels to virtual machines deployed on the server for use, so that each virtual machine can be considered to be connected with one second storage equipment on the virtual machine side, further, after the storage expansion device receives a data access request sent by the reference virtual machine on the server, the storage expansion device converts the data access of the reference storage equipment into the data access of the storage space corresponding to the reference storage channel in the first storage equipment through the reference storage channel identified as the reference storage equipment in the one or more storage channels, thereby realizing the expansion of the storage equipment connected with the service on the premise that only one storage equipment needs to be connected with the server, realizing the expansion of the storage equipment connected with the service on the premise that the use logic of the storage equipment is not changed and the storage equipment is not needed to be newly increased, and further realizing the expansion of the storage equipment connected with the service on the premise that each virtual machine is still independent storage equipment, and the flexibility and the expansion performance of the storage equipment are still higher. The problem that the expansion efficiency of the storage equipment is lower when the storage equipment connected with the server is expanded in the related technology can be solved, and the effect of improving the expansion efficiency of the storage equipment when the storage equipment connected with the server is expanded is achieved.
In the technical solution provided in step S102, the storage device is used to provide a data storage service for a virtual machine on a server, and in the embodiment of the present application, the storage device may include, but is not limited to, a hard disk drive, a Solid state disk, an optical disc, a flash memory drive, a magnetic tape, and so on, such as an NVMe (Non-Volatile Memory express, non-volatile memory host controller interface specification) storage device (for example, NVMe SSD (Solid STATE DRIVES, solid state disk)), which is not limited in this scheme.
Optionally, in the embodiment of the present application, the storage channel established in the storage expansion device may be pre-established, or may also be established according to a requirement of a virtual machine deployed on the server for the storage channel, which is not limited in this scheme.
Optionally, in the embodiment of the present application, one or more storage channels for accessing the corresponding storage space allocated on the first storage device are established in the storage expansion device, so that each storage channel is identified as a storage device on the server side.
Optionally, in the embodiment of the present application, one reference virtual machine may be allocated one or more storage channels in the storage expansion device, for example, N storage channels are allocated in the storage expansion device, a conventional allocation manner may be to allocate a corresponding storage channel to each reference virtual machine, or in order to implement access efficiency of the reference virtual machine to the storage device, M storage channels (where N is greater than or equal to M) may be selected for the reference virtual machine from N storage channels according to access requirements of the reference virtual machine to the storage device, and then the reference virtual machine may perform data access operation on reference storage devices corresponding to the M reference storage channels simultaneously by sending a data access request.
Alternatively, in the embodiment of the present application, the data access request may, but is not limited to, an operation request including performing a data read operation on the reference storage device, or may also be a data write request for writing data into the reference storage device, which is not limited in this aspect.
In the embodiment provided in step S104, the reference storage channel is used to implement one-to-one mapping between the reference storage device and the storage space corresponding to the reference storage channel in the first storage device, and since in this embodiment, one or more storage channels in the storage expansion apparatus are identified by the server as being connected to one or more storage devices, then the virtual machine on the server issues a data access request for performing the first data access to the reference storage device, and the storage expansion apparatus serves as a relay device connected between the first storage device and the server, to convert the first data access into a second data access operation for accessing the storage space corresponding to the reference storage channel on the first storage device.
Alternatively, in the embodiment of the present application, the conversion of the first data access into the second data access by the reference storage channel may be the conversion of the data access address, that is, the virtual machine is connected to the first storage device through the reference storage channel, and then the reference storage channel is used as a reference storage device on the virtual machine, the virtual machine accesses the storage device, so that the virtual machine needs to perform the data access according to the virtual address of the reference storage device, and the first data access needs to carry the first access address for performing the data access operation, and the reference storage channel side is connected to the first storage device, so that the reference storage channel needs to perform the data access operation on the storage area corresponding to the reference storage channel on the first storage device, and therefore, the reference storage channel needs to perform the address conversion operation on the first data access, and converts the access address into the second access address of the storage area corresponding to the reference storage channel on the first storage device.
Optionally, in the embodiment of the present application, the conversion of the first data access into the second data access by the reference storage channel may be an access of a data transmission protocol format, that is, the virtual machine is connected to the first storage device through the reference storage channel, and further, the reference storage channel is used as a reference storage device on the virtual machine, and the virtual machine accesses the storage device, and needs to perform data access according to a virtual address of the reference storage device, so that the first data access needs to be accessed according to the data transmission protocol format of the reference storage device, and because the data transmission protocol format of the reference storage device and the data transmission protocol format of the first storage device are different, the reference storage channel needs to perform conversion of the data transmission protocol format on the first data access, so as to convert the first data access into the second data access.
Optionally, in the embodiment of the present application, the reference storage space may be a continuous storage space corresponding to the reference storage channel in the first storage device, or may also be a plurality of discontinuous storage spaces, for example, the storage space of the first storage device may be divided into 5 storage spaces that are sequentially continuous, that is, a storage space a, a storage space B, a storage space C, and a storage space D, where the storage area of the storage space a is adjacent to the storage space starting position of the storage space B, the storage space ending position of the storage space B is adjacent to the storage space starting position of the storage space C, and the storage space C is adjacent to the storage space starting position of the storage space D, so that the storage space a or the storage space a and the storage space B may be simultaneously used as the storage space corresponding to one storage channel.
In the embodiment provided in step S106, a mapping relationship between a reference storage channel and a reference storage space allocated for the reference storage channel in the first storage device is maintained in the storage expansion device, so as to map the reference storage device to the reference storage space one to one, and then after the second data access operation needs to be executed, the reference storage space having a mapping relationship with the reference channel on the first storage device is found out through the mapping relationship.
The embodiment of the application can be applied to and can be used for expanding NVMe storage equipment, and the core idea is that an NVMe expanding device is added between a CPU and the NVMe storage equipment to realize the expansion of the number of channels of PCIe, one NVMe storage equipment is virtualized under each channel, and the mapping between the NVMe storage equipment and physical equipment is completed, so that a plurality of storage equipment are presented to the CPU under the condition that only one NVMe storage equipment is connected at the downstream, and the purpose of equipment expansion is achieved. The application realizes the NVMe storage equipment expansion function by using a hardware device, does not need excessive participation of software, has high performance and universality of PCIE SWITCH expansion scheme, and has flexibility and low cost of software virtualization scheme. The storage equipment expansion system comprises a server, a storage expansion device and a first storage equipment, wherein the storage expansion device is connected between the first storage equipment and the server, one or more virtual machines are deployed on the server, the topological form that a PCIe root node is downloaded and mounted with a plurality of virtual NVMe equipment (vDevice) can be presented on the CPU side of the server, and the number of the expanded virtual equipment can be dynamically adjusted according to the requirement of a user. In the application, all virtual devices share one NVMe storage device, so that the physical NVMe storage space addresses corresponding to a plurality of virtual devices cannot have coincident parts, and a special address mapping unit is responsible for space management.
As an alternative embodiment, the converting the first data access to the second data access by referring to the storage channel includes:
determining a reference storage channel identified as the reference storage device from storage channels established in the storage expansion means;
forwarding the first data access to the reference storage channel;
And converting the first data access into the second data access through the reference storage channel.
Optionally, in the embodiment of the present application, the storage expansion device may, but is not limited to, maintain a mapping relationship between a virtual machine and a storage channel, and is configured to map the virtual machine deployed on a server and the storage channel allocated for the virtual machine, so that after receiving a data access request sent by a reference virtual machine, the storage expansion device may find a reference storage channel allocated for the reference virtual machine according to the mapping relationship.
Through the above, since one or more storage channels are established in the storage expansion device, the storage channels are used for connecting the virtual machine and the storage space allocated for the storage channels on the first storage device, so that the server can identify the one or more storage channels as one or more second storage devices, and further after receiving the data access request sent by the reference virtual machine, the reference storage channel identified as the reference storage device by the reference virtual machine is determined from the one or more storage channels, so that the storage space on the first storage device, in which the data access operation needs to be executed, is also found, and further, the first data access is forwarded to the reference storage channel, so that the reference storage channel can convert the first data access for executing the data access operation on the reference storage device into the second data access for executing the data access operation on the first storage device and the reference storage space allocated for the reference storage device, thereby realizing the accuracy of converting the first data access into the second data access.
As an alternative embodiment, a first virtual bridge device is further disposed in the storage expansion apparatus, where the first virtual bridge device is connected to a storage channel established in the storage expansion apparatus, and the first virtual bridge device is further used to connect to the server and is recognized by the server as being connected to the one or more second storage devices, and the first virtual bridge device records a second storage device and a storage channel that have a corresponding relationship,
The determining a reference storage channel identified as the reference storage device from the storage channels established in the storage expansion apparatus includes: searching the reference storage channel corresponding to the reference storage device from the second storage device and the storage channel with corresponding relation through the first virtual bridge device;
the forwarding the first data access to the reference storage channel includes: forwarding, by the first virtual bridge device, the first data access to the reference storage channel.
Optionally, in this embodiment of the present application, the first virtual bridge device is configured to implement connection between the server and one or more storage channels, perform protocol analysis on data sent by the virtual machine or data that needs to be returned to the virtual machine, and in this embodiment, the first virtual bridge device may further perform route forwarding on the data packet according to a first pair of data packets, for example, address information of a reference storage channel allocated to the reference virtual machine is carried in a data access request sent by the reference virtual machine, so that the first virtual bridge device searches, according to the address information, a reference storage channel that matches the address information from the one or more storage channels, sends the data access request to the reference storage channel, or performs a data access operation on a storage space allocated to the reference storage channel in the first virtual bridge device, where the data access request needs to be fed back to the reference virtual machine (for example, the data access request is a request to read data), and at this time, the reference storage channel sends the data access result to the first virtual bridge device, and the first virtual bridge device forwards the data access result sent by the reference storage channel to the reference virtual machine corresponding to the reference storage channel.
Optionally, in the embodiment of the present application, a mapping relationship between a virtual machine and a storage channel allocated to the virtual machine may be maintained in the first virtual bridge device, so that data forwarding between the virtual machine and the storage channel is achieved through the first virtual bridge device, that is, the virtual machine sends a data access request to the first virtual bridge device, the first virtual bridge device searches a reference storage channel allocated to the virtual machine in one or more storage channels according to the stored mapping relationship, so that the data access request is forwarded to the reference storage channel, or the storage channel sends a data packet read from the first storage device to the first virtual machine bridge device, and then a reference virtual machine corresponding to the storage channel is found in the virtual machine deployed on the first virtual machine bridge device server, so that the data packet sent by the storage channel is forwarded to the reference virtual machine.
Through the above, the first virtual machine bridging device for linking the virtual machine and the one or more storage channels established in the storage expansion device is deployed in the storage expansion device, and because the first virtual machine bridging device is used for connecting the server and is identified by the server as being connected with the one or more second storage devices, the second storage devices and the storage channels with corresponding relations are recorded on the first virtual machine bridging device, so that the reference storage channel corresponding to the reference storage device is searched through the corresponding relations between the second storage devices and the storage channels on the first virtual bridging device, accurate searching of the storage channel is realized, and forwarding errors of first data access are avoided.
As an optional embodiment, the searching, by the first virtual bridge device, the reference storage channel corresponding to the reference storage device from the second storage device and the storage channel having a corresponding relationship includes:
Extracting, by the first virtual bridge device, a reference device identifier of the reference storage device from the data access request;
And searching the reference storage channel corresponding to the reference device identifier from the storage device identifiers and the storage channels which have a corresponding relationship through the first virtual bridge device, wherein the storage device identifier is a device identifier which is allocated to the corresponding storage channel and is allowed to be identified as a storage device.
Optionally, in the embodiment of the present application, the first virtual bridge device is responsible for performing transfer connection between the server and the storage channel, and the server side server considers that the server side server is connected with a plurality of second storage devices, so as to generate a data access request for calling the second storage devices, where the data access request carries a reference device identifier of the second storage device to be requested to access, and the first virtual bridge device maintains a correspondence between the storage device identifier and the storage channel, so as to determine a reference storage channel corresponding to the reference device identifier according to the correspondence, and then send the corresponding data access request to the reference storage channel, so as to call the first storage device.
Through the steps, the first virtual bridging device is set, the virtual bridging device is used as a server and a transfer connection device of a plurality of storage channels, and the storage channels are identified as a plurality of second storage devices by the server, so that the mapping is carried out between the device identification of the device requiring calling and the storage channels by maintaining the corresponding relation between the device identification and the storage channels on the first virtual bridging device, and the accurate searching of the storage channels is realized.
As an alternative embodiment, the method further comprises:
Creating an initial virtual bridge device in the storage expansion device;
Performing first attribute configuration of a transmission protocol on the initial virtual bridging device in a protocol configuration space of the initial virtual bridging device to obtain the first virtual bridging device, wherein the server and the storage expansion device are connected through a protocol bus conforming to the transmission protocol, and the first attribute is a protocol attribute of the bridging device which is allowed to be identified by the server as conforming to the transmission protocol;
and establishing connection between the first virtual bridge device and the one or more storage channels, and recording a second storage device and a storage channel with corresponding relation.
Optionally, in the embodiment of the present application, the first attribute configuration of the transport protocol performed on the initial virtual bridge device may include, but is not limited to, a PCIe configuration space, a Memory base, a Memory limit, and other register definitions, so that the first attribute configuration of the transport protocol can be identified as a normal bridge device in the server CPU enumeration process through the configuration of the transport protocol.
Optionally, in the embodiment of the present application, the virtual bridge device may be, but is not limited to, a virtual PCIe bridge device, configured to implement bridging between a PCIe bus of the server and the first storage device, or configured to implement connection between the server and a plurality of storage channels identified by the server as the second storage device, so as to perform protocol parsing on PCIe packets from upstream and downstream links, and perform routing and forwarding of the upstream and downstream packets according to address information in the server, which is not limited in this scheme.
Through the above, the first attribute configuration of the transmission protocol is performed on the initial virtual bridge device in the protocol configuration space of the initial virtual bridge device, so that the configured first virtual bridge device can be allowed to be identified by the server as the bridge device conforming to the transmission protocol, thereby realizing the normal identification of the first virtual bridge device on the cpu, and further providing protocol guarantee for the connection between the server and the plurality of storage channels.
As an alternative embodiment, the storage channel comprises a second virtual bridge device and a virtual storage device, a first virtual bridge device being connected to the second virtual bridge device, the second virtual bridge device being connected to the virtual storage device, the virtual storage device being configured to connect to the first storage device, wherein,
The converting the first data access to the second data access through the reference storage channel includes:
receiving, by the second virtual bridge device in the reference storage channel, the first data access;
Transmitting, by the second virtual bridge device in the reference storage channel, the first data access to the virtual storage device in the reference storage channel;
the first data access is converted to the second data access by the virtual storage device in the reference storage channel.
Optionally, in the embodiment of the present application, the virtual storage device has the logical functions of the device link layer and the transaction layer, and can perform the functions of unpacking, packaging, checking, etc. of the DLP and TLP data packets according to the protocol, and complete the bus transactions such as memory read-write, configuration read-write, etc. Before the CPU enumerates the device tree, the channel expansion logic may prepare necessary information for configuration space, address range, number of devices, etc. of the virtual devices as needed to accommodate upstream system calls.
Optionally, in the embodiment of the present application, the core functional unit in the virtual device may include, but is not limited to, a protocol parsing unit, a data buffering unit, and an address mapping unit, where the three parts are interconnected in the virtual device through a high-bandwidth bus; protocol analysis unit: the unit completes protocol analysis, encapsulation, verification and data processing of a PCIe data link layer and a transaction layer. In the data link layer, adopting an ACK/NAK response mechanism to carry out error retransmission and flow control between a transmitting end and a receiving end, carrying out DLLP message processing according to a data link layer protocol to realize reliable data transmission in a channel, and defining the sending sequence of different types of messages to ensure that important information is responded preferentially. At the transaction layer, the unit completes route addressing by using the bus number, the device number and the function number ID in the TLP data packet, realizes reading and writing of configuration space content, and realizes reading and writing of a memory space and an IO space according to addressing fields in the TLP packet; the data caching unit is used for temporarily transmitting data transmitted between the server and the first storage device, and when the virtual storage device accesses the PCIe upstream and downstream ports at high frequency, the problem of untimely data packet processing can occur due to unmatched response rates of the upstream and downstream ports. The size of the data cache unit of each virtual storage device can be adjusted and is used for adapting to different application scenes; an address mapping unit: the final data storage medium in the present application is the first storage device to which the downstream port is connected, not the virtual storage device that the server CPU "sees". Each virtual storage device has a section of corresponding space in the first storage device, and data read-write operation performed on the virtual storage device is finally performed on the space, so that bidirectional data forwarding is required between the virtual storage device and the first storage device, and data read-write access of the CPU to the virtual storage device is mapped to the first storage device.
Optionally, in the embodiment of the present application, the virtual storage device is used as a data transfer device between the server and the first storage device, that is, the data access request sent by the server is temporarily stored, and according to the data access request, data access operation is performed on a storage area corresponding to the virtual storage device on the first storage device, and the accessed data is temporarily stored and transferred to the server, so that in order to achieve this purpose, cache control may be allocated to the virtual device in the storage expansion device, that is, the storage space of the storage expansion device may be allocated to the plurality of virtual storage devices evenly, or the storage space may also be allocated according to the service to be handled by each virtual storage device, which is not limited by the present scheme.
Through the above, the second virtual bridging device and the virtual storage device are arranged in the storage channel, so that the first data access request is received through the second virtual bridging device, and the virtual storage device is used as the transfer device for the first data access and the second data access, so that the efficient and accurate access to the storage space of the connected first storage device through the storage channel is realized.
In an embodiment of the present application, the virtual bridge device may be, but is not limited to, a virtual PCIe bridge device, configured to implement bridging between two PCIe buses, and fig. 2 is a schematic diagram of a storage expansion apparatus according to an embodiment of the present application, as shown in fig. 2: the storage expansion device comprises a channel expansion logic module and an internal bus interface module, wherein the channel expansion logic module is used for being connected with a server port (a server or a virtual machine deployed on the server), the channel expansion logic module is also connected with an internal bus interface, the internal bus interface module is connected with a storage device port (a first storage device), and the internal bus interface is used for realizing data access of a storage channel created in the storage expansion device to the first storage device. vBridge are virtual PCIe bridge devices (including a first virtual bridge device and a second virtual bridge device) in the present application for implementing a bridge between two PCIe buses. The application includes register definitions such as PCIe configuration space, memory base, memory limit and the like in vBridge structures, and can be identified as normal PCIe Bridge in the process of CPU enumeration. In the application vBridge is responsible for establishing the connection relationship between the CPU and the virtual PCIE DEVICE (vDevice (virtual storage device)), carrying out protocol analysis on PCIe data packets from upstream and downstream links, and carrying out upstream and downstream data packet routing forwarding according to the address information.
In this embodiment, vDevice devices (virtual storage devices) are first PCIe Endpoint devices and further NVMe storage devices based thereon. The vDevice device in the invention has logic functions of a link layer and a transaction layer of the PCIe Endpoint device, can perform the functions of unpacking, packaging, checking and the like of DLP and TLP data packets according to a protocol, and completes bus transactions such as memory read-write, configuration read-write and the like. Before the CPU enumerates the PCIe device tree, the channel expansion logic may prepare vDevice's configuration space, address ranges, number of devices, etc. as necessary to accommodate upstream system calls. The core functional unit in vDevice includes three parts, namely a protocol parsing unit, a data buffering unit and an address mapping unit, which are interconnected by a high bandwidth bus in vDevice. The details are as follows:
Protocol analysis unit: the unit completes protocol analysis, encapsulation, verification and data processing of a PCIe data link layer and a transaction layer. In the data link layer, adopting an ACK/NAK response mechanism to carry out error retransmission and flow control between a transmitting end and a receiving end, carrying out DLLP message processing according to a data link layer protocol to realize reliable data transmission in a channel, and defining the sending sequence of different types of messages to ensure that important information is responded preferentially. At the transaction layer, the unit completes route addressing by using the bus number, the device number and the function number ID in the TLP data packet, realizes reading and writing of configuration space content, and realizes reading and writing of a memory space and an IO space according to addressing fields in the TLP packet.
On the basis of PCIe, the unit also supports vDevice the logic function of NVMe equipment. By implementing the standard NVMe control register set, vDevice is enabled with the ability to fetch CQ and SQ queue information from Host, respond to Host's DoorBell command, and send MSI interrupts. For writing and reading data in the NVMe storage space, the unit forwards data and commands between the upstream server CPU and the downstream NVMe equipment through the vDevice data caching unit and the address mapping unit, so that actual reading and writing operations are completed.
A data caching unit: when vDevice high-frequency accesses to PCIe upstream and downstream ports, the problem of untimely processing of data packets occurs due to mismatch in upstream and downstream port response rates. The invention sets a data buffer unit in vDevice to solve the problem of inconsistent data processing speed. The size of the data buffer unit of each vDevice can be adjusted to adapt to different application scenarios. The medium SRAM or DRAM of the data buffer unit is located or located inside the NVMe expansion device, and may also be located outside the NVMe expansion device.
An address mapping unit: the final data storage medium in the present application is the NVMe storage device to which the downstream port is connected, not vDevice "seen" by the server CPU. Each vDevice has a corresponding space in the NVMe storage device, and data read/write performed on vDevice will ultimately be applied to the space, so bidirectional data forwarding between vDevice and the NVMe storage device is required, and the data read/write access of the CPU to vDevice is mapped to the NVMe storage device. Fig. 3 is an alternative storage device space allocation schematic diagram according to an embodiment of the present application, as shown in fig. 3, the storage space of the first storage device is allocated according to a target granularity (may be according to the number of virtual storage devices or the requirement of the virtual storage devices on the storage space), so as to obtain a plurality of allocated storage spaces, and further, the storage spaces may be allocated to corresponding virtual storage devices according to the data storage requirement of each virtual machine.
The above-described work is accomplished by the address mapping unit in vDevice in the present application. The core idea is to one-to-one map each vDevice PCIe BAR space with the portion of the BAR space of the NVMe storage device allocated to that vDevice. Whenever vDevice receives a PCIe packet, the packet is forwarded to the corresponding address of the NVMe storage device according to the mapping relationship. As is the case for reverse transmission, vDevice, each time a packet is received from the NVMe storage device, forwards the packet to the PCIe BAR space corresponding to the Host. The one-to-one mapping is realized through a special register set, the first and last addresses of PCIe BAR space on vDevice sides and NVMe storage equipment sides are respectively stored by using registers, and address matching and offset are realized by using hardware logic.
As an alternative embodiment, said converting said first data access to said second data access by said virtual storage device in said reference storage channel comprises:
Acquiring a first access address of the access indicated by the first data access through the virtual storage device in the reference storage channel, wherein the first access address is a virtual address of the reference storage device indicated by the reference virtual machine;
querying, by the virtual storage device in the reference storage channel, a first storage address corresponding to the first access address from a virtual address range and a storage address range having a corresponding relationship, where the virtual address range is an address range of the reference storage device identified by the server, and the storage address range is an address range of a storage space allocated on the first storage device by the virtual storage device in the reference storage channel;
And replacing the first access address in the first data access with the first storage address to obtain the second data access.
Optionally, in an embodiment of the present application, the virtual address range may be, but is not limited to, an address range of a storage space allocated for a corresponding storage channel on the network connection device for caching data.
Optionally, in the embodiment of the present application, in order to improve the utilization ratio of the storage space in the first storage device, the storage space of the first storage device may be divided according to the target granularity, so that the storage space is divided into multiple parts, and further, a corresponding storage address range is allocated to the virtual storage device according to the data storage amount and the memory occupation condition in the storage space, where the storage addresses in the storage address range may be continuous storage addresses or may also be discontinuous storage addresses, that is, in order to improve the utilization ratio of the storage space, data storage may be performed by using storage spaces corresponding to a plurality of discontinuous storage address ranges that are not occupied in the storage space, and this scheme is not limited.
Through the steps, the virtual address range is allocated for the reference virtual machine corresponding to the reference storage device in the storage space of the storage expansion device, so that the virtual storage device can temporarily store data transmitted between the server and the first storage device, the problem of inconsistent data delay is avoided, the corresponding storage address range is allocated for the reference virtual machine on the first storage device, the virtual address range and the storage address range with corresponding relation are further allocated on the virtual storage device, and the virtual storage device can replace the first address of the first data access with the first storage address when data access conversion operation is required according to the mapping relation, so that the accuracy of the data access is improved.
As an optional embodiment, the querying, by the virtual storage device in the reference storage channel, the first storage address corresponding to the first access address from the virtual address range and the storage address range having the corresponding relationship includes:
Querying, by the virtual storage device in the reference storage channel, the first storage address mapped by the first access address from an address mapping between an addressing space of a transport protocol of the reference storage device and a storage space of a storage protocol of the first storage device, wherein the reference storage device is identified by the server as a device conforming to the transport protocol, the virtual address range includes the addressing space of the transport protocol, the first storage device is a storage device of the storage protocol, and the storage address range includes the storage space of the storage protocol.
As an alternative embodiment, the method further comprises:
Creating a plurality of groups of initial virtual bridging devices and initial virtual storage devices in the storage expansion device;
Configuring a second attribute of a transmission protocol of the initial virtual bridge device in a protocol configuration space of the initial virtual bridge device to obtain the second virtual bridge device, wherein the server and the storage expansion device are connected through a protocol bus conforming to the transmission protocol, and the second attribute is a protocol attribute of the bridge device which is allowed to be identified by the server as conforming to the transmission protocol;
And performing third attribute configuration of a transmission protocol on the initial virtual storage device in a transmission protocol configuration space of the initial virtual storage device, and performing fourth attribute configuration of a storage protocol on the initial virtual storage device in a storage protocol configuration space of the initial virtual storage device to obtain the virtual storage device, wherein the first storage device is a storage device conforming to the storage protocol, the third attribute is a protocol attribute of a terminal device which is allowed to be identified by the server as conforming to the transmission protocol, and the fourth attribute is a protocol attribute of a storage device which is allowed to be identified by the server as conforming to the storage protocol.
Optionally, in the embodiment of the present application, the data transmission between the server and the storage expansion device may be performed by, but not limited to, PCIe protocol, that is, the server is a PCIe server, the virtual bridge device is configured to forward a data packet between the PCIe server and the virtual storage device, and each second virtual bridge device value is connected to one virtual storage device, so that standard PCIe attribute setting needs to be performed in a PCIe configuration space, and information such as a bus number of a downstream device may be automatically determined when the CPU enumerates a PCIe device tree, where the task is implemented by a system management unit of the internal bus interface.
Optionally, in the embodiment of the present application, the third attribute configuration may include, but is not limited to, PCIe attribute configurations including DID/VID and BAR spaces.
Optionally, in the embodiment of the present application, the fourth attribute configuration may include, but is not limited to, NVMe attribute configurations including storage capacity and number of queues, and private attribute configurations including cache size and priority.
As an optional embodiment, the virtual storage device has a protocol parsing unit deployed therein, and the performing, in a transport protocol configuration space of the initial virtual storage device, a third attribute configuration of a transport protocol on the initial virtual storage device includes:
and performing configuration of data link layer attribute and transaction layer attribute of the transmission protocol on the initial virtual storage device in a transmission protocol configuration space of the initial virtual storage device, wherein the third attribute configuration comprises: the protocol analysis unit is used for carrying out protocol analysis of a data link layer on the received data access by using the data link layer attribute and/or carrying out protocol analysis of a transaction layer on the received data access by using the transaction layer attribute.
Alternatively, in embodiments of the present application, the data link layer attributes may include, but are not limited to, rate of data transmission, coding scheme, frame format, etc., while the transaction layer attributes include order of data transmission, acknowledgement mechanism, error handling, etc. For example, in the data link layer, an ACK/NAK acknowledgement mechanism is used to perform error retransmission and flow control between the transmitting end and the receiving end, DLLP message processing is performed according to the data link layer protocol to realize reliable data transmission in the channel, and the sending sequence of different types of messages is defined to ensure that important information is responded preferentially.
Optionally, in the embodiment of the present application, at the data link layer, routing addressing is completed by using the bus number, the device number and the function number ID in the TLP packet, so as to implement reading and writing of the configuration space content, and Address according to the Address field in the TLP packet, so as to implement reading and writing of the memory space and the IO space.
Through the above, the transmission protocol of the initial virtual storage device can be effectively managed and controlled through the configuration of the data link layer attribute and the transaction layer attribute, so that reliable transmission and processing of data are realized.
As an optional embodiment, the performing, in the storage protocol configuration space of the initial virtual storage device, a fourth attribute configuration of a storage protocol on the initial virtual storage device includes:
And performing configuration of storage capacity attribute and queue number attribute of the storage protocol on the initial virtual storage device in a storage protocol configuration space of the initial virtual storage device, wherein the fourth attribute configuration comprises: the storage capacity attribute and the queue number attribute are for use by the server to identify storage devices that conform to the storage protocol.
Optionally, in the embodiment of the present application, the configuration of the storage capacity attribute includes, but is not limited to, setting a total capacity size of the virtual storage device, and a capacity ratio allocated to different storage protocols, and the like.
Optionally, in the application embodiment, the configuration of the queue number attribute includes, but is not limited to, setting parameters such as the number of read-write queues of the storage protocol, the queue depth, and the like, so as to optimize storage performance and improve concurrent access capability.
Through the above, the storage capacity attribute and the queue number attribute of the storage protocol are configured, and the customized storage protocol configuration can be performed on the virtual storage device according to the actual requirements and the application scene, so that the storage performance and the flexibility are improved, and the storage requirements of different applications are met.
As an alternative embodiment, the memory expansion device further comprises a target bus and a bus interface, the bus interface is used for connecting a plurality of memory channels connected to the target bus to the first memory device, the bus interface has a mapping relationship between a bus address space of the target bus and a memory address space of the first memory device,
The controlling the reference storage space of the first storage device to perform the second data access includes:
Receiving the second data access via the target bus;
And forwarding the second data access to the reference storage space according to the mapping relation through the bus interface.
Optionally, in the embodiment of the present application, the bus interface is a high-bandwidth parallel bus interface that connects each virtual storage device with a storage device port, where the bandwidth is far greater than the transmission bandwidth of PCIe, and is capable of implementing data transmission between each virtual storage device and a downstream port with a very low delay.
As an alternative embodiment, the bus interface comprises: a bus arbitration unit and a memory device driving unit having therein a mapping relationship of a bus address space of the target bus and a memory address space of the first memory device,
The forwarding the second data access to the reference storage space through the bus interface according to the mapping relationship includes:
determining whether to process bus access of the reference memory channel according to the priority of the reference memory channel through the bus arbitration unit;
And forwarding the second data access to the reference memory space by the memory device driving unit according to the mapping relation under the condition of determining to process the bus access of the reference memory channel.
Alternatively, in the embodiment of the present application, with respect to the bus arbitration unit, there is a problem of bus collision when a plurality of virtual memory devices access the memory device ports at the same time. The bus arbitration unit in the application arbitrates according to different priority settings of each virtual memory device, and decides who to use the internal bus preferentially when the bus access conflicts, so as to ensure the service quality. A special priority field is set in the configuration space of each virtual storage device, for example, an 8-bit register can correspond to 256 priority levels, and the smaller the value, the higher the priority.
Optionally, in the embodiment of the present application, the internal bus interface is provided with an NVMe device driving module (storage device driving unit) for interacting with a controller of the NVMe storage device to complete initialization of the NVMe storage device, including BAR space content reading and storage capacity obtaining, interrupt and DMA configuration, setting hardware attributes such as AER, configuring CQ and SQ queues, and the like. Meanwhile, during presentation of the NVMe storage device to the plurality vDevice (virtual storage device) through the internal bus interface, the NVMe driver may implement mapping of PCIe address space to internal bus address space of the NVMe storage device for vDevice access.
In this embodiment, the bus interface is a high-bandwidth parallel bus interface that connects each virtual storage device and a storage device port, where the bandwidth is far greater than the transmission bandwidth of PCIe, and data transmission between each virtual storage device and a downstream port can be implemented with very low latency. FIG. 4 is a schematic diagram of an alternative bus interface according to an embodiment of the present application, where the storage expansion apparatus shown in FIG. 4 includes a channel expansion logic module and an internal bus interface module, where the channel expansion logic module is configured to connect to a server port (server or a virtual machine deployed on the server), and the channel expansion logic is further configured to connect to an internal bus interface, where the internal bus interface module is connected to a storage device port (first storage device), and where the channel expansion logic module is configured to create one or more storage channels on the storage expansion apparatus that are allowed to be identified by the server as a second storage device, where the internal bus interface includes a bus arbitration unit, an NVMe driving unit (storage device driving unit), and a system management unit, where the functions of the channel expansion logic is as follows:
Bus arbitration unit: there is a problem of bus collision when multiple vDevice accesses to the storage device port simultaneously. The bus arbitration unit in the invention arbitrates according to different priority settings of each vDevice, and decides who to use the internal bus preferentially when the bus access conflicts, so as to ensure the service quality. Setting a special priority field in the configuration space of each vDevice, such as an 8-bit register, can correspond to 256 priority levels, and the smaller the value, the higher the priority.
NVMe drive unit: in the invention, the internal bus interface is provided with an NVMe device driving module which is used for interacting with a controller of NVMe storage equipment to finish initialization work of the NVMe storage equipment, wherein the initialization work comprises hardware attribute setting such as BAR space content reading and storage capacity acquisition, interrupt and DMA configuration, AER and the like, and configuration CQ and SQ queues and the like. Meanwhile, in the process that the NVMe storage device presents to the plurality vDevice through the internal bus interface, the NVMe driver may implement mapping from the PCIe address space to the internal bus address space of the NVMe storage device for vDevice access.
System management unit: the system management unit is responsible for providing the device management interface of the invention, and realizing vDevice addition and removal, vDevice storage capacity configuration, vDevice priority management, device and operation information reporting and other user management functions. Since vDevice is implemented entirely by internal logic units, no external interfaces of the device are involved, and thus can be flexibly modified according to application scenarios. Besides managing the internal bus interface, the system management unit also has the functions of receiving the setting of the terminal user, modifying the parameter setting of the whole system including vBridge and vDevice, processing the system abnormal event and the like so as to ensure the reliable operation of the system.
As an alternative embodiment, after said controlling said reference memory space of said first memory device to perform said second data access, said method further comprises:
receiving access result information returned by the reference storage space in response to the second data access;
forwarding the access result information to the reference storage channel;
And sending the access result information to the server through the reference storage channel.
Optionally, in the embodiment of the present application, the access result information returned by the reference storage space in response to the second data access may include, but is not limited to, data read from the reference storage space, or may also be a write result returned after the data protocol to be written into the reference storage space refers to the storage space, which is not limited in this scheme.
Optionally, in the embodiment of the present application, the mapping relationship between the second data access and the virtual machine initiating the data access on the server may be, but not limited to, saved in the reference storage channel, so that the reference storage channel finds the target virtual machine corresponding to the access result information according to the mapping relationship, and then feeds back the access result to the target virtual machine on the server.
Optionally, in the embodiment of the present invention, the storage expansion device may, but is not limited to, configure a corresponding data buffer space for each reference storage channel, so as to buffer data in the reference storage channel, and further, when the reference storage channel is in an occupied state with a data transmission link between the server when receiving the access result information, may temporarily store the access result information in the data buffer space.
As an optional embodiment, the forwarding the access result information to the reference storage channel includes:
Extracting a source address from the access result information to obtain the reference storage space;
determining the reference storage channel corresponding to the reference storage space from the storage channels established in the storage expansion device;
And sending the access result information to the reference storage channel.
Optionally, in the embodiment of the present application, the reference storage channel may be, but is not limited to, a storage expansion device that determines from one or more storage channels according to a mapping relationship between a stored reference storage space and a storage channel, which is not limited in this scheme.
As an optional embodiment, the sending the access result information to the server through the reference storage channel includes:
Replacing a source address in the access result information with the reference storage device from the reference storage space to obtain reference access result information;
and sending the reference access result information to the server.
As an alternative to this embodiment,
The method further comprises one of:
determining the number of channels of the storage channels established in the storage expansion device according to the resource information on the storage expansion device;
determining the number of channels of the storage channels established in the storage expansion device according to the resource information on the storage expansion device and the equipment information of the first storage equipment;
And determining the channel number of the storage channels established in the storage expansion device according to the resource information on the storage expansion device, the equipment information of the first storage equipment and the virtual machine information of the virtual machine deployed on the server.
Optionally, in the embodiment of the present application, the number of storage channels in the storage expansion device may be flexibly configured according to requirements.
Optionally, in the embodiment of the present application, the resource information on the storage expansion device may be, but is not limited to, an available space capacity of a storage resource, or a number of candidate storage spaces obtained by dividing the storage resource according to a target granularity, where each candidate storage space corresponds to one storage channel.
Optionally, in the embodiment of the present application, the virtual machine information may include, but is not limited to, the total number of virtual machines deployed on the current server, and the number of virtual machines required for accessing the data currently stored on the server, which is not limited in this scheme.
As an optional embodiment, the determining, according to the resource information on the storage expansion device and the device information of the first storage device, the number of channels of the storage channels established in the storage expansion device includes:
Detecting a connection with the first storage device;
loading a storage device driver upon detecting that a connection has been made to the first storage device;
Acquiring the device information from the first storage device through the storage device driver;
And determining the channel number of the one or more storage channels according to the resource information on the storage expansion device and the equipment information of the first storage equipment.
The application relates to a storage expansion device, which consists of four parts including a server port, a channel expansion logic, an internal bus interface and a storage device port, wherein fig. 5 is an optional storage device connection system architecture diagram according to an embodiment of the application, as shown in fig. 5, the storage expansion device comprises four parts including a server port, a channel expansion logic, an internal bus interface and a PCIe downstream port, and the functions of the storage expansion device are as follows:
1. server port:
the server port is a hardware interface between the NVMe extension device and the server CPU and is responsible for establishing physical connection with the CPU PCIe Root Complex. The server port contains electrical attribute and logic attribute of PCIe physical layer, can complete the transmission and power supply of signals such as transceiver, clock, reset, etc., implement serial-parallel conversion, link training, etc., and can be in the form of connector, golden finger, cable, etc.
2. Storage device port:
The storage equipment port is a hardware interface between the NVMe extension device and the NVMe storage equipment and is responsible for establishing connection with the NVMe storage equipment. The storage device port contains electrical and logical attributes in a physical layer in a PCIe protocol, can transmit signals such as a transceiver, a clock, reset and the like and supply power, can complete functions such as serial-parallel conversion and link training, and can be in the form of a connector, a golden finger, a cable and the like.
3. Channel expansion logic:
The channel expansion logic is an important part of the invention, and is responsible for dynamically creating a plurality of virtual PCIe devices (vEP) inside the NVMe expansion device, and the virtual PCIe devices are mainly composed of virtual PCIe Bridge (vBridge) and virtual PCIe devices (vDevice). The channel expansion logic is coupled to the memory device port via an internal bus. For servers, these vDevice are the same as the properties of other physical PCIe devices, and virtual machine management software (VMM) can easily allocate them for use by Virtual Machines (VMs). For the downstream NVMe storage device, the NVMe expansion device is embodied as an NVMe master control device, and can perform operations such as reading, writing, erasing and the like on the NVMe storage device. The channel expansion logic is internally divided into upstream and downstream channels. The data direction of the downlink channel is the direction from the CPU to the NVMe device, and the data direction of the uplink channel is the direction from the NVMe device to the CPU.
FIG. 6 is a flowchart of an alternative storage expansion device operation according to an embodiment of the present application, as shown in FIG. 6, including at least the following steps:
S601, internal bus interface configuration. And (3) enumerating and managing NVMe storage equipment (first storage equipment) through a downstream PCIe port, loading a driver, and acquiring equipment information. Such as PCIe bit width and speed class, storage capacity, etc., provide information for the number and capacity parameter configuration of the subsequent vDevice.
S602, vDevice (virtual storage) configuration. The content comprises PCIe attribute configuration such as DID/VID and BAR space, NVMe attribute configuration such as storage capacity and queue number, and private attribute configuration such as buffer size and priority. This is done by the system management unit of the internal bus interface.
S603, vBridge (virtual bridge device, including the first virtual bridge device and the second virtual bridge device) configuration. In the invention vBridge is used for forwarding the data packet between PCIe RC and vDevice, and each vBridge is connected with only one vDevice downstream device, so that standard PCIe attribute setting is required in the PCIe configuration space, and the information such as the bus number of the downstream device can be automatically determined when the CPU enumerates PCIe device tree. This is done by the system management unit of the internal bus interface.
S604, starting a server OS, enumerating PCIe bus devices, and successfully enumerating N Endpoint (vDevice) and N+1 Bridge (vBridge) devices.
S605, the server loads NVMe standard drive, and further identifies N NVMe devices. N NVMe devices are allocated to a plurality of virtual machines under the action of a Virtual Machine Manager (VMM).
S606, accessing storage equipment by an application program on each virtual machine, and finally accessing the entity NVMe storage equipment through a PCIe upstream port VBridge < - > VDevice < - > internal bus interface < - > PCIe downstream port < - > NVMe storage equipment data channel.
FIG. 7 is an alternative storage device access flow chart according to an embodiment of the application, as shown in FIG. 7, including at least the steps of:
S701, the channel expansion logic initializes vDevice (virtual storage device), vBridge (virtual bridge device, including the first virtual bridge device and the second virtual bridge device) configuration. The CPU enumerates the PCIe device tree, identifies N virtual vDevice devices, and is assigned to each virtual machine by the VMM.
S702, a certain virtual machine initiates a read-write request of vDevice to which the virtual machine belongs. The PCIe RC of the CPU sends the request to the upstream port in the present invention according to the accessed address range.
S703, after being imported through the upstream port, the data packet is analyzed through vBridge protocol, and forwarded to the corresponding vDevice according to PCIe ID number or address space range.
And S704, sequentially analyzing PICe and NVMe protocols in vDevice, and converting address access to vDevice into access to an internal bus address through an address mapping link.
And S705, vDevice, after obtaining the bus use right through bus arbitration, converting the internal bus address access into read-write access to the downstream physical NVMe storage device with the assistance of NVMe drive, and sending the read-write access to the PCIe downstream port.
S706, the access request is sent to the NVMe storage device via the PCIe downstream port and processed. The returned data is reversely transmitted back to the CPU through the links in sequence.
In the embodiment, by introducing the NVMe storage equipment extension device, the common NVMe SSD has a hardware-supported virtual-like function, key links such as protocol analysis, equipment management and address mapping are realized by channel extension logic in the FPGA, and the compatibility problem brought by complex configuration processes of traditional virtualization technologies such as SR-IOV is solved.
The embodiment of the application can be applied to the extension of the NVMe storage equipment, and can realize that a single NVMe storage equipment can be presented to a plurality of virtual NVMe equipment of a CPU by adding one NVMe channel extension device in a link. The application is different from the existing software virtualization scheme and hardware PCIE SWICH expansion scheme, and has the high efficiency of the hardware expansion scheme and the high flexibility of the software virtualization scheme.
The NVMe channel expansion device comprises a PCIe upstream port, a PCIe downstream port, channel expansion logic and an internal bus interface unit.
The PCIe upstream port is a hardware interface between the device and the server and is responsible for establishing physical connection with PCIe RC of the CPU of the server;
The PCIe downstream port is a hardware interface between the device and the NVMe storage device, and is responsible for establishing physical connection with the NVMe storage device.
The channel expansion logic completes protocol analysis and forwarding of respective PCIe link layers and transaction layers by creating a plurality of PCIe bridge devices (vBridge) and Endpoint devices (vDevice), and realizes the expansion function of PCIe channels.
The internal bus interface unit completes address mapping between the PCIe addressing space and the NVMe storage space, and achieves multichannel access between the plurality of extended PCIe devices (vDevice) and the NVMe storage devices.
The PCIe upstream port and the PCIe downstream port have the electrical and logic functions of a PCIe physical layer, can transmit signals such as a transceiver, a clock, reset and the like and supply power, complete serial-parallel conversion, link training and the like, and can be in the form of connectors, golden fingers, cables and the like.
The vBridge and vDevice can realize protocol analysis and data forwarding of a PCIe data link layer and a transaction layer, and non-core links such as a physical layer, consistency test, power management and the like are not realized, so that hardware and logic resources are saved, and the overall link delay is reduced.
The number of vBridge and vDevice can be flexibly adjusted by the system management unit in the invention.
The internal bus interface unit consists of three units, namely a bus president, an NVMe drive and a system management unit.
The bus arbitration unit implements a priority based bus arbitration mechanism in which a plurality vDevice of the bus arbitration units communicate with one NVMe storage device.
The NVMe driving unit realizes address conversion and access control of the internal bus space and the NVMe storage space.
The system management unit realizes the management functions of channel expansion capacity configuration, operation information reporting, exception handling and the like.
Also provided in this embodiment is a storage expansion device, and fig. 8 is a block diagram of an alternative storage expansion device according to an embodiment of the present application, as shown in fig. 8, including:
A server port, a processing chip and a storage device port, wherein the server port is connected with the processing chip, the processing chip is connected with the storage device port, one or more storage channels are established in the processing chip,
The server port is used for connecting a server, wherein one or more virtual machines are deployed on the server, the one or more storage channels allow the server to identify the one or more storage channels as one or more second storage devices, and the one or more second storage devices are distributed to the virtual machines deployed on the server;
The storage device port is used for connecting a first storage device, wherein the storage channel is allocated with a corresponding storage space on the first storage device;
The processing chip is used for realizing the steps of the access method of the storage device.
Through the above, the storage expansion device is deployed between the server and the first storage device deployed by the server, and one or more storage channels established by the storage expansion device allow the server to identify the storage channels as one or more second storage devices and allocate the storage channels to the virtual machines deployed on the server for use, so that each virtual machine can be considered to be connected with one second storage device on the virtual machine side, further, after the storage expansion device receives a data access request sent by the reference virtual machine on the server, the storage expansion device converts the data access of the reference storage device into the data access of the storage space corresponding to the reference storage channel in the first storage device through the reference storage channel identified as the reference storage device in the one or more storage channels, thereby realizing that services can be provided for a plurality of virtual machines on the server only by connecting one storage device, realizing the expansion of the storage devices connected with the service on the premise that the service logic of the storage device is not changed and the storage device is not required to be newly increased, and further providing each virtual machine for the server with higher flexibility and better universal storage performance. The problem that the expansion efficiency of the storage equipment is lower when the storage equipment connected with the server is expanded in the related technology can be solved, and the effect of improving the expansion efficiency of the storage equipment when the storage equipment connected with the server is expanded is achieved.
As an alternative embodiment, a first virtual bridge device is disposed in the processing chip, where the first virtual bridge device is connected to the one or more storage channels, and the first virtual bridge device is further configured to connect to the server port and is recognized by the server as being connected to the second storage device, where the first virtual bridge device records the second storage device and the storage channel that have a corresponding relationship,
The first virtual bridge device is configured to search, when a first data access is received, a reference storage channel corresponding to a reference storage device from the second storage devices and storage channels having a corresponding relationship, where the reference storage device is a storage device allocated to a reference virtual machine that initiates a data access request in the one or more second storage devices, and the reference storage channel is a storage channel identified as the reference storage device in the one or more storage channels; forwarding the first data access to the reference storage channel.
As an alternative embodiment, the storage channel comprises a second virtual bridge device and a virtual storage device, the second virtual bridge device being connected to the first virtual bridge device, the second virtual bridge device being connected to the virtual storage device, the virtual storage device being connected to the first storage device, wherein,
The second virtual bridge device is configured to forward the first data access to the virtual storage device connected to the second virtual bridge device when the first data access is received;
The virtual storage device is used for converting the first data access into a second data access for accessing the storage space corresponding to the reference storage channel in the first storage device.
Optionally, in the embodiment of the present application, a data storage resource is deployed in a processing chip, which is used for temporarily storing data that needs to be accessed by a virtual storage device in a first storage device, and for one or more storage channels established in a storage expansion device, the processing chip needs to allocate the data storage resource according to the number of creation of the storage channels, and when allocating, the data storage resource may be allocated according to the resource requirements of each storage channel, or may also be equally divided according to the number of creation of the storage channels, which is not limited by this scheme.
As an optional embodiment, the storage expansion device further deploys an external storage device, where the external storage device is connected to the virtual storage device,
The external storage device is used for caching the data to be accessed by the virtual storage device in the first storage device.
Optionally, in the embodiment of the present application, the external storage device is configured to provide a data storage service for the storage channels when the data storage resources of the processing chip are not enough, one external storage device may be configured for each storage channel, or a plurality of storage channels may be connected to a common external storage device, and the external storage device is used to provide a data storage service for all storage channels.
As an alternative embodiment, a target processor is also disposed in the processing chip, the target processor being coupled to the one or more memory channels via an internal high-speed bus, the target processor also being coupled to the memory device port, wherein,
And the target processor is used for responding to the data access request transmitted by the storage channel and performing storage control and data access on the first storage device.
Optionally, in the embodiment of the present application, the storage expansion device may be, but is not limited to, configured with a storage device connected to the target processor, such as a DDR memory, where a Linux operating system is run under the support of DDR, and management and access to NVMe storage devices are implemented under the drive of standard NVMe.
The processing chip can realize the virtualization of the NVMe storage device through the FPGA accelerator in the form of a PCIe Add-in card, but is not limited to the processing chip. The device comprises two PCIe ports, wherein one PCIe port is configured into PCIe EndPoint interfaces (server ports) in the embodiment, and a golden finger is connected with a server CPU; the other PCIe port is configured as a PCIe RC interface (storage device port) to connect the downstream NVMe storage device through one MCIO connector. The PCIe channel expansion logic in the application is realized in the IP of the FPGA (processing chip), and the management and access to the NVMe storage equipment are realized through the NVMe driving IP. FIG. 9 is a block diagram of an alternative memory expansion device according to an embodiment of the present application, where as shown in FIG. 9, the PCIe upstream port (server port) is in the form of a PCIe golden finger for plugging into a PCIe slot of a server, and the PCIeRC interface of a connector CPU. The PCIe downstream port (storage device port) is a MCIO-form high-speed connector, and may be connected to an NVMe SSD through a cable. In the channel expansion logic, according to the resource condition of the FPGA, 16 vBridge (second virtual bridge device) and corresponding vDevice are set in total, which share an 8TB NVMe storage device (first storage device) connected to a downstream port, and each vDevice (virtual storage device) is averagely allocated to 512GB capacity. With the support of a Virtual Machine Manager (VMM), 16 vDevice (virtual storage devices) are allocated to 16 different virtual machines for storing user data that varies frequently and has relatively low security requirements. The FPGA includes a processor unit (PS side) having a four-core ARM a53 processor as a core, in addition to a logic unit (PL side). And the PS side runs a Linux operating system under the support of an external DDR, and realizes the management and access of the NVMe storage equipment under the drive of a standard NVMe. The PL side is also configured with an external DDR of 8GB, which can be used as a vDevice data cache when the SRAM resources inside the FPGA are insufficient. The system management function is realized through the SMBUS. Through the SMBUS, the storage virtualization device can receive out-of-band instructions from the server, system performance settings such as virtual machine quantity setting, vDevice priority setting, data cache adjustment, virtual capacity allocation and the like are realized, and device information reading such as power consumption, temperature and the like of the whole device, and system resetting, powering up and powering down and the like can also be realized.
From the description of the above embodiments, it will be clear to a person skilled in the art that the method according to the above embodiments may be implemented by means of software plus the necessary general hardware platform, but of course also by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present application.
The embodiment also provides an access device for a storage device, which is used for implementing the foregoing embodiments and preferred embodiments, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
Fig. 10 is a block diagram of the structure of an access apparatus of a storage device according to an embodiment of the present application, as shown in fig. 10,
The storage expansion device is applied to a storage expansion device, the storage expansion device is connected between a first storage device and a server, one or more virtual machines are deployed on the server, one or more storage channels which are allowed to be identified as second storage devices by the server are established in the storage expansion device, the second storage devices are allocated to the virtual machines deployed on the server, the storage channels are allocated with corresponding storage spaces on the first storage devices, and the storage expansion device comprises:
a receiving module 1002, configured to receive a data access request sent by a reference virtual machine, where the one or more virtual machines include the reference virtual machine, where the data access request is used to request a first data access to a reference storage device allocated to the reference virtual machine in the second storage device;
A conversion module 1004, configured to respond to the data access request, and convert the first data access into a second data access through a reference storage channel, where the reference storage channel is a storage channel identified as the reference storage device in a storage channel established in the storage expansion apparatus, and the second data access is a data access to a reference storage space corresponding to the reference storage channel in the first storage device;
a control module 1006, configured to control the reference storage space of the first storage device to perform the second data access.
Through the above, the storage expansion device is deployed between the server and the first storage device deployed by the server, and one or more storage channels established by the storage expansion device allow the server to identify the storage channels as one or more second storage devices and allocate the storage channels to the virtual machines deployed on the server for use, so that each virtual machine can be considered to be connected with one second storage device on the virtual machine side, further, after the storage expansion device receives a data access request sent by the reference virtual machine on the server, the storage expansion device converts the data access of the reference storage device into the data access of the storage space corresponding to the reference storage channel in the first storage device through the reference storage channel identified as the reference storage device in the one or more storage channels, thereby realizing that services can be provided for a plurality of virtual machines on the server only by connecting one storage device, realizing the expansion of the storage devices connected with the service on the premise that the service logic of the storage device is not changed and the storage device is not required to be newly increased, and further providing each virtual machine for the server with higher flexibility and better universal storage performance. The problem that the expansion efficiency of the storage equipment is lower when the storage equipment connected with the server is expanded in the related technology can be solved, and the effect of improving the expansion efficiency of the storage equipment when the storage equipment connected with the server is expanded is achieved.
Optionally, the conversion module includes:
a first determining unit configured to determine a reference storage channel identified as the reference storage device from among storage channels established in the storage expansion means;
A forwarding unit, configured to forward the first data access to the reference storage channel;
And the first conversion unit is used for converting the first data access into the second data access through the reference storage channel.
Optionally, a first virtual bridge device is further disposed in the storage expansion apparatus, the first virtual bridge device is connected with the storage channel established in the storage expansion apparatus, the first virtual bridge device is further used for connecting the server and is identified by the server as being connected with the second storage device established in the storage expansion apparatus, the first virtual bridge device records the second storage device and the storage channel with corresponding relation,
The first determining unit is configured to: searching the reference storage channel corresponding to the reference storage device from the second storage device and the storage channel with corresponding relation through the first virtual bridge device;
the forwarding the first data access to the reference storage channel includes: forwarding, by the first virtual bridge device, the first data access to the reference storage channel.
Optionally, the first determining unit is configured to:
Extracting, by the first virtual bridge device, a reference device identifier of the reference storage device from the data access request;
And searching the reference storage channel corresponding to the reference device identifier from the storage device identifiers and the storage channels which have a corresponding relationship through the first virtual bridge device, wherein the storage device identifier is a device identifier which is allocated to the corresponding storage channel and is allowed to be identified as a storage device.
Optionally, the apparatus further includes:
The first creation module is used for creating an initial virtual bridge device in the storage expansion device;
A first configuration module, configured to configure a first attribute of a transmission protocol for the initial virtual bridge device in a protocol configuration space of the initial virtual bridge device, so as to obtain the first virtual bridge device, where the server and the storage expansion device are connected through a protocol bus conforming to the transmission protocol, and the first attribute is a protocol attribute of the bridge device that is allowed to be identified by the server as conforming to the transmission protocol;
And the processing module is used for establishing the connection between the first virtual bridging device and the storage channel established in the storage expansion device and recording the second storage device and the storage channel with corresponding relations.
Optionally, the storage channel includes a second virtual bridge device and a virtual storage device, the first virtual bridge device is connected to the second virtual bridge device, the second virtual bridge device is connected to the virtual storage device, the virtual storage device is used to connect to the first storage device,
The conversion module comprises:
a first receiving unit, configured to receive the first data access through the second virtual bridge device in the reference storage channel;
a transmission unit, configured to transmit, by the second virtual bridge device in the reference storage channel, the first data access to the virtual storage device in the reference storage channel;
and the second conversion unit is used for converting the first data access into the second data access through the virtual storage device in the reference storage channel.
Optionally, the second converting unit is configured to:
Acquiring a first access address of the access indicated by the first data access through the virtual storage device in the reference storage channel, wherein the first access address is a virtual address of the reference storage device indicated by the reference virtual machine;
querying, by the virtual storage device in the reference storage channel, a first storage address corresponding to the first access address from a virtual address range and a storage address range having a corresponding relationship, where the virtual address range is an address range of the reference storage device identified by the server, and the storage address range is an address range of a storage space allocated on the first storage device by the virtual storage device in the reference storage channel;
And replacing the first access address in the first data access with the first storage address to obtain the second data access.
Optionally, the second converting unit is configured to:
Querying, by the virtual storage device in the reference storage channel, the first storage address mapped by the first access address from an address mapping between an addressing space of a transport protocol of the reference storage device and a storage space of a storage protocol of the first storage device, wherein the reference storage device is identified by the server as a device conforming to the transport protocol, the virtual address range includes the addressing space of the transport protocol, the first storage device is a storage device of the storage protocol, and the storage address range includes the storage space of the storage protocol.
Optionally, the apparatus further includes:
the second creation module is used for creating a plurality of groups of initial virtual bridging devices and initial virtual storage devices in the storage expansion device;
A second configuration module, configured to configure a second attribute of a transmission protocol of the initial virtual bridge device in a protocol configuration space of the initial virtual bridge device, so as to obtain the second virtual bridge device, where the server and the storage expansion device are connected through a protocol bus conforming to the transmission protocol, and the second attribute is a protocol attribute of the bridge device that is allowed to be identified by the server as conforming to the transmission protocol;
and the third configuration module is used for carrying out third attribute configuration of a transmission protocol on the initial virtual storage device in a transmission protocol configuration space of the initial virtual storage device, and carrying out fourth attribute configuration of a storage protocol on the initial virtual storage device in a storage protocol configuration space of the initial virtual storage device to obtain the virtual storage device, wherein the first storage device is a storage device conforming to the storage protocol, the third attribute is a protocol attribute of a terminal device which is allowed to be identified by the server as conforming to the transmission protocol, and the fourth attribute is a protocol attribute of a storage device which is allowed to be identified by the server as conforming to the storage protocol.
Optionally, the virtual storage device has a protocol parsing unit deployed therein, and the third configuration module includes:
A configuration unit, configured to perform, in a transport protocol configuration space of the initial virtual storage device, configuration of a data link layer attribute and configuration of a transaction layer attribute of the transport protocol on the initial virtual storage device, where the third attribute configuration includes: the protocol analysis unit is used for carrying out protocol analysis of a data link layer on the received data access by using the data link layer attribute and/or carrying out protocol analysis of a transaction layer on the received data access by using the transaction layer attribute.
Optionally, the configuration unit is configured to:
And performing configuration of storage capacity attribute and queue number attribute of the storage protocol on the initial virtual storage device in a storage protocol configuration space of the initial virtual storage device, wherein the fourth attribute configuration comprises: the storage capacity attribute and the queue number attribute are for use by the server to identify storage devices that conform to the storage protocol.
Optionally, a target bus and a bus interface are further disposed in the storage expansion device, where the bus interface is used to connect the plurality of storage channels connected to the target bus to the first storage device, and the bus interface has a mapping relationship between a bus address space of the target bus and a storage address space of the first storage device,
The execution module comprises:
a second receiving unit configured to receive the second data access through the target bus;
And the forwarding unit is used for forwarding the second data access to the reference storage space according to the mapping relation through the bus interface.
Optionally, the bus interface includes: a bus arbitration unit and a memory device driving unit having therein a mapping relationship of a bus address space of the target bus and a memory address space of the first memory device,
The forwarding unit is configured to:
determining whether to process bus access of the reference memory channel according to the priority of the reference memory channel through the bus arbitration unit;
And forwarding the second data access to the reference memory space by the memory device driving unit according to the mapping relation under the condition of determining to process the bus access of the reference memory channel.
Optionally, the apparatus further includes:
the receiving module is used for receiving access result information returned by the reference storage space in response to the second data access after the reference storage space of the first storage device is controlled to execute the second data access;
the forwarding module is used for forwarding the access result information to the reference storage channel;
and the sending module is used for sending the access result information to the server through the reference storage channel.
Optionally, the forwarding module includes:
The extraction unit is used for extracting a source address from the access result information to obtain the reference storage space;
A second determining unit, configured to determine, from storage channels established in the storage expansion device, the reference storage channel corresponding to the reference storage space;
and the sending unit is used for sending the access result information to the reference storage channel.
Optionally, the sending unit is configured to:
Replacing a source address in the access result information with the reference storage device from the reference storage space to obtain reference access result information;
and sending the reference access result information to the server.
Optionally, the apparatus further comprises one of:
the first determining module is used for determining the number of channels of the storage channels established in the storage expansion device according to the resource information on the storage expansion device;
the second determining module is used for determining the channel number of the storage channels established in the storage expansion device according to the resource information on the storage expansion device and the equipment information of the first storage equipment;
And the third determining module is used for determining the channel number of the storage channels established in the storage expansion device according to the resource information on the storage expansion device, the equipment information of the first storage equipment and the virtual machine information of the virtual machine deployed on the server.
Optionally, the first determining module includes:
a detection unit configured to detect a connection with the first storage device;
a loading unit configured to load a storage device driver in a case where it is detected that the first storage device has been connected;
An obtaining unit, configured to obtain the device information from the first storage device through the storage device driver;
And the determining unit is used for determining the channel number of the storage channels established in the storage expansion device according to the resource information on the storage expansion device and the equipment information of the first storage equipment.
It should be noted that each of the above modules may be implemented by software or hardware, and for the latter, it may be implemented by, but not limited to: the modules are all located in the same processor; or the above modules may be located in different processors in any combination.
Embodiments of the present application also provide a computer readable storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the method embodiments described above when run.
In one exemplary embodiment, the computer readable storage medium may include, but is not limited to: a usb disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory RAM), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing a computer program.
An embodiment of the application also provides an electronic device comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.
In an exemplary embodiment, the electronic device may further include a transmission device connected to the processor, and an input/output device connected to the processor.
Specific examples in this embodiment may refer to the examples described in the foregoing embodiments and the exemplary implementation, and this embodiment is not described herein.
It will be appreciated by those skilled in the art that the modules or steps of the application described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may be implemented in program code executable by computing devices, so that they may be stored in a storage device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than that shown or described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps of them may be fabricated into a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.
The above description is only of the preferred embodiments of the present application and is not intended to limit the present application, but various modifications and variations can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the principle of the present application should be included in the protection scope of the present application.

Claims (22)

1. A method for accessing a memory device, characterized in that,
The method is applied to a storage expansion device, the storage expansion device is connected between a first storage device and a server, one or more virtual machines are deployed on the server, one or more storage channels which are allowed to be identified as second storage devices by the server are established in the storage expansion device, the second storage devices are allocated to the virtual machines deployed on the server, and corresponding storage spaces are allocated on the first storage devices by the storage channels, and the method comprises the following steps:
Receiving a data access request sent by a reference virtual machine, wherein the one or more virtual machines comprise the reference virtual machine, and the data access request is used for requesting first data access to a reference storage device allocated to the reference virtual machine in the second storage device;
Responding to the data access request, and converting the first data access into second data access through a reference storage channel, wherein the reference storage channel is a storage channel which is identified as the reference storage device in a storage channel established in the storage expansion device, and the second data access is data access to a reference storage space corresponding to the reference storage channel in the first storage device;
controlling the reference storage space of the first storage device to execute the second data access.
2. The method of claim 1, wherein the step of determining the position of the substrate comprises,
The converting the first data access to a second data access by referring to a storage channel includes:
determining a reference storage channel identified as the reference storage device from storage channels established in the storage expansion means;
forwarding the first data access to the reference storage channel;
And converting the first data access into the second data access through the reference storage channel.
3. The method of claim 2, wherein the step of determining the position of the substrate comprises,
The storage expansion device is further provided with a first virtual bridge device, the first virtual bridge device is connected with a storage channel established in the storage expansion device, the first virtual bridge device is further used for connecting the server and is recognized by the server as being connected with the second storage device, the first virtual bridge device records a second storage device and a storage channel with corresponding relations,
The determining a reference storage channel identified as the reference storage device from the storage channels established in the storage expansion apparatus includes: searching the reference storage channel corresponding to the reference storage device from the second storage device and the storage channel with corresponding relation through the first virtual bridge device;
the forwarding the first data access to the reference storage channel includes: forwarding, by the first virtual bridge device, the first data access to the reference storage channel.
4. The method of claim 3, wherein the step of,
The searching, by the first virtual bridge device, the reference storage channel corresponding to the reference storage device from the second storage device and the storage channel having a corresponding relationship, includes:
Extracting, by the first virtual bridge device, a reference device identifier of the reference storage device from the data access request;
And searching the reference storage channel corresponding to the reference device identifier from the storage device identifiers and the storage channels which have a corresponding relationship through the first virtual bridge device, wherein the storage device identifier is a device identifier which is allocated to the corresponding storage channel and is allowed to be identified as a storage device.
5. The method of claim 3, wherein the step of,
The method further comprises the steps of:
Creating an initial virtual bridge device in the storage expansion device;
Performing first attribute configuration of a transmission protocol on the initial virtual bridging device in a protocol configuration space of the initial virtual bridging device to obtain the first virtual bridging device, wherein the server and the storage expansion device are connected through a protocol bus conforming to the transmission protocol, and the first attribute is a protocol attribute of the bridging device which is allowed to be identified by the server as conforming to the transmission protocol;
and establishing connection between the first virtual bridge equipment and the storage channel established in the storage expansion device, and recording a second storage equipment and the storage channel with corresponding relations.
6. The method of claim 1, wherein the step of determining the position of the substrate comprises,
The storage channel comprises a second virtual bridge device and a virtual storage device, a first virtual bridge device is connected with the second virtual bridge device, the second virtual bridge device is connected with the virtual storage device, the virtual storage device is used for connecting the first storage device,
The converting the first data access to the second data access through the reference storage channel includes:
receiving, by the second virtual bridge device in the reference storage channel, the first data access;
Transmitting, by the second virtual bridge device in the reference storage channel, the first data access to the virtual storage device in the reference storage channel;
the first data access is converted to the second data access by the virtual storage device in the reference storage channel.
7. The method of claim 6, wherein the step of providing the first layer comprises,
Said converting said first data access to said second data access by said virtual storage device in said reference storage channel comprising:
Acquiring a first access address of the access indicated by the first data access through the virtual storage device in the reference storage channel, wherein the first access address is a virtual address of the reference storage device indicated by the reference virtual machine;
querying, by the virtual storage device in the reference storage channel, a first storage address corresponding to the first access address from a virtual address range and a storage address range having a corresponding relationship, where the virtual address range is an address range of the reference storage device identified by the server, and the storage address range is an address range of a storage space allocated on the first storage device by the virtual storage device in the reference storage channel;
And replacing the first access address in the first data access with the first storage address to obtain the second data access.
8. The method of claim 7, wherein the step of determining the position of the probe is performed,
The querying, by the virtual storage device in the reference storage channel, a first storage address corresponding to the first access address from a virtual address range and a storage address range having a corresponding relationship, includes:
Querying, by the virtual storage device in the reference storage channel, the first storage address mapped by the first access address from an address mapping between an addressing space of a transport protocol of the reference storage device and a storage space of a storage protocol of the first storage device, wherein the reference storage device is identified by the server as a device conforming to the transport protocol, the virtual address range includes the addressing space of the transport protocol, the first storage device is a storage device of the storage protocol, and the storage address range includes the storage space of the storage protocol.
9. The method of claim 6, wherein the step of providing the first layer comprises,
The method further comprises the steps of:
Creating a plurality of groups of initial virtual bridging devices and initial virtual storage devices in the storage expansion device;
Configuring a second attribute of a transmission protocol of the initial virtual bridge device in a protocol configuration space of the initial virtual bridge device to obtain the second virtual bridge device, wherein the server and the storage expansion device are connected through a protocol bus conforming to the transmission protocol, and the second attribute is a protocol attribute of the bridge device which is allowed to be identified by the server as conforming to the transmission protocol;
And performing third attribute configuration of a transmission protocol on the initial virtual storage device in a transmission protocol configuration space of the initial virtual storage device, and performing fourth attribute configuration of a storage protocol on the initial virtual storage device in a storage protocol configuration space of the initial virtual storage device to obtain the virtual storage device, wherein the first storage device is a storage device conforming to the storage protocol, the third attribute is a protocol attribute of a terminal device which is allowed to be identified by the server as conforming to the transmission protocol, and the fourth attribute is a protocol attribute of a storage device which is allowed to be identified by the server as conforming to the storage protocol.
10. The method of claim 9, wherein the step of determining the position of the substrate comprises,
The protocol analysis unit is deployed in the virtual storage device, and the performing, in the transport protocol configuration space of the initial virtual storage device, third attribute configuration of a transport protocol on the initial virtual storage device includes:
and performing configuration of data link layer attribute and transaction layer attribute of the transmission protocol on the initial virtual storage device in a transmission protocol configuration space of the initial virtual storage device, wherein the third attribute configuration comprises: the protocol analysis unit is used for carrying out protocol analysis of a data link layer on the received data access by using the data link layer attribute and/or carrying out protocol analysis of a transaction layer on the received data access by using the transaction layer attribute.
11. The method of claim 10, wherein the step of determining the position of the first electrode is performed,
The performing, in the storage protocol configuration space of the initial virtual storage device, a fourth attribute configuration of a storage protocol on the initial virtual storage device includes:
And performing configuration of storage capacity attribute and queue number attribute of the storage protocol on the initial virtual storage device in a storage protocol configuration space of the initial virtual storage device, wherein the fourth attribute configuration comprises: the storage capacity attribute and the queue number attribute are for use by the server to identify storage devices that conform to the storage protocol.
12. The method of claim 1, wherein the step of determining the position of the substrate comprises,
The memory expansion device is further provided with a target bus and a bus interface, wherein the bus interface is used for connecting a plurality of memory channels connected to the target bus to the first memory device, and has a mapping relation between a bus address space of the target bus and a memory address space of the first memory device,
The controlling the reference storage space of the first storage device to perform the second data access includes:
Receiving the second data access via the target bus;
And forwarding the second data access to the reference storage space according to the mapping relation through the bus interface.
13. The method of claim 12, wherein the step of determining the position of the probe is performed,
The bus interface includes: a bus arbitration unit and a memory device driving unit having therein a mapping relationship of a bus address space of the target bus and a memory address space of the first memory device,
The forwarding the second data access to the reference storage space through the bus interface according to the mapping relationship includes:
determining whether to process bus access of the reference memory channel according to the priority of the reference memory channel through the bus arbitration unit;
And forwarding the second data access to the reference memory space by the memory device driving unit according to the mapping relation under the condition of determining to process the bus access of the reference memory channel.
14. A storage expansion device is characterized in that,
Comprising the following steps:
A server port, a processing chip and a storage device port, wherein the server port is connected with the processing chip, the processing chip is connected with the storage device port, one or more storage channels are established in the processing chip,
The server port is used for connecting a server, wherein one or more virtual machines are deployed on the server, the one or more storage channels allow the server to identify the one or more storage channels as one or more second storage devices, and the one or more second storage devices are distributed to the virtual machines deployed on the server;
The storage device port is used for connecting a first storage device, wherein the storage channel is allocated with a corresponding storage space on the first storage device;
the processing chip having disposed thereon the method of any of claims 1 to 13.
15. The storage expansion device of claim 14, wherein the memory is configured to store the data,
The processing chip is provided with a first virtual bridge device, the first virtual bridge device is connected with the one or more storage channels, the first virtual bridge device is also used for connecting the server port and is identified by the server as being connected with the second storage device, the first virtual bridge device records the second storage device and the storage channels with corresponding relations,
The first virtual bridge device is configured to search, when a first data access is received, a reference storage channel corresponding to a reference storage device from the second storage devices and storage channels having a corresponding relationship, where the reference storage device is a storage device allocated to a reference virtual machine that initiates a data access request in the one or more second storage devices, and the reference storage channel is a storage channel identified as the reference storage device in the one or more storage channels; forwarding the first data access to the reference storage channel.
16. The storage expansion device of claim 15, wherein the memory is configured to store the data,
The storage channel comprises a second virtual bridge device and a virtual storage device, the second virtual bridge device is connected with the first virtual bridge device, the second virtual bridge device is connected with the virtual storage device, the virtual storage device is connected with the first storage device, wherein,
The second virtual bridge device is configured to forward the first data access to the virtual storage device connected to the second virtual bridge device when the first data access is received;
The virtual storage device is used for converting the first data access into a second data access for accessing the storage space corresponding to the reference storage channel in the first storage device.
17. The storage expansion device of claim 16, wherein the memory is configured to store the data,
An external storage device is also arranged in the storage expansion device, the external storage device is connected with the virtual storage device, wherein,
The external storage device is used for caching the data to be accessed by the virtual storage device in the first storage device.
18. The storage expansion device of claim 15, wherein the memory is configured to store the data,
A target processor is also disposed in the processing chip, the target processor being coupled to the one or more memory channels via an internal high-speed bus, the target processor also being coupled to the memory device port, wherein,
And the target processor is used for responding to the data access request transmitted by the storage channel and performing storage control and data access on the first storage device.
19. An access device for a storage device, characterized in that,
The storage expansion device is applied to a storage expansion device, the storage expansion device is connected between a first storage device and a server, one or more virtual machines are deployed on the server, one or more storage channels which are allowed to be identified as second storage devices by the server are established in the storage expansion device, the second storage devices are allocated to the virtual machines deployed on the server, the storage channels are allocated with corresponding storage spaces on the first storage devices, and the storage expansion device comprises:
The receiving module is used for receiving a data access request sent by a reference virtual machine, wherein the one or more virtual machines comprise the reference virtual machine, and the data access request is used for requesting to perform first data access on a reference storage device allocated to the reference virtual machine in the second storage device;
The conversion module is used for responding to the data access request and converting the first data access into second data access through a reference storage channel, wherein the reference storage channel is a storage channel which is identified as the reference storage device in a storage channel established in the storage expansion device, and the second data access is data access to a reference storage space corresponding to the reference storage channel in the first storage device;
and the control module is used for controlling the reference storage space of the first storage device to execute the second data access.
20. A computer-readable storage medium comprising,
The computer readable storage medium has stored therein a computer program, wherein the computer program when executed by a processor realizes the steps of the method as claimed in any of claims 1 to 13.
21. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that,
The processor, when executing the computer program, implements the steps of the method as claimed in any one of claims 1 to 13.
22. A computer program product comprising a computer program, characterized in that,
Which computer program, when being executed by a processor, carries out the steps of the method as claimed in any one of claims 1 to 13.
CN202410376341.2A 2024-03-29 2024-03-29 Storage device access method and device, storage medium and electronic device Pending CN117971135A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410376341.2A CN117971135A (en) 2024-03-29 2024-03-29 Storage device access method and device, storage medium and electronic device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410376341.2A CN117971135A (en) 2024-03-29 2024-03-29 Storage device access method and device, storage medium and electronic device

Publications (1)

Publication Number Publication Date
CN117971135A true CN117971135A (en) 2024-05-03

Family

ID=90846433

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410376341.2A Pending CN117971135A (en) 2024-03-29 2024-03-29 Storage device access method and device, storage medium and electronic device

Country Status (1)

Country Link
CN (1) CN117971135A (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116774933A (en) * 2023-06-19 2023-09-19 北京火山引擎科技有限公司 Virtualization processing method of storage device, bridging device, system and medium
CN117608757A (en) * 2023-12-01 2024-02-27 北京火山引擎科技有限公司 Storage device control method and device, electronic device and storage medium
WO2024041412A1 (en) * 2022-08-24 2024-02-29 阿里云计算有限公司 Storage system and method, and hardware offload card

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024041412A1 (en) * 2022-08-24 2024-02-29 阿里云计算有限公司 Storage system and method, and hardware offload card
CN116774933A (en) * 2023-06-19 2023-09-19 北京火山引擎科技有限公司 Virtualization processing method of storage device, bridging device, system and medium
CN117608757A (en) * 2023-12-01 2024-02-27 北京火山引擎科技有限公司 Storage device control method and device, electronic device and storage medium

Similar Documents

Publication Publication Date Title
US20200278880A1 (en) Method, apparatus, and system for accessing storage device
US10534552B2 (en) SR-IOV-supported storage resource access method and storage controller and storage device
US8645594B2 (en) Driver-assisted base address register mapping
WO2021000717A1 (en) Method and apparatus for processing io
WO2019233322A1 (en) Resource pool management method and apparatus, resource pool control unit, and communication device
US11829309B2 (en) Data forwarding chip and server
US8566416B2 (en) Method and system for accessing storage device
US11741039B2 (en) Peripheral component interconnect express device and method of operating the same
EP3716084A1 (en) Apparatus and method for sharing a flash device among multiple masters of a computing platform
US11966585B2 (en) Storage device and storage system
WO2023174146A1 (en) Offloading-card namespace management system and method, and input/output request processing system and method
US8032675B2 (en) Dynamic memory buffer allocation method and system
US11314673B2 (en) Configurable multi-function PCIe endpoint controller in an SoC
CN115811509A (en) Bus communication method and related equipment
US10176133B2 (en) Smart device with no AP
WO2023186143A1 (en) Data processing method, host, and related device
US20060277326A1 (en) Data transfer system and method
CN117971135A (en) Storage device access method and device, storage medium and electronic device
US20200327091A1 (en) Device communication control module and device communication control method
JP4432388B2 (en) I / O controller
CN114238156A (en) Processing system and method of operating a processing system
US20230350824A1 (en) Peripheral component interconnect express device and operating method thereof
WO2024113704A1 (en) Pcie data transmission method and related apparatus
JP4930554B2 (en) I / O controller
US20180181440A1 (en) Resource allocation system, apparatus allocation controller and apparatus recognizing method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination