JP2003006137A - Storage access system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce latency by simplifying data transfer operation between a node device, such as a PC and a WS and an external input/output device, and also to secure a data transfer band (rate) that can fully utilizes the performance of an IB switch, when the IB switch is introduced. SOLUTION: This system comprises a virtual address management means 5 for making at least virtual address information associate with physical address information, showing a particular storage area of a storage means 25 of the external input/output device 2 side to manage them, in order to handle respective storage means 12 and 25 of the node device 1 and the external input/output device 2 side as virtual distributed shared memories, and a control means 6 for controlling data transfer between the storage area of the storage means 25 and the storage means 12 of the node device 1 via a connection means 3, on the basis of the virtual address information managed by the virtual address management means 5.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a storage access system, and more particularly to a personal computer (P
The present invention relates to a technique for accessing a node device such as C) or a workstation (WS) to an external input / output device such as a disk device.

[0002]

2. Description of the Related Art FIG. 21 shows an example of an existing storage access system. As shown in FIG. 21, the present system includes a node device 100 (hereinafter, simply referred to as a PC, WS, etc.
"Node 100") and SCSI (Small Comput
The system includes an input / output (I / O) device 200 connected to the node 100 via a bus system 300, and the node 100 includes a CPU 101, a main storage unit (main memory) 102, and a SCSI card 103. And the like, which are connected to each other via an internal bus 104 so that they can communicate with each other. In addition, the I / O device 200
Is equipped with a disk controller 201, a buffer 202, and a disk device 203.

The SCSI bus 300 has a plurality of I's.
The / O device 200 may be connected in some cases. Further, another interface such as a fiber channel (FC) may be used to connect the node 100 (main memory 102) and the I / O device 200. In such a system, the node 100 (main memory 102)
When data is transferred between the I / O device 200 and the I / O device 200 (disk device 203), the SCSI is set from the node 100 side.
It is necessary to start the disk controller 201 of the I / O device 200 by using the above protocol. For example, when the file system 105 writes data to the disk device 203, conventionally, the following procedure is executed.

That is, first, as shown in FIG. 21 and FIG. 22, the file system 105 on the node 100 side is SC
A disk write request is requested to the SI driver 106 (step A1). The above file system 10
5 and SCSI driver 106 are both normally O
It is incorporated as one function of S (operating system) and the like, and each function is realized by the CPU 101 reading and operating the file system data and the driver data stored in the main memory 102.

Now, the SCSI driver 106 that has received the above-mentioned disk write request operates as a target I / O device 200.
Disk controller 201 and SCSI bus 300
After setting the connection by executing the negotiation for determining the data transfer rate and the like several times (step A2), the data transfer is actually started and the disk controller 201 is requested to write the data ( Step A3). The SCSI driver 106
Disconnects the connection with the disk controller 201 once when it takes time to write data to the disk device 203 in the I / O device 200.

In the I / O device 200, the disk controller 201 is connected to the node 10 via the SCSI bus 300.
The data received from 0 is once stored in the buffer 202 (step A4) and then written in the disk device 203 (step A5). When the internal processing (data writing) in this I / O device 200 is completed (steps A6 and A).
7) The disk controller 201 raises an interrupt to the SCSI driver 106 to report that the transfer (write) is completed (step A8), and the SCSI driver 106 notifies the file system 105 of the transfer completion (step A8). A9).

[0007]

However, in such a conventional system, it is necessary to activate the disk controller 201 on the I / O device 200 side from the node 100 using a protocol such as SCSI or FC. However, the operation until data transfer actually occurs is complicated (need to negotiate several times between the node 100 and the disk controller 201), and therefore there is a problem that the latency is large.

Regarding the transfer band (transfer rate) between the node 100 and the I / O device 200, SCSI,
Both FC have been improved (up to 160MB / s), but when InfiniBand (IB) is introduced as an internal network, the transfer speed of the internal bus 104 on the node 100 side will be several GB.
It is possible to raise it to about / s. However, in the existing interface standards such as current SCSI and FC,
Since the latency is large as described above, the performance cannot be fully utilized.

The present invention was conceived in view of the above problems, and simplifies the data transfer operation to reduce the latency, and at the same time, when the high-speed internal network such as IB is introduced, its performance is sufficiently improved. An object of the present invention is to provide a storage access system that can secure a data transfer bandwidth (rate) that can be utilized.

[0010]

In order to achieve the above object, a storage access system of the present invention comprises: (1) a node device having a node device side storage means; and (2) an external input / output device side storage device. An external input / output device equipped with means,
(3) connection means for connecting these node devices and external input / output devices to enable data transfer between the node devices and external input / output devices, and (4) virtual distribution of the above storage means. In order to treat it as a shared memory, at least a virtual address management unit that manages the virtual address information and the physical address information indicating the specific storage area of the input / output side device storage unit in association with each other, (5) this virtual address A control means for controlling data transfer via a connection means between the storage area of the external input / output device side storage means and the node device side storage means based on virtual address information managed by the management means. (Claim 1).

In the storage access system of the present invention configured as described above, triggered by the access to the virtual distributed shared memory space using the virtual address information in the node device, the physical address information corresponding to the virtual address information is indicated. Since the storage area of the external input / output device side storage means is accessed, it is not necessary to perform complicated pre-processing such as negotiation prior to data transfer as in the prior art.

Here, a plurality of the above node devices may be connected through the above connecting means. In this case,
Each virtual address management means of these plurality of node devices,
It is preferable that the physical address information of the same external input / output device side storage means is managed in association with the above-mentioned virtual address information. As a result, each of the plurality of node devices can access a single external input / output device without performing complicated pre-processing such as negotiation (claim 2).

On the other hand, a plurality of the above-mentioned external input / output devices may be connected via the above-mentioned connecting means, and in this case, the above-mentioned virtual address management means is provided for each of these plurality of external input / output devices. It is preferable that the physical address information of the external input / output device side storage means is managed in association with the virtual address information. As a result, the physical address information of the node device side storage means and the physical address information of the plurality of external input / output device side storage means are mapped in the same memory space, and each memory is treated as a virtual distributed shared memory. , The node device can access the same memory space without performing any pre-processing such as complicated negotiation for any external input / output device side storage means (claim 3). ).

In this case, the virtual address management means may manage the physical address information of different external input / output device side storage means by associating it with the continuous virtual address information. With this configuration, by accessing the above memory space with successive virtual address information, different external I / O device side storage means are successively accessed (claim 4).

When a plurality of the external input / output devices are connected, the control means generates parity data for the data to be transferred and written in the storage area of the external input / output device side storage means. And a parity transfer unit that transfers the parity data generated by the parity data generation unit so as to be written in a specific external input / output device-side storage means.
Thereby, the data error can be detected by the parity check (claim 5).

Further, the above virtual address management means is
A plurality of pieces of physical address information of different external input / output device side storage means may be configured to be associated with the same virtual address information and managed. In this way, by accessing the virtual distributed shared memory space using certain virtual address information, it is possible to realize simultaneous access to a plurality of different external input / output device side storage means (claim 6).

The external input / output device side storage means includes a recording device and an external input / output device side buffer for temporarily buffering data transferred between the recording device and the node device. In this case, the virtual address management means may include the physical address information of the kernel buffer as the node device side storage means and the physical address information of the external input / output device side buffer, respectively. As virtual address information,
It may be configured to be managed in association with the physical address information of the recording device.

With this configuration, the physical address information of the buffer in the kernel on the node device side and the physical address information of the buffer on the external input / output device side are mapped in the same memory space, and these buffers are virtually distributed and shared. Since it is handled as a memory, it is possible to access the recording device without performing complicated pre-processing such as negotiation for the external input / output device by simply accessing the same memory space. It is also possible to reduce the space (claim 7).

Further, the virtual address management means receives the physical address information indicating a specific storage area of the external input / output device side memory from the external input / output device, and thereby the virtual address information is added to the physical address information. Virtual address allocation means for constructing the correspondence between the above-mentioned virtual address information and physical address information by allocation, and virtual address allocation for notifying the external input / output device of the allocation result of the virtual address information by this virtual address allocation means. It may have a result notifying means.

In this way, any physical address information of the external input / output device side storage means can be mapped in the virtual distributed shared memory space, so that the above memory access (storage access) can be flexibly realized. (Claim 8). As the connection means, for example, Infiniband is preferably applied as a high-speed internal network (claim 9).

[0021]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. (A) Description of First Embodiment FIG. 1 is a block diagram showing a configuration of a storage access system according to a first embodiment of the present invention. The system shown in FIG. 1 includes a node device 1 such as a PC and a WS. The external input / output (I / O) device 2 and these node devices 1 (hereinafter,
An IB (Infiniband) network (IB switch 3) as a connecting means for connecting data between the node 1 and the I / O device 2 by simply connecting the I / O device 2 and the I / O device 2 to each other. ) And is configured. The I / O device 2 may be, for example, an external hard disk device, MO, CD-R (R / W), DVD-R.
AM etc. are mentioned.

As shown in FIG. 1, the node 1
Include, for example, the CPU 11 and the main storage unit (main memory;
Node device side storage means) 12, memory controller 13
And host channel adapter 14 are provided,
These components are connected by an internal bus 15 so that they can communicate with each other. Further, the I / O device 2 includes a target channel adapter 21 and a memory controller 2
2, buffer 23, disk controller 24 and disk device (recording device; external input / output device side storage means) 25
(Hereinafter, also simply referred to as “disk 25”), each of these components also has an internal bus 26.
Are connected so that they can communicate with each other.

Here, in the node 1, the CPU 11
Controls the operation of the node 1. For example, by reading the file system data from the main memory 12 and operating it, a file system incorporated as one function of the OS or the like is realized.
In addition, the main memory (hereinafter, also simply referred to as "memory")
Reference numeral 12 is for storing various data such as the above-mentioned file system data, other application data, and user data, and the host channel adapter 1
Reference numeral 4 is for taking an interface with the IB switch 3 (IB network).

Then, the memory controller 13 accesses the memory 12 (writes / reads data),
It controls access (data transfer processing) to the disk 25 on the I / O device 2 side via the host channel adapter 14 and the IB switch 3, and in the present embodiment, the memory 12 and I / O on the node 1 side. Disk 2 on device 2 side
5 is treated as a virtual distributed shared memory, for example, as shown in FIG.
It has a virtual address table 131 as shown in (A), and each access described above is controlled based on the contents of this table 131.

Here, specifically, in the virtual address table 131, as shown in FIG. 2A, a virtual address area 132, a local physical address area 133, and a node address area. 134, Physical / Sector Addr
An ess) area 135 and a flag (Flags) area 136 are prepared. The virtual address area 132 has a virtual address [NUMA (Non-Uniform Memory Access) based virtual address in the memory space accessed by the file system. Address] is stored. The method of assigning the virtual address will be described later.

Further, the local physical address area 133 stores address information in which data cached locally (memory 12) within the own node 1 is stored, and the node address area 134 contains data entity or cache. I of node 1 (or I / O device 2)
B (Infiniband) address information (IB switch 3
Connection location information) is stored. In addition, this IB
The address is also provided with a bit that distinguishes the node where the substance of the data exists from the cache node.

Further, the physical / sector address area 135
Stores the position information (sector address) of the storage area (sector) where the substance of the data exists in the disk 25 on the I / O device 2 side. The flag area 136 stores the locally cached data content or I A flag (for example, "1" is valid, "0" is invalid) for determining whether or not the data cached (written) is valid is stored in the disk 25 of the / O device 2 side. There is.

By equipping the memory controller 13 with the virtual address table 131 having the above information, the address (physical address information) of the memory 12 on the node 1 side and the sector address of the disk 25 on the I / O device 2 side are provided. (Physical address information) is the same memory space 4A (see FIG.
It is possible to handle the memory 12 and the disk 25 as a virtual distributed shared memory.

That is, when the file system accesses (writes / reads data) the same memory space (virtual distributed shared memory space) 4A with a certain virtual address, the memory controller 13 corresponds to the virtual address. Access to the local physical address or physical / sector address to be executed.

For example, the file system may have a memory 12 in the above memory space 4A by a virtual address.
When the memory controller 13 accesses the area where the address of the disk 25 is mapped, the memory controller 13 accesses the memory 12, while when the file system accesses the area where the sector address of the disk 25 is mapped by a certain virtual address, the memory controller 13 13 executes access (data transfer processing) to the disk 25 on the I / O device 2 side via the host channel adapter 14 and the IB switch 3. The specific operation of accessing the disk 25 will be described later.

Next, in the I / O device 2, the target channel adapter 21 serves to interface with the IB switch 3 side (IB network), and the memory controller 22 accesses the buffer 23 (data In the first embodiment, since the disk 25 and the memory 12 on the node 1 side are treated as a virtual distributed shared memory as described above, the virtual address table 131 on the node 1 side is controlled. Virtual address table 221 similar to
(See (B)] is implemented.

However, the local physical address area 223 of the virtual address table 221 on the I / O device 2 side is local to the I / O device 2 (buffer 23).
The storage location (buffer address) of the cached data is stored in. As a result, the buffer address of the data temporarily cached in the buffer 23 (Buffer
Address0, 1, 2 etc.) and the data is actually the disk 25
Address when stored in (Sector0,1,2, etc.)
Will be associated with.

Then, the disk controller 24 transfers the data once stored in the buffer 23 to the disk 2
5 (the sector address at this time is the physical / sector address area 2 of the virtual address table 221).
On the other hand, the data stored in the sector of the disk 25 indicated by the corresponding physical / sector address in the virtual address table 221 when read-accessed from the node 1 side using the virtual address is temporarily buffered by the buffer 23. It has a function of reading out to the node 1 and sending it to the node 1. Also in FIG. 2B, reference numeral 222 represents a virtual address area, and reference numeral 224 represents a note address area.

That is, the memory controller 13 on the side of the node 1 and the memory controller 22 on the side of the I / O device 2 have the virtual address tables 131 and 221.
By implementing, in order to treat the memory 12 and the disk 25 as a virtual distributed shared memory, at least the physical address information (sector address) indicating the virtual address in the memory space 4A and a specific storage area (sector) of the disk 25. And a function as a virtual address management unit 5 that manages the above, and a sector (storage area) and a node of the disk 25 on the I / O device 2 side based on the virtual address managed by the virtual address management unit 5. This means that it functions as a control unit 6 that controls data transfer via the IB switch 3 (IB network) with the memory 12 on the first side.

The operation of the system configured as described above will be described in detail below. Method for assigning virtual address to sector of disk 25 on I / O device 2 side First, when the I / O device 2 is powered on, as shown in FIG. The controller 22 informs the memory controller 13 on the node 1 side of the sector information (may be all sectors or some of the disk 25) to be mapped in the memory space 4A (step S1).

The memory controller 13 on the node 1 side is
A virtual address is assigned to the above sector information received from the memory controller 22 on the I / O device 2 side, and a virtual address table 131 (virtual address area 131 and physical / sector address area 135) in the own memory controller 13 is assigned. ) (Step S2),
The list of assigned virtual addresses is notified to the memory controller 22 on the I / O device 2 side (step S3).

As a result, the memory controller 22 on the I / O device 2 side becomes the memory controller 13 on the node 1 side.
The list of virtual addresses sent from is written in the virtual address table 221 in the own memory controller 22 (step S4). As described above, the I / O device 2
Disk 25 requested by the side (memory controller 22)
Virtual addresses are allocated to the sectors of the above, and virtual address tables 131 and 221 are constructed and held in the memory controllers 13 and 22, respectively.

That is, in this embodiment, the memory controller 13 (virtual address management unit 5) has the following functions. (1) By receiving a specific sector address of the disk 25 from the I / O device 2 and assigning a virtual address to the sector address, a virtual address and a sector address are associated with each other (virtual address table 131). Function as Address Allocation Unit 51 (2) Function as Virtual Address Allocation Result Notification Unit 52 for Notifying the I / O Device 2 (Memory Controller 22) of Virtual Address Allocation Result by the Virtual Address Allocation Unit 51 May be assigned from the I / O device 2 side.

Data transfer (writing) processing to the I / O device 2 (disk 25) Next, as described above, the memory controllers 13 and 2
2, virtual address tables 131 and 22 respectively
4, it is assumed that the file system of the node 1 writes to the area of the virtual distributed shared memory space 4A in which a certain sector of the disk 25 is mapped, as shown in FIG. 4 (step S11). .

Then, the write data is a message for IB (including a virtual address; hereinafter, "IB message") under the control of the memory controller 13.
Host channel adapter 14 and IB
It is sent to the I / O device 2 side via the switch 3 (IB network) (steps S12 and S13). At this time, the memory controller 13 updates the flag area 136 of the virtual address table 131 (stores information indicating that the data is valid).

On the I / O device 2 side, the memory controller 22 temporarily stores the data (IB message) sent from the node 1 as described above in the empty area of the buffer 23 (step S14), and then the disk. The controller 24 is activated, and the disk controller 24 reads the data accumulated in the buffer 23 and writes it on the disk 25 (step S15). When writing is finished,
The memory controller 22 uses the virtual address table 22.
The flag area 226 of No. 1 is updated similarly to the virtual address table 221 on the node 1 side, and the processing is ended.

When the data writing to the disk 25 is completed in this way, the disk controller 24 notifies the memory controller 22 to that effect, as shown in FIG.
Data write completion notification is sent to the memory controller 13 on the side (steps S16 to S18), and the memory controller 13 notifies the file system to that effect and completes the processing (step S19).

Data transfer (reading) processing from the I / O device 2 (disk 25) On the other hand, in the node 1, the file system maps the sector of the disk 25 in the virtual distributed shared memory space 4A for reading data. When the area is accessed by the virtual address (step S21), the memory controller 12 sends an IB message including the virtual address to the memory controller 22 on the I / O device 2 side.

Memory controller 22 on the side of I / O device 2
Uses the virtual address included in the received IB message as a search key to search the virtual address table 221 to identify the sector address of the corresponding disk 25, and notifies the disk controller 24 of the sector address. The disk controller 24 once reads the data stored at the notified sector address into the buffer 23 (step S22), and the memory controller 22 sequentially reads the read data from the buffer 23 to cause the memory controller on the node 1 side. It is sent to 13 (steps S23 and S24). In the node 1, the memory controller 13 writes the received data from the I / O device 2 into the (local) memory 12 and updates the contents of the virtual address table 131 (the value of the flag area 136 is a value meaning no data). (For example, it is updated to "0") and the process ends (steps S25 and S26).

As described above, according to the storage system of this embodiment, the data transfer between the node 1 and the I / O device 2 is completed only by accessing (writing / reading) the memory space 4A. Traditional SCSI
Complex operations such as negotiations before data transfer in are reduced, and a significant improvement in latency can be expected.
Also, due to the significant improvement in latency, the IB switch 3
Data transfer band (rate) by (IB network)
Can be effectively utilized, and the data transfer rate can be significantly improved as compared with the conventional one.

Further, in the above-described example, as described above with reference to FIG. 3, the virtual address table is notified by notifying the node 1 of the sector of the disk 25 to be mapped in the memory space 4A from the I / O device 2 side. 131,221
Is constructed, any sector of the disk 25 can be mapped to the memory space 4A. Therefore, the sectors of the disk 25 can be wholly or partially mapped in the memory space 4A as required, and flexible storage access can be realized.

(A1) Description of First Modification of First Embodiment As shown in FIG. 7, for example, a plurality of (two or more) nodes 1 are connected to each other via an IB switch 3 (IB network). May be. In this case, the address of the memory 12 of each node 1 and the disk 25 (in FIG. 7, other components on the I / O device 2 side are not shown (FIG. 9, FIG. 12, FIG. 13, FIG. 15 to FIG. 20)] are allocated to the same memory space 4B (mapping) to treat the memory 12 of each node 1 and the disk 25 common to each node 1 as a virtual distributed shared memory. Will be possible. It should be noted that, in the following, components having the same reference numerals as those described above are the same as those described above unless otherwise specified.

To construct such a memory space 4B, the virtual address table 13 as shown in FIG.
1, the memory controller 22 on the I / O device 2 side
For example, a virtual address table 221 as shown in FIG. That is, in this case, the virtual address management units 5 of the plurality of nodes 1 have the same I
The sector address (physical address information) of the disk 25 on the side of the / O device 2 is managed in association with each virtual address in the memory space 4B.

As a result, in each node 1, the file system has an area in which the sectors of the disk 25 of the I / O device 2 are mapped in the same memory space 4B [FIG. 8].
In the case of (A), the virtual address "100" or later] is simply accessed, and the control unit 6 (memory controller 13,
22) enables shared access to the same disk 25.

However, in this case, since read / write instructions for the same memory space 4B may be issued from a plurality of nodes 1 at the same time, exclusive control is required. This is, for example, "atomic" of IB. It can be realized by using "operation" or the like. Excluding this exclusive control, the access method from each node 1 to the disk 25 is the same access method as in the first embodiment described above.

Therefore, for example, even if some of the nodes 1 are down, it is possible to switch to another node 1 (standby system) at high speed, so that the reliability of the entire system is greatly improved. You can do it. (A2) Description of Second Modification of First Embodiment Next, as shown in FIG. 9, for example, the above-mentioned I / O devices 2 are provided to a plurality of nodes 1 via an IB switch 3 (IB network). It is also possible to connect the units. In this case, the address of the memory 12 of the node 1 and a plurality of I / O
By mapping the sector address of the disk 25 of the device 2 in the same memory space 4C, the memory 12 and the plurality of disks 25 can be treated as a virtual distributed shared memory.

For example, the memory controller 13 on the node 1 side is provided with a virtual address table 131 as shown in FIG. 10A, and the memory controller 22 on the I / O device 2 side is shown as FIG. 10B, respectively. By providing such a virtual address table 221, the above memory space 4C is constructed. Here, these FIG.
As shown in FIGS. 10 (A) and 10 (B), if the sectors of the plurality of disks 25 are sequentially (continuously) assigned to the virtual addresses in the memory space 4C one by one, the file system becomes continuous. If the area in which the sectors of the disk 25 are mapped in the memory space 4C by the virtual address to be accessed (in the case of FIG. 10A, the virtual address "100" and thereafter) is continuously accessed, the control unit 6
As a result, access (data transfer) to the plurality of disks 25 occurs continuously.

That is, in this case, the virtual address management unit 5
However, the sector addresses of the disks 25 of different I / O devices 2 are managed in association with the continuous virtual addresses in the memory space 4C. This allows
So-called “striping” for the plurality of disks 25 is realized. Therefore, a data transfer band can be earned, and a high-speed data transfer process can be realized by making the most effective use of the band of the IB switch 3 (IB network).

(A3) Description of Third Modified Example of First Embodiment When a plurality of I / O devices 2 are provided as described above, for example, as shown in FIG.
5 can also be assigned as a disk for storing data for checking the parity of data recorded on another disk (parity data). In the following, for convenience of description, the data recording disk 25a, the parity data recording disk 25b, the I / O device 2 having the disk 25a is the I / O device 2a, and the disk 25b is the I / O device. 2 is referred to as an I / O device 2b.

In this case, when the file system of the node 1 accesses (writes data) the mapped area of the sector of the disk 25a in the memory space 4C, the memory controller 13 'writes the data to the corresponding sector. Create an IB message (including virtual address and write data) for writing and I /
In addition to sending to the O device 2a, a parity operation is performed on the above-mentioned write data, an IB message including the operation result (parity data) is created, and sent to the I / O device 2b.

On the side of the I / O device 2a (2b), the memory controller 22 and the disk controller 24 are
The write data [parity data (hereinafter, also simply referred to as “parity”)] received as a message is the I
The data is written in the corresponding sector of the disk 25a (25b) specified by the virtual address included in the B message. That is, as shown in FIG. 11, the memory controller 13 ′ (control unit 6) of the present modification generates parity data for data to be transferred and written to the sector of the disk 25 on the I / O device 2 side. Data generator 61
And a function as a parity transfer unit 62 for transferring the parity data generated by the parity data generation unit 61 so as to be written in the specific disk 25.

On the other hand, in the case of data reading, the node 1
By writing the virtual address to which the parity is written in the memory controller 13 'on the side, the write address of the parity (virtual address) corresponding to the virtual address to which the file system has read-accessed the memory space 4C. Since the IB message containing the virtual address is sent to the I / O device 2b, the parity for the data read from the disk 25a can be read from the disk 25b.

As a result, the memory controller 13 'on the node 1 side performs a parity check on the data read from the disk 25a and the parity read from the disk 25b corresponding to the data, and as a result (normal / Error) will be notified to the file system together with the read data. That is, in the system of this modification, the memory controller 1 is based on the architecture of the embodiment described above with reference to FIGS.
By implementing 3 ', so-called RAID (Redundant
Array of Independent Disks) -3,4 storage architecture with parity check function has been realized. Therefore, in addition to the same advantages as those of the above-described embodiment, a data error can be found by a parity check, so that higher data reliability and system safety (processing based on error data can be performed). It is possible to secure the prevention)

As shown in FIG. 12, there are a plurality of I / O devices 2 (disks 25) for writing data, and in order to write the same data to those I / O devices 2, the memory controller 13 'is used. Request to write data I
The B message may be sent to each I / O device 2. Further, the disks 25 for writing the parity do not have to be fixedly determined in advance, and may be distributed and written in a plurality of disks 25 like so-called RAID-5. Furthermore, specific disk 25 such as RAID-4
To correct errors such as garbled bits.
ror-Correcting Code) can also be used for recording.

(A4) Description of Fourth Modification of First Embodiment Next, as described above with reference to FIG. 9, a plurality of I / O devices 2 are used.
In the configuration in which the (disk 25) is connected to the IB switch 3 (IB network), even if sectors of a plurality of disks 25 are mapped to one area (virtual address) of the memory space 4C as shown in FIG. 13, for example. Good.

That is, the memory controller 13 on the node 1 (Node Address = 0) side is provided with a virtual address table 131 as shown in FIG.
By providing the memory controller 22 (not shown in FIG. 13) on the O device 2 side with a virtual address table 221 as shown in FIG. 14B, for example,
Multiple disks 25 in one area (same virtual address)
The memory space 4C is constructed by mapping the sectors of.
That is, in this case, in the virtual address management unit 5,
A plurality of sector addresses of the disks 25 of different I / O devices 2 are managed in association with the same virtual address.

As a result, for example, if the file system of the node 1 accesses the area (for example, the area indicated by the virtual address "100") of the disk 25 in the memory space 4C to which the sector is mapped, the memory controller 13 , Virtual address table 1 shown in FIG.
A plurality of I's corresponding to the virtual address “100” in 31
Since the sector address (Sector0,1) of the disk 25 in the I / O device 2 (Node Address = # 0, # 1) is assigned, an IB message is sent to the memory controller 22 on each I / O device 2 side. Will be sent.

On the other hand, when the memory controller 22 on the I / O device 2 side receives the IB message, the virtual address ("1" included in the IB message is received.
00 ”) as a search key and the virtual address table 221
Is searched and the sector of the disk 25 corresponding to the own node address is accessed. In this way
The file system on the node 1 side has a memory space 4
When the sector of the disk 25 of C is accessed (data writing / reading), the control unit 6 causes the plurality of disks 25 of the same data to be accessed.
Access occurs simultaneously.

That is, in this modification, so-called "mirroring" is realized, in which the stored data is duplicated, based on the storage architecture of the embodiment described above with reference to FIGS. Therefore, even if one of the disks 25 is destroyed, the data can be read from the other disk 25, and the same effects and advantages as the above-described embodiment can be obtained.
The additional advantage is that the security of data storage is greatly improved.

(B) Description of Second Embodiment In the first embodiment described above, the memory 12 on the node 1 side and the disk 25 on the I / O device 2 side are handled as distributed shared memories. As shown in 15,
O device 2 side device (hard disk, CD-R (R
/ W), DVD-RAM, etc.) buffer 23 for 25 '
And a buffer area (kernel layer internal buffer area) 161 in a device (I / O) driver (hereinafter, simply referred to as “driver”) 16 for the device 25 ′ as in the first embodiment. It can also be handled as shared memory. Note that in FIG. 15, the CP of the node 1
Illustration of other components such as the U11 and the memory 12 is omitted (the same applies to FIGS. 16 to 20).

In this case, the kernel layer buffer area 1
A buffer address of 61 (hereinafter simply referred to as "buffer 161") (may be part or all) and a buffer address of the buffer 23 for the device 25 'on the I / O device 2 side (may be part or all). ) And the same memory space 4D that the driver 16 accesses in the kernel layer (in the virtual address management unit 5, the physical address information of the buffer 161 and the physical address information of the buffer 23 on the I / O device 2 side are set as virtual addresses). It may be managed in association with each other.

That is, in this case, the driver 16 on the node 1 side manages addresses up to the buffer 23 on the I / O device 2 side. Although not shown in FIG. 15, the address management in this case is also performed by providing the virtual address tables 131 and 221 described above in the first embodiment on the node 1 side and the I / O device 2 side. Will be realized. This point is the same in each of the following modifications.

As a result, the driver 16 on the node 1 side
However, the data transfer between the buffer 23 for the device 25 'and the driver 16 is completed only by accessing the area in the memory space 4D where the buffer address of the buffer 23 on the I / O device 2 side is mapped. Similar to the embodiment, complicated operations such as negotiation before data transfer in the conventional SCSI are reduced, and a significant improvement in latency can be expected. Further, as a result, the data transfer band (rate) by the IB switch 3 (IB network) can be effectively utilized, and higher-speed data transfer than conventional can be realized.

(B1) Description of First Modified Example of Second Embodiment Although the buffer 23 and the device 25 'are not associated with each other in the above-described second embodiment, as shown in FIG. 16, for example. If the device 25 'is the disk 25, the table 27 which manages the buffer address of the buffer 23 and the sector address of the disk 25 in association with each other is also made accessible on the memory space 4D, so that the memory space 4D and the disk Data transfer to and from 25 can also be realized. The table 27 is the virtual address tables 131 and 221 described in the first embodiment.
Corresponds to an expanded (deformed) version of the above, and the same applies to subsequent modified examples.

That is, in this case, for example, the command area 271 is provided in the table 27, and the driver 16 on the node 1 side writes an access command (write command / read command) in the command area 271. Data transfer between the buffer 23 and the disk 25 can be instructed. For example, when writing to the memory space 4D, first check the table 27 and check the memory space 4D (buffer 23).
Check if there is a free area (buffer address). If there is a free area, a write command is written in the command area 271 corresponding to the address to write the data in the free area.

If there is no free area, the data cached in the buffer 23 (old data, infrequently used data, etc.) is flushed to the disk 25, and a free area is created in the buffer 23. alike,
Data writing is executed by writing a write command in the command area 271 corresponding to the empty area.

On the other hand, when reading data from the memory space 4D, first, the table 27 is checked to see if there is a free area (buffer address) in the buffer 23. If there is a free area, that address is checked. By writing a read command in the command area 271 corresponding to, the data is read from the corresponding sector of the disk 25 into the empty area.

If there is no free area, in this case as well,
The data cached in the buffer 23 (old data, infrequently used data, etc.) is flushed to the disk 25, a free area is created in the buffer 23, and then a read command is written in the command area 271 corresponding to the free area. By doing so, the reading of the data into the empty area is executed.

As described above, in this modification, the memory space 4
Just write an access command to the command area 271 of the table 27 on D, and the memory space 4D and the disk 2
Since the data transfer with the 5 is completed, the conventional SCSI
The complicated operations such as negotiation before data transfer in are reduced, and a significant improvement in latency can be expected. Therefore, also in this case, the data transfer band (rate) by the IB switch 3 (IB network) can be effectively utilized, and data transfer at a higher speed than in the past can be realized.

(B2) Description of Second Modified Example of Second Embodiment In the first modified example described above, there is one node 1, but of course, as shown in FIG. Of I
Even when a plurality of nodes 1 are connected to the I / O device 2 through the IB switch 3, the plurality of nodes 1 are connected as described above.
Each of the buffers 161 and the buffer 23 on the I / O device 2 side can be treated as a virtual distributed shared memory.

That is, in this case, a plurality of buffers 161
Buffer address (may be part or all) and I /
The buffer address (may be part or all) of the buffer 23 on the O device 2 side is mapped to the same memory space 4E that the driver 16 accesses in the kernel layer. As a result, the driver 16 causes the memory space 4
By simply accessing E, access to each buffer 161 and 23 occurs via the IB switch 3 (IB network), and data transfer between desired buffers is completed. In this case as well, before data transfer in the conventional SCSI. The complicated operations such as the negotiation of can be reduced, and the latency can be expected to be significantly improved. As a result, the data transfer band (rate) by the IB switch 3 (IB network) can be effectively used, which is more effective than before. High-speed data transfer can be realized.

Also in this modification, the above-mentioned second
Similar to the first modified example of the embodiment, the buffers 23 and 16
If a table 271 that manages the buffer address of 1 and the sector of the disk 25 in association with each other is provided in the memory space 4E, data transfer between the memory space 4E and the disk 25 becomes possible. (B3) Description of Third Modified Example of Second Embodiment Contrary to the second modified example of the second embodiment, as shown in FIG. 18, for example, as shown in FIG. Even when the I / O device 2 is connected via the IB switch 3 (IB network), similarly to the above, the buffer 161 of the node 1 and the buffers 23 on the side of the plurality of I / O devices 2 are provided.
And can be treated as virtual distributed shared memory.

That is, in this case, the buffer address of the buffer 161 (may be part or all) and the buffer addresses of the plurality of buffers 23 (may be part or all).
And are mapped to the same memory space 4F accessed by the driver 16 in the kernel layer. As a result, the driver 16 only accesses the memory space 4E to access the buffers 161 and 23 via the IB switch 3 (IB network), and the data transfer between the desired buffers is completed. However, complicated operations such as negotiation before data transfer in the conventional SCSI can be reduced, and a significant improvement in latency can be expected. As a result, the IB switch 3 (IB
It is possible to effectively utilize the data transfer band (rate) by the network, and it is possible to realize higher-speed data transfer than before.

Also in this modification, for example, FIG.
As shown in FIG.
The table 27 for managing the buffer address of 3 and the sector of the disk 25 in association with each other is stored in the memory space 4F
In this case, data can be transferred between the memory space 4F and the disk 25. (B4) Description of Fourth Modification of Second Embodiment In the configuration shown in FIG. 18, buffer addresses of a plurality of buffers 23 are assigned to the same virtual address in the memory space 4F as shown in FIG. 20, for example. If so, the so-called “mirroring” is realized for the plurality of buffers 23, as in the fourth modified example of the first embodiment. Therefore, even if one of the buffers 23 is destroyed, it is possible to read the data from the other buffer 23, and in this case as well, there is an advantage that the safety of data storage is significantly improved. .

Also in this modification, each buffer 2
Table 27 that manages the buffer address of the buffer 23 and the sector of the disk 25 in association with each other
If 1 is provided in each of the memory spaces 4F, data transfer between the memory space 4F and the disk 25 is also possible. (C) Others In each of the above-described embodiments and modified examples, one node 1 or I / O device 2 has the memory 1
Although the case where there is only one disk 2 or one disk 25 has been described, it goes without saying that the memory 12 or disk 2 is used.
Needless to say, even when there are a plurality of 5, if the respective addresses (storage areas) are mapped in the same memory space, the same operational effects as those of the above-described respective embodiments can be obtained.

The present invention is not limited to the above-described embodiments and modified examples, and can be variously modified and implemented without departing from the spirit of the present invention.

[0082]

As described in detail above, according to the present invention,
Since the data transfer via the connection means between the node device and the external input / output device is completed only by accessing (writing / reading data) to the memory space (virtual distributed shared memory), the negotiation before the data transfer in the conventional SCSI is completed. It can be expected that the latency will be greatly improved by reducing complicated operations such as. Also, due to the significant improvement in latency, it is possible to effectively utilize the data transfer band (rate) by the connection means (high-speed internal network),
The data transfer rate can be greatly improved as compared with the conventional one.

Further, the above-mentioned memory space (virtual distributed shared memory) is constructed by notifying the node device of the physical address information of the external input / output device side storage means that the external input / output device side wants to map to the memory space. Because it is done
Any physical address information of the external input / output device side storage means can be mapped in the memory space. Therefore, it is possible to map the physical address information of the external input / output device side storage means to the memory space in whole or in part as needed, and it is possible to realize flexible storage access.

Further, when a plurality of node side devices are connected to the connection means, the physical address information of the same external input / output device side storage means is managed in association with each virtual address information in the same memory space. Due to
Since shared access to the same external input / output device side storage means is possible, for example, even if some node side devices are down, switching to another node device (standby system) can be performed at high speed. Therefore, the reliability of the entire system can be greatly improved.

Further, when a plurality of external input / output device side storage means are connected to the connecting means, the physical address information of different external input / output device side storage means respectively correspond to consecutive virtual addresses in the memory space. By adding and managing it, so-called "striping" can be realized for the storage means on the external input / output device side, so that the data transfer bandwidth through the connection means can be earned and the bandwidth of the connection means can be maximized. High-speed data transfer processing that is effectively used is realized.

Further, in this case, a storage architecture with a parity check function, which is so-called RAID-4,5, can be realized, so that a data error can be found by the parity check, and higher data reliability can be obtained. , It is possible to secure the safety of the system (prevention of the progress of processing based on error data).

Further, if a plurality of physical address information of different external input / output device side storage means are managed in association with the same virtual address information, access to a plurality of external input / output device side storage means for the same data occurs simultaneously. In other words, the so-called “mirroring” is realized, and the advantage that the safety of data storage is significantly improved is further obtained.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a storage access system as a first embodiment of the present invention.

2A is an example of a virtual address table on the node side shown in FIG. 1, and FIG. 2B is a diagram showing an example of a virtual address table on the I / O device side shown in FIG.

FIG. 3 is a diagram for explaining a procedure for constructing a virtual address table shown in FIGS. 2 (A) and 2 (B).

FIG. 4 is a diagram for explaining an operation (a data writing procedure) of the system shown in FIG.

5 is a sequence diagram for explaining an operation (data writing procedure) of the system shown in FIG.

FIG. 6 is a diagram for explaining an operation (a data reading procedure) of the system shown in FIG.

FIG. 7 is a block diagram showing a first modification of the first embodiment.

8A shows an example of a virtual address table on the node side shown in FIG. 7, and FIG. 8B shows an example of a virtual address table on the I / O device side shown in FIG.

FIG. 9 is a block diagram showing a second modification of the first embodiment.

10A is an example of a virtual address table on the node side shown in FIG. 9, and FIG. 10B is an example of a virtual address table on the I / O device side shown in FIG. 9;

FIG. 11 is a block diagram showing a third modified example of the first embodiment.

12 is a block diagram showing another configuration of the system shown in FIG.

FIG. 13 is a block diagram showing a fourth modified example of the first embodiment.

14A is a diagram showing an example of a virtual address table on the node side shown in FIG. 13, and FIG. 14B is a diagram showing an example of a virtual address table on the I / O device side shown in FIG.

FIG. 15 is a block diagram showing a configuration of a storage access system as a second exemplary embodiment of the present invention.

FIG. 16 is a block diagram showing a first modification of the second embodiment.

FIG. 17 is a block diagram showing a second modification of the second embodiment.

FIG. 18 is a block diagram showing a third modified example of the second embodiment.

19 is a block diagram showing another configuration of the system shown in FIG.

FIG. 20 is a block diagram showing a fourth modified example of the second embodiment.

FIG. 21 is a block diagram showing an example of an existing storage access system.

22 is a sequence diagram for explaining an operation (a data writing procedure) of the system shown in FIG.

[Explanation of symbols]

1 Node Device 2 External Input / Output (I / O) Device 3 Infiniband (IB) Switch 4A-4F Memory Space 5 Virtual Address Management Unit 6 Control Unit 11 CPU 12 Main Storage Unit (Main Memory; Node Device Side Storage Means) 13 , 13 ', 22 memory controller 14 host channel adapter 15, 26 internal bus 16 device (I / O) driver 21 target channel adapter 23 buffer 24 disk controller 25 disk (external input / output device side storage means) 25' device 27 table 51 Virtual address allocation unit 52 Virtual address allocation result notification unit 61 Parity data generation unit 62 Parity transfer unit 131,221 Virtual address table 132,222 Virtual address area 133,223 Local physical address area 134,22 4 Node address area 135, 225 Physical / sector address (Physical / Sec)
tor Address area 136, 226 Flags area 161 Kernel layer buffer area 271 Command area

Claims (9)

[Claims] 1. A node device having a storage means, an external input / output device having a storage means, and a connection means for connecting the node device and the external input / output device to enable data transfer between the storage means. And a virtual address management means for managing at least virtual address information and physical address information indicating a specific storage area of the external input / output device side storage means in association with each other so as to treat each of the storage means as a virtual distributed shared memory. And controlling the data transfer between the storage area of the external input / output device side storage means and the node device side storage means via the connection means based on the virtual address information managed by the virtual address management means. A storage access system characterized by having a control means. 2. A plurality of the node devices are connected via the connecting means, and each virtual address management means of the plurality of node devices respectively stores the physical address information of the external input / output device side storage means. The storage access system according to claim 1, wherein the storage access system is configured to be managed in association with the virtual address information. 3. The plurality of external input / output devices are connected via the connecting means, and the virtual address management means of the node device respectively stores the physical address information of the plurality of external input / output device side storage means. The storage access system according to claim 1, wherein the storage access system is configured to be managed in association with the virtual address information. 4. The virtual address management means is configured to manage the physical address information of different external input / output device side storage means in association with each successive virtual address information. The storage access system according to claim 3. 5. The parity data generation unit for generating parity data for data to be transferred to and written in the storage area of the external input / output device-side storage unit, the control unit being generated by the parity data generation unit. 4. The storage access system according to claim 3, further comprising a parity transfer unit that transfers the parity data so as to write the parity data into a storage unit on a specific external input / output device side. 6. The virtual address management means is configured to manage a plurality of physical address information of different external input / output device side storage means in association with the same virtual address information. Item 3. The storage access system according to item 3. 7. The external input / output device side storage means is configured to have an external input / output device side buffer for temporarily buffering data transferred to and from the node device, and the virtual address. The management means is configured to manage the physical address information of the kernel layer buffer as the node device side storage means and the physical address information of the external input / output device side buffer respectively in association with the virtual address information. Characterized by that
The storage access system according to claim 1. 8. The virtual address management means receives physical address information indicating a specific storage area of the external input / output device side storage means from the external input / output device, and the virtual address information is added to the physical address information. A virtual address assigning means for constructing a correspondence between the virtual address information and the physical address information by assigning the virtual address information, and a virtual address for notifying the external input / output device of the assignment result of the virtual address information by the virtual address assigning means. The storage access system according to claim 1, further comprising allocation result notifying means. 9. The storage access system according to claim 1, wherein the connecting means is a high-speed internal network.