JP2003067350A - Processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform fast communication among a plurality of processor elements. SOLUTION: An address space of a microprocessor 305 of processing elements 301-304 adopts a memory mapped I/O system and deals with an address space of a main memory and an I/O address space unitiedly. The address space consists of the main memory address space 401 and the I/O address space 402. A broadcast area 403, a first local port area 404, a second local port area 405, a third local port area 406, a fourth local port area 407, a remote port window area 408 and other I/O device area 409 are allocated in the I/O address space 402. The remote port window area 408 receives data from an external system.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a processing system having a plurality of processing elements, and more particularly to a communication system capable of performing high-speed communication between a plurality of processing elements using an external memory.

[0002]

2. Description of the Related Art In recent years, so-called massively parallel clusters in which a large number of workstations are connected by a network are being put to practical use. However, in such a cluster, the communication speed of the network is slow, and this network has been a bottleneck for high-speed processing. Ethernet (registered trademark), which is widely used at present, is used for many networks, and its speed is 100 Mbps (Mbi) at most.
t / sec) or about 1 Gbps. 100M
When a bps Ethernet device is used, a system can be constructed at a relatively low cost, but a sufficient speed cannot be realized. On the other hand, when 1 Gbps Ethernet equipment is used, the speed can be relatively increased, but the system becomes very expensive. Whichever is used, since a large number of workstations are arranged side by side, the problems of installation area, weight and power consumption cannot be ignored.

[0003]

SUMMARY OF THE INVENTION The present invention has been made in consideration of the above circumstances, and an object thereof is to connect a plurality of computers at low cost and at high speed.

[0004]

According to the present invention, in order to achieve the above-mentioned object, the structure as described in the claims is adopted. That is, according to one aspect of the present invention, a memory space is allocated to a plurality of processing elements and an I / O address area of a memory-mapped I / O system address space of each of the plurality of processing elements in the processing system. And an external memory.

In this configuration, since the external memory can be accessed independently of the memory management of each processing element, the external memory can be written without considering the memory coherency of each processing element. In a network, data is often sent in a state where the processing element does not detect it. This configuration is suitable for such a communication process.

According to another aspect of the present invention, the processing system is provided with a plurality of processing elements and a plurality of external memories respectively accessed by the corresponding pairs of the plurality of processing elements. A plurality of address areas respectively corresponding to a plurality of destination processing elements are assigned to each address space of the processing element, and the plurality of address areas and the corresponding external memory are
The processing elements are associated with each other. In this way, the network space is directly mapped to the external memory.

The allocation of the plurality of address areas in the processing element may be common to the processing elements.

In this configuration, the processing elements (communication nodes) have a common address map,
The address area of each processing element (destination communication node) is provided in this address map. When each processing element communicates with the destination processing element, the destination ID is used to specify the corresponding address area. That is, the device driver of the processing element acquires the address area corresponding to the destination ID as a resource and prepares for writing / reading to / from the external memory. Then, the corresponding external memory is selected according to the destination ID (address area). If the processing elements are different, different external memories are selected even with the same destination ID. For example, when there are processing elements 1, 2, and 3, the external memory is 1-2 (accessed from a pair of the processing elements 1 and 2. The external memories 1-3 and 2-3 have the same meaning. Yes, there are 1-3, 2-3. Processing element 1 is processing element 2
The external memory 1-2 is used when communication is made with the destination as the destination, and the external memory 2-3 is used when the processing element 3 is made as the destination with the processing element 2 as the destination. In this configuration, the destination ID and the external memory are associated with each other for each processing element.

With this configuration, even if the processing elements are different, the destination can be designated in common at the level of the software executed on the processing elements.

Further, the processing elements may operate independently. Further, one processor may dispatch a command or the like to another processor to establish a master-slave relationship.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below.

FIG. 1 shows an overall processing system 100 according to an embodiment of the present invention. In this figure, the processing system 100 is a network device 200 and a plurality of processing elements 301 to 301.
It is configured to include 04. As shown in FIG. 2, the processing elements 301 to 304 include a microprocessor 305, an external cache memory 306, a main memory 307, and a system logic 308.

The network device 200 is mounted on one chip as a whole, and is arranged on a board on which the processing system 100 is mounted. The network device 200 and the processing elements 301 to 304 are connected to the board of the processing system 100 via a connector.

In FIG. 1, the network device 200
Is a broadcast memory 201, remote communication memories 202-205, peer-to-peer memory 206-
211, local ports 212 to 215, an input remote port 216a, and an output remote port 216b. Processing elements 301 to 304 are connected to the local ports 212 to 215. An external system (not shown; can be configured with other similar processing systems or other types of communication systems) is connected to the remote ports 216a, 216b. The input remote port 216a is used to write data from the external system into the external communication memories 202 to 205. Output remote port 216b
Is used to send data from the processing elements 301-304 to an external system.

In this embodiment, the basic processing between the processing system 100 and the external system is communication processing (data transmission / reception). This communication process is executed by writing data from one system to the memory of the other system and reading the data by the processing element of the one system. If one system wants to send a reply to the other system, one system writes to the other system's memory. When receiving, the receiving system does not need to read from the memory of the other system. Of course, reading may be performed from the memory of another system for the purpose of acquiring management information.

The local ports 212 to 215 connect the processing elements 301 to 304 to the network device 2.
Bind to 00. A connector (not shown) for the input remote port 216a and the output remote port 216b
The external system is detachably connected by.

The broadcast memory 201 can be read and written by the communication nodes connected to the local ports 212 to 215, that is, the processing elements 301 to 304. Only writing is performed from the input remote port 216a to the broadcast memory 201. Local port 212-
The data written from the input port 215 and the input remote port 216a is broadcast to all the local ports 212 to 215 (processing elements 310 to 304).

For the remote communication memory 202,
Writing is performed from the input remote port 216a,
Local port 212 (processing element 30
Reading is performed from 1). In this way, the communication from the input remote port 216a to the local port 212 is performed by the remote communication memory 202. The remote communication memories 203, 204, 205 are similarly connected, and the input remote port 216a to the local ports 213, 214, 215 are connected respectively.
Is communicated to.

The peer-to-peer memory 206 is adapted to read and write from the local ports 212 and 213 (processing elements 301 and 302). In this way peer-to-peer memory 2
06, the local ports 212 and 213 can communicate with each other. Peer-to-peer memory 207, 20
8, 209, 210 and 211 are similarly connected. And peer-to-peer memories 207, 208, 209, 2
10, 211 between the local ports 212 and 214 (processing elements 301 and 303) and the local ports 213 and 214 (processing element 3).
02, 303) between local ports 213, 215
Communication is performed between the (processing elements 302, 304) and the local ports 214, 215 (processing elements 303, 304).

FIG. 3 shows the processing element 301.
~ 304 show the address space of the microprocessor 305. This address space is memory mapped I / O
The system adopts the system, and the address space of the main memory and the I / O address space are handled in a unified manner. In FIG. 3, the address space is composed of a main memory address space 401 and an I / O address space 402, and in the I / O address space 402, a broadcast area 403, a first local port area 404, a second local port area 405,
A third local port area 406, a fourth local port area 407, a remote port window area 408 and another I / O device area 409 are allocated. The remote port window area 408 is for receiving data from an external system. Instead of the remote port window 408, an access space may be assigned to each of the processing elements 301 to 304.

FIG. 4 shows the processing element 301.
, 304 to the broadcast area 40.
3, the first to fourth local port areas 404 to 407 and the remote port window area 408 are shown, which resources are selected. This selection is made by the addressing mechanism of system logic 308 (FIG. 3).

As shown in FIG. 4, in any of the processing elements 301 to 304, the broadcast memory 201 is selected when the broadcast area 403 is addressed. The device driver of the processing elements 301 to 304 manages which address of the broadcast memory 201 is accessed. The same applies to the other memories 202 to 211.

On the other hand, the first to fourth local port areas 40
4 to 407 are processing elements 301 to 304
Different memories 206 to 211 are selected for each. For example, in the processing element 301, the second
Local port area 405, third local port area 4
06 and the fourth local port area 407 by addressing the peer-to-peer memories 206, 207, respectively.
208 is selected. Processing element 302
In the first local port area 404, the third local port area 406, and the fourth local port area 407.
The peer-to-peer memories 206, 209 and 210 are respectively selected by the addressing of. In the processing element 303, the first local port area 404, the second local port area 405, and the fourth local port area 407 are addressed by peers, respectively.
Two-peer memories 207, 209, 211 are selected. In the processing element 304, the first
Local port area 404, second local port area 4
05, by specifying the address of the third local port area 406, the peer-to-peer memories 208, 210, respectively.
211 is selected.

The remote port window 408 also selects different remote communication memories 202-205 for the respective processing elements 301-304. That is,
In each of the processing elements 301 to 304, the remote communication memories 202 to 205 are selected by addressing the remote port window 408. The transmission to the external system is executed by the I / O access procedure via the output remote port 216b.

Note that, as indicated by "x" in FIG. 4, in the processing element 301, the first local port area 404 is addressed to itself and is not used. Similarly, the processing elements 302, 30
In 3 and 304, the second local port area 405, the third local port area 406, and the fourth local port area 407 are not used, respectively.

In this way, the broadcast memory 20
1. Since the respective address spaces of the remote communication memories 202 to 205 and the peer-to-peer memories 206 to 211 are assigned to the I / O address space 402, the memory management mechanism of the microprocessor 305 of the processing elements 301 to 304 can be used. It is possible to write to those memories independently. Therefore, the data can be transmitted regardless of the memory management mechanism, that is, the memory 20.
1 to 211 can be written, and then the related microprocessor 305 stores the memory 20 in the main memory 307.
Read in to read data from 1-211. However, this operation is usually DMed to avoid overhead.
Perform at A.

Further, the processing element 301
To 304, a common address map is adopted, and access to other nodes can be designated by the same address, so that the software specifications of the processing elements 301 to 304 can be made common.

Even if a complicated procedure between communication nodes, for example, a handshake procedure is performed using a destination ID, it is easy to write, read, and update data in the peer-to-peer memories 206 to 211 and the broadcast memory 201. Can be done.

FIG. 5 shows a more specific structure of the network device 200 of FIG. 5, the parts corresponding to those in FIG. 1 are designated by the corresponding reference numerals. In FIG.
Broadcast memory 201 is a single write port
It consists of an SRAM (Static Random Access Memory) with a 5-read port configuration, to which 64-bit data lines and 32-bit address / command lines are connected. Remote communication memories 202-20
5. The peer-to-peer memories 206 to 211 are composed of dual port SRAMs. The input remote port 216a (see FIG. 1) and the processing elements 301, 302, ... Are interfaces 216c, 212.
are connected to the data line and the address / command line via a, 213a. Interface 216
Reference numerals c, 212a, 213a ... Buffer the clocks and the like.

Arbitration is performed using the arbiter 220 for writing in the broadcast memory 201. 221 and 222 are bus control logics. An output drive circuit 223 forms the output remote port 216b.

Instead of using a normal multi-port memory as shown in FIG. 5, a plurality of memory planes 231 to 234 and corresponding sense amplifier arrays 235 to 238 as shown in FIG. 231-
Data may be written in duplicate in 234.
Although four memory planes 231 to 234 are shown in FIG. 6, five memory planes are required when used as the broadcast memory 201 of this embodiment, and remote communication memories 202 to 205 and peer-to-peer memories are required. When applied to 206 to 211, two memory planes are required.

In the configuration of FIG. 6, the sense amplifier array 23
The same data is provided to 5-238 at substantially the same time. The bottleneck caused by the bus with the normal memory configuration is eliminated.

According to the embodiment described above, the broadcast memory 201 is used to make local ports 212 to 21.
Processing elements 301 to 30 connected to
4 and remote ports 216a and 216b, mutual communication can be easily performed with a system or a communication node. In addition, corresponding one-to-one communication can be easily performed using the remote communication memories 202 to 205 and the peer-to-peer memories 206 to 211. Moreover, each of the processing elements 301 to 304 can directly perform corresponding communication with the actual memory address.

The memories 201 to 211 are SRAs.
Instead of M, another semiconductor memory such as DRAM (dynamic random access memory) can be used.

FIG. 7 shows a composite communication system in which a plurality of processing systems 100 of the above-mentioned embodiment are connected. In FIG. 7, the composite communication system 500 includes a plurality of processing systems 100, a switch 600, and a master I / O manager 700. As shown in FIG. 8, the switch 600 includes a switch matrix 601 and a system I / O bus 602, and an output port 603 and an input port 604 are connected to the corresponding processing system 100. The switching elements 605 are switched based on the control data of the system I / O bus 602, and desired processing systems 100 are connected to each other.

FIG. 9 shows a connection state with external I / O devices. A master I / O manager 700 is provided between the system I / O bus 602 and expansion bus 705 to manage the interface with external devices. It is supposed to do. In this example, the expansion bus 705 has a secondary storage device 701, a network interface card 702, and a video card 70.
3. A pointing device 704 is connected.

As described above, according to this embodiment,
A communication node such as a processing element can directly communicate via the memory. Moreover, the communication mode is a broadcast mode in which communication is collectively performed with other communication nodes, or a peer that connects communication nodes in a one-to-one relationship.
Two-peer mode is available and can be adapted to various communication applications.

In the above example, the peer-to-peer mode is used as another mode of broadcasting, but 3
One communication node may communicate with each other. That is, any subset of components may be connected to one memory so that they can communicate with each other.

In the above embodiment, the remote port 21
6a is connected to the broadcast memory 210, but as shown in FIG.
a, an output remote port 217b may be provided, and the processing elements 310 to 304 may be connected only in a one-to-one manner via the remote communication memories 202a to 205a. That is, the remote port 217
a is not connected to the broadcast memory 201.
Of course, the remote ports 216a, 216b, 217
a, 217b and remote communication memories 202 to 20
5, 202a to 05a may be provided together. Of course, three or more sets of remote ports may be provided.

In the above example, as shown in FIG.
A physical address space common to the processing elements 301 to 304 is adopted,
Although a common interface is provided for each software of 1 to 304, FIG.
As shown in (b), the processing element 301
The specification of the address space may be different for each of the to 304. That is, FIG. 11A shows the processing element 30.
The physical address space of one microprocessor 305 is shown. In this figure, the first local port area 404 corresponding to the access addressed to itself is omitted. Further, FIG. 11B shows a physical address space of the microprocessor 305 of the processing element 302.
Also in FIG. 11B, the second local port area corresponding to the access addressed to itself is omitted. The physical address space of the microprocessor 305 of the other processing elements 303 and 304 is similarly configured.

In this embodiment, the memory mapped I
Although the memory spaces of the memories 201 to 211 are assigned to the I / O address areas of the / O type address space, the memory spaces of these memories 201 to 211 may be assigned to the memory address areas as shown in FIG. .
In this case, it is necessary to consider memory coherency.

Further, as shown in FIG. 9, the master I / O
Using the manager 700, the secondary storage device 701, the network interface card 702, and the video card 70
3, the I / O devices such as the pointing device 704 are connected. However, as shown in FIG. 13, an expansion bus 700 may be provided in each of the processing elements 301 to 304 to connect the I / O devices. Good. In this case, the master I / O manager 700 becomes unnecessary.

A computer network switch, a massively parallel computer, a vector computer, a fault tolerant computer, a database search engine, an image processing engine, a high speed printer / copy machine, a disk server, a web using the processing system or network device of this embodiment. Server, telephone exchange, RAID (Redundant Array of)
Independent Disks), network routers, network switches, etc. can be configured.

The following effects are realized in the above embodiment.

The broadcast memory enables simultaneous transmission of 1: N. That is, in the conventional communication model, N transmissions are required when transmitting to N nodes, but in this embodiment, only one transmission is required, and the communication speed is N times as effective. Broadcast targets in this case include not only local nodes but also remote communication nodes connected via a remote port. Therefore, the range of N can easily be in the order of tens to hundreds, with 1 for more communication nodes.
It is possible to communicate at the same time with one traffic, which improves communication efficiency. Further, since the information is transmitted via a single memory, it becomes possible to transmit information without delay.

With the peer-to-peer memory, communication between local nodes can be performed without causing any competition with other nodes including the communication partner, and communication can be performed at the maximum speed of the communication channel. In addition, the data communication is not serial but the memory data path width (32 to 25
Since it is performed in parallel communication of about 6 bits), several hundred M
Very high speed communication of bytes can be realized.

By connecting the modules with both the switch matrix and the bus structure, it is possible to achieve both high-speed data communication and an I / O structure with high expandability.

By incorporating the bus structure itself into the chip, it is possible to significantly reduce the signal propagation delay on the bus structure, the electrical load capacitance, and the resulting waveform distortion, which are major problems in the physical design of the module. And physical design becomes easier.

From the viewpoint of software development, each CP
By making U an independent node on the network, application software written based on the existing software interface for massively parallel computation can be executed as it is by recompiling / linking without changing the source code. Becomes Also, the operating system can be ported by changing network drivers.

By directly mapping the network space between the nodes onto the physical address, the overhead for the communication procedure can be basically eliminated, and more efficient communication becomes possible.

By mounting a large number of nodes on the board as the minimum number of cores required for calculation, the required installation area and weight per node or computing capacity can be reduced by about an order of magnitude as compared with a workstation.

[0052]

As described above, according to the present invention, the overhead for the communication procedure can be basically eliminated by directly mapping the network space between the processor elements on the physical address, and Efficient communication is possible.

[Brief description of drawings]

FIG. 1 is a block diagram showing a processing system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of processing elements of the processing system.

FIG. 3 is a diagram illustrating a real address space of the processing element.

FIG. 4 is a diagram illustrating mapping between destinations and memories in the processing element.

FIG. 5 is a block diagram showing a specific configuration example of the processing element.

FIG. 6 is a diagram illustrating another configuration example of the processing element.

FIG. 7 is a diagram showing a composite communication system configured using an embodiment of the present invention.

FIG. 8 is a diagram showing a switch matrix of the composite communication system.

FIG. 9 is a diagram illustrating a master I / O manager of the composite communication system.

FIG. 10 is a diagram illustrating a modified example of the above-described embodiment.

FIG. 11 is a diagram illustrating a modified example of the above-described embodiment.

FIG. 12 is a diagram illustrating a modified example of the above-described embodiment.

FIG. 13 is a diagram illustrating a modified example of the above-described embodiment.

[Explanation of symbols]

100 processing system 200 network equipment 201 Broadcast memory 202-205 memory for remote communication 206-211 peer-to-peer memory 212-215 Local port 216a Remote port for input Remote port for 216b output 301-304 Processing Element 305 microprocessor 306 External cache memory 307 main memory 308 system logic 401 main memory address space 402 I / O address space 403 Broadcast memory mapping area 404 First local port area 405 Second local port area 406 Third local port area 407 Fourth local port area 408 Remote port window area 409 Other I / O device areas 500 complex communication system 600 switch 601 switch matrix 602 system I / O bus 603 output port 604 input port 700 Master I / O manager 701 secondary storage device 702 Network Interface Card 703 video card 704 pointing device 705 expansion bus

   ─────────────────────────────────────────────────── ─── Continued front page    (71) Applicant 599122293             Visual Technology Co., Ltd.             1-9-15 Kaigan, Minato-ku, Tokyo (71) Applicant 899000035             Tohoku Techno Arch Co., Ltd.             468 Aoba, Aramaki, Aoba-ku, Sendai-shi, Miyagi Prefecture (72) Inventor Mitsumasa Koyanagi             1-2-5 Yurigaoka, Natori City, Miyagi Prefecture (72) Inventor Hiroyuki Kurino             2-17-9 Yurigaoka, Natori City, Miyagi Prefecture (72) Inventor Hiroshi Sato             1-16-17 Asaya Kita, Suginami-ku, Tokyo F term (reference) 5B014 FB04 GA43 HB02 HB28                 5B045 BB32 GG01                 5B060 CA12 KA02 KA06

Claims (3)

[Claims] 1. A plurality of processing elements, and an external memory to which a memory space is allocated in an I / O address area of an address space of the memory-mapped I / O method of each of the plurality of processing elements. A processing system that does. 2. A plurality of processing elements and a plurality of external memories, each of which is accessed by a corresponding pair of the plurality of processing elements, and a plurality of destination processings in respective address spaces of the plurality of processing elements. A processing system characterized by allocating a plurality of address areas respectively corresponding to elements, and associating the plurality of address areas with the corresponding external memories for each of the processing elements. 3. The processing system according to claim 2, wherein the allocation of the plurality of address areas is common among the plurality of processing elements.