JPH0883258A - Information processing system and its method - Google Patents

Abstract

PURPOSE: To improve the calculation efficiency of a node and the utilization efficiency of a connection path by redistributing data through a data distributor which distributes the data to respective nodes. CONSTITUTION: In addition to a packet information management device 21 which manages packets sent from the respective nodes 100, 200, 300, and 400, an arbiter 20 which arbitrates the utilization of the connection path includes a path selection information management device 22 which manages path request information in information includes in the packets, an additional information management device 23 which temporarily stores additional information accompanying data transfer such as successively sent addresses, and a cache information management device 24 which temporarily stores information required to hold the consistency of a cache memory in the packets and performs control for redistributing protocol information to the respective nodes 10.... On the basis of information sent from the nodes 100..., part or the whole of the information is redistributed from the arbiter 20 to the respective nodes 100....

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a parallel computing system mainly composed of a group of information processing devices having one or more processors, memories and cache memories, and a connection path connecting the groups of information processing devices. It is about.

[0002]

2. Description of the Related Art When a plurality of groups of information processing devices each having one or more processors and memories (hereinafter referred to as nodes) are connected for the purpose of exchanging data between nodes,
There are a method of connecting using various LANs, and a method of connecting nodes at the address level of the memory regardless of the LAN. As an example of the latter, the applicant of the present invention has a Japanese Patent Application No. 5-286.
No. 876, filed for an information processing device using an optical wavelength multiplexing system. According to this method, different data transfer can be realized simultaneously among a plurality of nodes by using a plurality of identifiable wavelengths.

Further, as an improved example of the system, each node sends the additional information related to the data transfer to the arbiter at the same time when the application is filed as Japanese Patent Application No. 5-288271, when the connection route is requested prior to the data transfer. The arbiter sends that information to the connection destination node through the arbitration signal line when setting the connection route, and the node that receives the connection request overlaps the route setup and the preparation of the data to be transmitted and received between the nodes. By doing so, an information processing apparatus has been devised that improves the efficiency of data transfer after setting a connection path.

[0004]

Further, in order to improve the calculation efficiency of a node by using a cache memory for an information processing apparatus using this optical wavelength multiplexing system,
It is possible to realize the caching of information between nodes by distributing the data for maintaining the consistency of the data in the cache memory to each node by using the arbiter and the signal path for arbitration.

However, as described above, in the method of using the arbiter and the signal path for arbitration to distribute the data for maintaining the consistency of the data of the cache memory to each node, the nodes are sequentially arranged in the arbiter. Since the packets are sent to the node, if the number of nodes increases, there is a possibility that processing delay will occur and performance will be degraded.

Furthermore, when the arbitration signal path is used for the original optical connection path use request, the arbitration signal path cannot be used for cache maintenance until the use by the request is completed, and the processing therefor is required. There was a possibility that the performance would deteriorate due to the delay of.

Therefore, an object of the present invention is to realize high-speed caching of information between nodes.

[0008]

In order to solve the above problems, according to the present invention, each node has a plurality of nodes, one or more processors, a cache memory, and a bus inside the node. A connection capable of simultaneously connecting a plurality of sets between the plurality of nodes in the information processing including a monitoring unit that monitors the signal of 1) and an updating unit that updates the internal state of the cache memory based on the monitoring result of the monitoring unit. A route, a data distribution unit arranged on the connection route and distributing data to each node, an arbitration unit for arbitrating a use request of the connection route, and a connection between the arbitration unit and each node An arbitration signal path for transmitting, and a transmission means for transmitting a part or all of information necessary for data transfer in and between nodes from the node to the arbitration means by using the arbitration signal path, Based on the information transmitted from the node by sending means, some or all of the information, from the arbitration unit via the data distribution unit, comprising a redistribution means for redistributing on each node.

According to another aspect of the present invention, a plurality of nodes are provided, and each node includes one or more processors.
An information processing method, comprising: a cache memory; a monitoring unit for monitoring a signal on a bus inside a node; and an updating unit for updating an internal state of the cache memory based on a monitoring result of the monitoring unit. In the arbitration step for arbitrating by the arbitration unit the use request of the connection path capable of simultaneously connecting a plurality of sets between the plurality of nodes, and using the arbitration signal path respectively connecting between the arbitration unit and each node, A transmission step of transmitting a part or all of information necessary for data transfer within a node and between nodes from the node to the arbitration unit using the arbitration signal path, and from the node by the transmission step. Based on the information, a part or all of the information is arranged on the connection path from the arbitration unit using the arbitration signal path, and the data is distributed to each node. A redistribution step of redistributing to each node through a data distributor, and a caching step of caching information between the nodes by reflecting the information distributed to each node by the redistribution step inside the node. Equipped with.

[0010]

With the arbitration unit arbitrating the use request of the connection path capable of simultaneously connecting a plurality of sets between a plurality of nodes, the arbitration signal path connecting each of the arbitration unit and each node is used. And a part or all of the information required for data transfer between nodes is transmitted from the node to the arbitration unit using the arbitration signal path, and based on the information transmitted from the node, one of the information is transmitted. All or part of the arbitration unit is redistributed to each node through a data distributor that is arranged on the connection path and distributes data to each node. By reflecting the information inside the nodes, the caching of information between the nodes is performed at high speed.

[0011]

1 is a block diagram of an embodiment of a system for realizing the present invention.

Reference numerals 100, 200, 300 and 400 are nodes, and each node is connected to the concentrator 30 by a connection path composed of a pair of optical fibers 31, 32, 33 and 34. The concentrator 30 contains a star coupler 50 for redistributing the signals coming from the connection paths formed by the optical fibers 31, 32, 33, 34.

Each node has a CPU 101, 20
1, 301, 401, memories 102, 202, 302,
402, interface circuits 103, 203, 303, 403 for connecting a connection path formed by the optical fibers 31, 32, 33, 34 and the inside of each node.
Arbitration interface circuits 104, 204, 304, 404 for requesting use of a connection path formed by the optical fibers 31, 32, 33, 34,
And internal buses 105, 205, 305, 405 for interconnecting them inside the nodes, and cache maintenance interface circuits 107, 207, 307, 40.
7. The wavelength multiplexers 108, 208, 308 and 408 are included. Here, the cache memory 1 in each node
06, 206, 306, 406 are processor 1 respectively
01, 201, 301, 401 are built in. However, the configuration is not limited to this.

Reference numeral 20 denotes optical fibers 31, 32, 33, 3
The arbiter 20 is an arbiter for arbitrating the use of the connection path configured by the reference numeral 4.
They are connected by 0 and 410.

Also, inside the arbiter, a packet information management device 2 for managing packets sent from each node.
In addition to 1, the route selection information management device 22 that manages the route request information among the information included in the packet, and the additional information that temporarily stores the additional information accompanying the data transfer such as the address sent subsequently to the information. An information management device 23 and a cache information management device 24 for temporarily storing information necessary for maintaining the consistency of the cache memory in the packet and controlling for redistributing protocol information to each node are installed. Has been done. From the cache information management device 24, the optical signal path 3 for cache maintenance is used.
5 is connected to the star coupler 50 inside the concentrator 30.

This embodiment shows an example in which the consistency of the cache memory is guaranteed by using the invalidation protocol of the write-through cache memory in the system having the configuration shown in FIG.

Specifically, the node 100 is replaced by the node 200.
It is shown how, while the above data is being cached, the modification of the data and the operation of maintaining the coherency of the cache memory associated therewith are carried out.

FIG. 2 is an address map of the entire system. In this embodiment, 4 gigabytes of the address space of the entire system is distributed to four nodes for use.
Now, the node 100 has 41 nodes on the RAM of the node 200.
Suppose that the data at address 000000h (4 bytes) is being cached and is about to be changed.

The processor 101 in the node 100
R in the node 200 to the internal cache memory 106
The process of caching data at AM 41000000h is as shown in Japanese Patent Application No. 5-288271,
Data is read between nodes, and the data is stored in the cache memory. For the process of storing this data in cache memory,
Since the cache memory system is well known, its details are omitted.

FIG. 3 is a block diagram of the arbiter interface 104.

The address decoder 140 existing inside the arbiter interface 104 constantly monitors the internal bus 105 (composed of the data signal line 151, the control signal line 152, and the address signal line 153) of the node 101 and performs caching. External node accompanying the data write-through operation that was performed (node 200 in this case)
When it is recognized that an access to the bus (write operation to the address 41000000h) has occurred on the bus, the node arbitration control processor 14 is notified by the external access detection signal 144 and the write request detection signal 150.
Pass control to the program running on 1.

At the same time, the address latch register 142 latches the address currently output on the address signal line 153, and the control signal latch register 143 can read / write request type (write) / transfer byte number (4 bytes) and can be cached. Latch control information such as being an area. Although the one-chip microcontroller is used as the node arbitration control processor 141 in this embodiment, the node arbitration control processor 141 is not limited to this configuration and may be configured by hardware logic or the like.

The node arbitration control processor 141 reads the latched signals from the address latch 142 and the control signal latch 143, determines the connection destination, creates an arbitration request packet as shown in FIG. 4, and performs parallel / serial conversion. Bowl 1
Write to 61. The packet shown in FIG. 4 is a route request signal and also includes additional information relating to data transfer as an internal data format.

The parallel / serial converter 161 converts the written information into serial data, and the light emitting element 1
Output to 63. The light emitting element 163 photoelectrically converts the input signal and outputs it to the arbiter 20 as an optical signal of wavelength λ1 through the communication path 110 formed of an optical fiber. This configuration is common to all nodes.
Note that the light emitting element here is an element such as an LED or a laser, and the light receiving element is an element represented by a photodiode.

FIG. 5 shows a block diagram of the arbiter 20. Reference numerals 601, 603, 605, and 607 are light receiving elements. The optical signal of wavelength λ1 emitted from each node, that is, the arbitration request signal is received and converted into an electric signal. Now, the request signal arrives from the node 100 and is input to the serial / parallel converter 611. The serial / parallel converter 611 converts the input serial electric signal into a parallel signal and, at the same time, notifies the packet information management device 21 of the data reception detection signal 622.

In this embodiment, the packet information management device 21
Is the ROM storing the program and the R used for processing
It is configured by a micro controller 621 having a built-in AM. Further, it is assumed that this microcontroller also includes a part that plays a role of the route selection information management device 22, the additional information management device 23, and the cache information management device 24. However, the configuration is not limited to this.

When the packet information management device 21 receives the data reception detection signal (1) 622, it selects the serial / parallel converter 611 by the device select signal 619, and the node 100 from the internal register through the data bus 620. Read out the sent request packet.

Of the information included in the request packet, the information such as the request source node number and the connection destination node number is
The part of the additional information that is transferred to the route selection information management device 22 and is involved in the data transfer is stored in the part that functions as the additional information management device 23 in this microcontroller. Furthermore, if it is determined that this packet is associated with a write operation, it is necessary to maintain the cache coherency in each node. Therefore, the address and transfer byte number in the request packet, the request source node number, the connection destination node Information such as a number is stored in a portion of the microcontroller that functions as the cache memory management device 24.

The route selection information management device 22 analyzes the received data, and a request to use this transmission line is sent to the node 10
It recognizes that the connection request is from 0 to the node 200, checks the transmission path use status flag provided in the path selection information management device 22 and the wavelength being used, and if it is in the usable status, the flag Is set to the in-use state, the connection preparation request packet shown in FIG. 6 is created, and is written in the parallel / serial converters 612 and 614.

The connection preparation packet also includes the information provided from the part of the additional information management device 23 in the microcontroller and the wavelength information. These two connection preparation request packets use the optical signal of λ1 as in the case of the optical arbiter interface,
It is output to the nodes 100 and 200. In addition,
Here, the four nodes can simultaneously perform two-system communication between the one-to-one nodes by using different wavelengths λ2 and λ3 for data communication.

Following this, the cache information management device 2
4 uses node 3 to maintain the coherency of cache memory between nodes based on the information provided to node 3.
For 00 and 400, address 41000000h
When the data of 1 is held in the cache memory, the cache memory invalidation packet as shown in FIG. 7 is created to instruct to invalidate it, and the packet is written in the parallel / serial converters 616 and 618.

The written packet is the light emitting element 610.
Through the optical signal path 35 for cache maintenance through the wavelength λc and is input to the star coupler 50 inside the concentrator 30. At this time, λc is different from the wavelengths λ2 and λ3 used for data transmission in order to prevent interference. The cache memory invalidation packet input to the star coupler 50 is equally demultiplexed to each node and output to each node through the optical fibers 31, 32, 33, 34.

The subsequent data transfer from the node 100 to the node 200 will be omitted, and the cache memory coherency holding operation in the nodes 300 and 400 will be described by taking the operation in the node 300 as an example.

FIG. 8 shows a block diagram of the cache maintenance interface 307 and the optical multiplexer 308.

The optical signal input through the fiber 33 is separated by the optical demultiplexer 176 into light (λ2 or λ3) toward the connection path interface 303 and light (λc) toward the cache maintenance interface 307. Cache maintenance interface 30
The light having the wavelength λc input to 7 is converted into an electric signal by the light receiving element 169, and the serial / parallel converter 1
At the same time as being converted into a parallel signal by 70, it is notified to the cache control microcontroller 172 by the cache maintenance packet reception signal 171.

When this notification is detected, the cache control microcontroller 172 of the node 300 receives the device select signal 1 from the serial / parallel converter 170.
73, the data bus 175 is used to read the cache memory invalidation packet shown in FIG. 7, and request the use permission of the internal bus 305 toward the inside of the node. When the use permission of the internal bus is given, the cache control microcontroller 172 uses the cache maintenance request signal group 174 to send the address 4100 to the connection route interface 303 based on the contents of the packet of FIG.
It instructs the cache memory that caches 4-byte data of 0000h to invalidate the block.

FIG. 9 shows the connection path interface unit 303.
An example is shown below. Here, the cache maintenance request signal group 174 sent from the cache control microcontroller 172 instructs the address driver 130 to the address (41000000h) and the data transfer sequencer 131 to request the cache memory block invalidation. In this case, specifically, the writing of the dummy data of the address 41000000h onto the internal bus 305 is instructed.

The sequencer 131 receives the signal 134,
Address 4100000 for the address driver 130
The drive of 0h is instructed, and then the control driver 132 is instructed to drive the control signal of the transfer size (4 bytes) and the read / write signal (write) to the bus through the signal line 136. Furthermore, the signal line 137
To instruct the data buffer 133 to drive dummy data onto the bus.

The in-node processor 301 snooping this dummy data write process inspects the address tag of the cache memory of the internal cache memory 307,
When the corresponding block exists, that block is invalidated.

On the other hand, since the corresponding address does not exist in the memory 302, this write processing is ignored. In order to prevent the bus from timing out, the data transfer sequencer 131 instructs the control driver 132 after a certain delay to drive the acknowledge signal.

A similar operation is performed in node 400.

As a result, the cache coherency keeping operation accompanying the data write operation of the node 100 is realized.

The same processing is performed in the transfer between other nodes.

However, the node 100 which is the source and the node 200 which is the destination of the data to be transferred receive the cache invalidation request packet shown in FIG. 7, interpret the contents, and the own node number is When the packet is found to be in the request source or connection destination field inside the packet, the subsequent processing supposed to be performed by the packet will be ignored.

In this embodiment, the arbitration signal paths 110, 210, 310 and 410 shown in FIG. 1 are used.
Light of wavelength λ1 is used for the optical signal on the
The optical signals on 1, 32, 33 and 34 have wavelengths λ2 and λ3.
(Λ2 and λ3 are different wavelengths), but λ1 = λ
2. Even if λ1 = λ3, there is no problem in the structure.

Next, in the case of changing the data to the cacheable area within the own node, specifically, the node 100
RAM in own node that can be cached by other nodes
When the data (4 bytes) at the address 01000000h above is cached and an attempt is made to change the data, how the coherency holding operation of the cache memory is carried out is shown.

In FIG. 3, the external access detection signal 144 does not react this time, and only the write request detection signal 150 causes the node arbitration control processor 141 to operate.
Pass control to the program running above. At the same time, in the address latch register 142, the address signal line 15
3 is latched, and control information such as the number of transfer bytes (4 bytes) is latched in the control signal latch register 143.

The node arbitration control processor 141 reads the address and control information latched from the address latch 142 and the control signal latch 143, creates a cache maintenance request packet as shown in FIG. 10, and parallel / serial converter 161. Write in. The parallel / serial converter 161 converts the written information into serial data and outputs it to the light emitting element 163. The light emitting element 163 photoelectrically converts the input signal into an optical signal of wavelength λ1,
The data is output to the arbiter 20 through the communication path 110 composed of an optical fiber.

In FIG. 5, when a cache maintenance request packet arrives from the node 100 and is input to the serial / parallel converter 611, the serial / parallel converter 611 converts the input serial electric signal into a parallel signal. At the same time, the packet information management device 21 is notified by the data reception detection signal 622.

Upon receiving the data reception detection signal (1) 622, the packet information management device 21 selects the serial / parallel converter 611 by the device select signal 619, and the internal register through the data bus 620.
The cache maintenance request packet sent from the node 100 is read out. Then, information such as the address in the packet, the number of transfer bytes, and the requesting node number
The data is stored in a part that functions as the cache memory management device 24 in the microcontroller.

The cache information management device 24, based on the information provided to itself, keeps the address 0100 for each node in order to maintain the consistency of the cache memory.
When the data of 0000h is held in the cache memory, the cache memory invalidation packet as shown in FIG. 7 is created and written to the parallel / serial converter 627 in order to instruct to invalidate it. However, in this case, the connection destination node field of the packet includes a node number that does not actually exist. The written packet is converted into an optical signal by 628 and distributed to all nodes through the star coupler 50.

The operation after this is the same as that of the previous example, and therefore its explanation is omitted.

In the embodiment shown so far, FIG.
Arbitration signal paths 110 and 31, at
Although 210 and 32, 310 and 33, and 410 and 34 are assumed to be physically different signal paths, if these lines can be logically separated, the same signal path (ie, the same signal path) is physically used. It may pass through an optical fiber). However, in that case, in order to prevent interference when wavelength multiplexing, λ1,
It is necessary that λ2, λ3, and λc have different wavelengths.

[0054]

As described above, according to the present invention,
Part or all of the information necessary for data transfer within and between nodes is transmitted from the node to the arbiter using the arbitration signal path, and based on the information transmitted from the node, part or all of that information is transmitted. , Arbiter distributes data to each node by redistributing data to each node via a data distributor that distributes data to each node. Reflect the above information in the node, execute the data consistency retention operation of cache memory, realize the caching of information between nodes at high speed, and improve the calculation efficiency in nodes and the utilization efficiency of connection paths. It is possible to realize a higher performance information processing system.

[Brief description of drawings]

FIG. 1 is a block diagram showing a functional configuration of an information processing system according to an embodiment of the present invention.

FIG. 2 is a diagram showing an address map of the system of the embodiment.

FIG. 3 is a diagram illustrating an arbiter interface of a node according to an embodiment.

FIG. 4 is a diagram showing a structure of an arbitration request packet.

FIG. 5 is a diagram showing a configuration of an arbiter according to an embodiment.

FIG. 6 is a diagram showing a structure of a connection preparation request packet.

FIG. 7 is a diagram showing a configuration of a cache memory invalidation packet.

FIG. 8 is a diagram showing a configuration of a cache maintenance interface unit.

FIG. 9 is a diagram showing a configuration of a connection path interface unit.

FIG. 10 is a diagram showing a configuration of a cache maintenance request packet.

[Explanation of symbols]

20 Arbiter 21 Packet Information Management Device 22 Route Selection Information Management Device 23 Additional Information Management Device 24 Cache Information Management Device 30 Concentrator 31, 32, 33, 34 Optical Fiber 35 Optical Signal Route for Cache Maintenance 50 Star Coupler 100, 200, 300, 400 node 101, 201, 301, 401 CPU (processor) 102, 202, 302, 402 memory 103, 203, 303, 403 connection path interface circuit 104, 204, 304, 404 arbiter interface circuit 105, 205, 305, 405 node Internal bus 106, 206, 306, 406 Cache memory 107, 207, 307, 407 Cache memory maintenance interface circuit 108, 208, 308, 408 Long multiplexer 110, 210, 310, 410 Arbitration signal path 130 Address driver 131 Data transfer sequencer 132 Control driver 133 Data buffer 134 Address drive signal 135 Acknowledge signal 136 Control driver control signal 137 Data buffer control signal 138 Data reception signal 139 Parallel / serial converter control signal 140 Address decoder 141 Node arbitration control processor 142 Address latch register 143 Control signal latch register 144 External access detection signal 145 Data signal line 146 Register select signal line 147 Device select signal line 148 Data reception signal 149 Data transmission / reception Request signal group 150 Write request detection signal 1 51 internal bus data signal line 152 internal bus control signal line 153 internal bus address signal line 161, 165, 612, 614, 616, 618, 6
27 Parallel / serial converter 162, 166, 170, 611, 613, 615, 6
17 Serial / Parallel Converter 163, 167, 602, 604, 606, 608, 6
28 light emitting elements 164, 168, 169, 601, 603, 605, 6
07 Light receiving element 171 Cache maintenance packet reception signal 172 Cache control microcontroller 173 Device select signal 174 Cache maintenance request signal group 175 Data signal line 619 Device select signal 620 Data bus 621 Micro controller 622, 623, 624, 625 Data detection signal 1,
2, 3, 4 626 control signal

Front Page Continuation (72) Inventor Masato Kosugi 3-30-2 Shimomaruko, Ota-ku, Tokyo Canon Inc.

Claims (10)

[Claims] 1. A plurality of nodes, each node comprising one or more processors, a cache memory, a monitoring means for monitoring a signal on a bus inside the node, and a monitoring result of the monitoring means. An information processing system comprising: an updating unit for updating an internal state of the cache memory, wherein a connection path capable of simultaneously connecting a plurality of sets between the plurality of nodes and a data path arranged on the connection path. Data distribution means for distributing to each other, an arbitration means for arbitrating a use request of the connection path, an arbitration signal path connecting each of the arbitration means and each node, and data within the node and between the nodes. Transmission means for transmitting a part or all of the information required for transfer from the node to the arbitration means by using the arbitration signal path, and information transmitted from the node by the transmission means. Based on
A part of or all of the information is re-distributed from the arbitration means to each node through the data distribution means, and the information distributed to each node by the re-distribution means,
An information processing system characterized in that information is cached between nodes by reflecting the information inside the nodes. 2. The information processing system according to claim 1, wherein the inter-node connection path is an optical wavelength-multiplexed connection path that connects using light of a plurality of wavelengths. 3. The information processing system according to claim 1, wherein the data distribution unit includes a star coupler. 4. The information processing system according to claim 1, wherein the inter-node connection path and the arbitration signal path are wavelength division multiplexed and configured by a common optical fiber. 5. The information processing system according to claim 1, wherein the protocol for maintaining the consistency of the data in the cache memory in the node is a write-through invalidation type protocol. 6. A plurality of nodes, each node comprising one or more processors, a cache memory, a monitoring means for monitoring a signal on a bus inside the node, and a monitoring result of the monitoring means. In an information processing system including update means for updating the internal state of the cache memory, an arbitration step of arbitrating a use request of a connection path capable of simultaneously connecting a plurality of sets between the plurality of nodes by an arbitration section; A part of or all of the information necessary for data transfer within the node and between the nodes by using the arbitration signal path connecting between the node and each node to the arbitration signal from the node. Based on the transmission process of transmitting using the route and the information transmitted from the node by the transmission process,
A redistribution step of redistributing a part or all of the information from the arbitration unit to the nodes through a data distributor that is arranged on the connection path and distributes data to the nodes. The information distributed to each node by
An information processing method comprising: a caching step of caching information between nodes by reflecting the information inside the nodes. 7. The information processing method according to claim 6, wherein light having a plurality of wavelengths is wavelength-multiplexed and used in the inter-node connection path. 8. The information processing system according to claim 6, wherein the data distributor has a star coupler. 9. The information processing method according to claim 6, wherein a common optical fiber is used after being wavelength-division multiplexed, as the inter-node connection path and the arbitration signal path. 10. The information processing method according to claim 6, wherein the node maintains the consistency of data in the cache memory by using a write-through invalidation type protocol.