JP3599381B2 - Information processing system and method - Google Patents

Description

[0001]
[Industrial applications]
The present invention generally relates to a group of information processing apparatuses having at least one processor, a memory, and a cache memory, and a parallel computing system including a connection path connecting the group of information processing apparatuses.
[0002]
[Prior art]
When connecting a plurality of information processing apparatuses each having one or more processors and a memory (hereinafter, referred to as nodes) for the purpose of exchanging data between nodes, a method of connecting using various LANs, There is a method of connecting the nodes at the address level of the memory regardless of the type. As an example of the latter, the present applicant has filed an application for an information processing apparatus using an optical wavelength multiplexing system as Japanese Patent Application No. 5-286876. According to this method, by using a plurality of identifiable wavelengths, different data transfer can be simultaneously realized between a plurality of nodes.
[0003]
Further, as an improved example of the system, each node, which was filed by the applicant of the present invention as Japanese Patent Application No. 5-288271, simultaneously sends additional information related to data transfer to the arbiter at the time of connection path request prior to data transfer. When setting the connection route, the information is sent to the connection destination node via the arbitration signal line, and the node receiving the connection request realizes the setup of the route and the preparation of the data to be sent and received between the nodes by overlapping. Information processing apparatuses that improve the efficiency of data transfer after setting a connection path have been devised.
[0004]
[Problems to be solved by the invention]
Further, in order to improve the computational efficiency of the node using the cache memory for the information processing apparatus using the optical wavelength multiplexing method, the data of the cache memory is utilized by using the arbiter and the arbitration signal path. By distributing data for maintaining consistency to each node, it is conceivable to realize information caching between nodes.
[0005]
However, as described above, in the method of using the arbiter and the signal path for arbitration to distribute data for maintaining data consistency of the cache memory to each node, packets are sequentially transmitted to each node inside the arbiter. If the number of nodes increases due to the transmission, the processing may be delayed, and the performance may be degraded.
[0006]
Further, when the arbitration signal path is used for the original optical connection path use request, the arbitration signal path cannot be used for cache maintenance until the use of the arbitration signal path is completed, which causes a delay in processing for that purpose. As a result, there is a possibility that the performance may be reduced.
[0007]
Therefore, an object of the present invention is to realize high-speed information caching between nodes.
[0008]
[Means for Solving the Problems]
In order to solve the above problems, according to the present invention, there are provided a plurality of nodes, each node having at least one processor, a cache memory, and monitoring means for monitoring a signal on a bus inside the node, The information processing system includes an updating unit that updates the internal state of the cache memory based on the monitoring result of the monitoring unit. A connection path capable of simultaneously connecting a plurality of sets of nodes, a data distribution means arranged on the connection path and distributing data to each node through the connection path, and an arbitration means for arbitrating a use request of the connection path And an arbitration signal path connecting the arbitration unit and each node, and a part or all of information necessary for data transfer within the node and between nodes, and Transmission means for transmitting using the arbitration signal path, and distributing a part or all of the information transmitted by the transmission means to each node from the arbitration means through the data distribution means and the connection path. And a distribution control unit for performing the control as described above, and a reflection unit for reflecting the information distributed by the distribution unit in a cache memory inside the node.
[0009]
According to another aspect of the present invention, there are provided a plurality of nodes, each of which has at least one processor, a cache memory, and a monitoring means for monitoring a signal on a bus inside the node; An information processing method in an information processing system comprising: an updating unit that updates an internal state of the cache memory based on a monitoring result of the unit. An arbitration step of arbitrating, by an arbitration unit, a use request for a connection path that can simultaneously connect a plurality of sets by connecting the arbitration unit, and an arbitration signal path for connecting the arbitration unit and each of the nodes; Transmitting a part or all of the information necessary for data transfer between the nodes to the arbitration unit using the arbitration signal path; and transmitting one or more pieces of information transmitted in the transmission step. Or all of the data is distributed from the arbitration unit to the nodes via the connection path and distributes the data to the nodes through the data distributor and the connection path, and the data distributor distributes the data to the nodes through the connection path. Reflecting the information to a cache memory inside the node.
[0011]
【Example】
FIG. 1 is a configuration diagram of an embodiment of a system for realizing the present invention.
[0012]
Reference numerals 100, 200, 300, and 400 denote nodes. Each node is connected to the concentrator 30 by a connection path including a pair of optical fibers 31, 32, 33, and. Concentrator 30 incorporates a star coupler 50 for redistributing signals resulting from the connection path formed by optical fibers 31, 32, 33, 34.
[0013]
Each node is used to connect a connection path constituted by CPUs 101, 201, 301, 401, memories 102, 202, 302, 402, and optical fibers 31, 32, 33, 34 therein, and the inside of each node. Arbitration interface circuits 104, 204, 304, 404 for requesting the use of connection paths constituted by the interface circuits 103, 203, 303, 403 and the optical fibers 31, 32, 33, 34; It includes internal buses 105, 205, 305, 405 for interconnection, cache maintenance interface circuits 107, 207, 307, 407, and wavelength multiplexing devices 108, 208, 308, 408. Here, the cache memories 106, 206, 306, and 406 in each node are built in the processors 101, 201, 301, and 401, respectively. However, it is not limited to this configuration.
[0014]
Reference numeral 20 denotes an arbiter for arbitrating the use of a connection path constituted by the optical fibers 31, 32, 33, and 34. The arbiter 20 is connected to each node by arbitration signal paths 110, 210, 310, and 410, respectively. Have been.
[0015]
In the arbiter, in addition to the packet information management device 21 that manages packets sent from each node, a route selection information management device 22 that manages route request information among the information included in the packets, And an additional information management device 23 for temporarily storing additional information associated with data transfer such as an address sent subsequent to the packet, and temporarily storing information necessary for maintaining consistency of a cache memory in a packet. And a cache information management device 24 for controlling the redistribution of protocol information to each node. From the cache information management device 24, a cache maintenance optical signal path 35 is connected to the star coupler 50 inside the concentrator 30.
[0016]
In the present embodiment, an example will be described in which the consistency of the cache memory is guaranteed using the invalidation protocol of the write-through cache memory in the system configured as shown in FIG.
[0017]
Specifically, it shows how, when the node 100 is caching data on the node 200, the change of the data and the accompanying operation of maintaining the consistency of the cache memory are performed.
[0018]
FIG. 2 is an address map of the entire system. In this embodiment, the address space of 4 gigabytes of the entire system is allocated to four nodes and used. Now, it is assumed that the node 100 has cached the data (4 bytes) at the address 4100000h on the RAM of the node 200 and is going to change it.
[0019]
As described in Japanese Patent Application No. 5-288271, data is read between nodes during the process of caching the data at the address 410000h of the RAM inside the node 200 into the cache memory 106 inside the processor 101 inside the node 100. , By storing the data in a cache memory. The process of storing this data in the cache memory is well-known as a cache memory system, and will not be described in detail.
[0020]
FIG. 3 is a block diagram of the arbiter interface 104.
[0021]
The address decoder 140 existing inside the arbiter interface 104 constantly monitors the internal bus 105 of the node 101 (constituted by the data signal line 151, the control signal line 152, and the address signal line 153) and performs caching. When it is recognized that access to the external node (in this case, node 200) (write operation to the address 4100000h) has occurred on the bus due to the data write-through operation, the external access detection signal 144 and the write request detection signal 150 , And passes control to a program operating on the node arbitration control processor 141.
[0022]
At the same time, the address latched at that time on the address signal line 153 is latched in the address latch register 142, and the read / write request type (write), the number of transferred bytes (4 bytes), and the cacheable area are stored in the control signal latch register 143. Latch control information such as things. In this embodiment, a one-chip microcontroller is used as the node arbitration control processor 141. However, the present invention is not limited to this configuration and may be configured by hardware logic or the like.
[0023]
The node arbitration control processor 141 reads the latched signal from the address latch 142 and the control signal latch 143, determines the connection destination, creates an arbitration request packet as shown in FIG. Write. The packet shown in FIG. 4 is a path request signal, and also includes additional information relating to data transfer as an internal data format.
[0024]
The parallel / serial converter 161 converts the written information into serial data and outputs the serial data to the light emitting element 163. The light emitting element 163 photoelectrically converts the input signal, and outputs the signal to the arbiter 20 through the communication path 110 constituted by an optical fiber as an optical signal of the wavelength λ1. This configuration is common to all nodes. Note that the light emitting element here is an element such as an LED or a laser, and the light receiving element refers to an element represented by a photodiode.
[0025]
FIG. 5 shows a block diagram of the arbiter 20. Reference numerals 601, 603, 605, and 607 are light receiving elements. A request signal arrives from the node 100 and receives the optical signal of the wavelength of λ1 emitted by the node, that is, the arbitration request signal and converts it into an electric signal. The request signal arrives from the node 100 and is input to the serial / parallel converter 611. The serial / parallel converter 611 converts the input serial electric signal into a parallel signal and, at the same time, notifies the packet information management device 21 with a data reception detection signal 622.
[0026]
In the present embodiment, the packet information management device 21 is configured by a microcontroller 621 having a ROM storing a program and a RAM used for processing. The microcontroller also includes a part that plays the role of a route selection information management device 22, an additional information management device 23, and a cache information management device 24 at the same time. However, the present invention is not limited to this configuration.
[0027]
Upon receiving the data reception detection signal (1) 622, the packet information management device 21 selects the serial / parallel converter 611 according to the device select signal 619, and is transmitted from the node 100 via the data bus 620 from an internal register. Read the request packet.
[0028]
Among the information included in the request packet, information such as the request source node number and the connection destination node number is passed to the route selection information management device 22, and additional information related to data transfer is stored in the microcontroller. It is stored in a part that plays a role as the additional information management device 23. Further, when it is determined that this packet is accompanied by a write operation, since an operation of maintaining cache coherency in each node is necessary, the address and the number of transfer bytes in the request packet, the request source node number, the connection destination node Information such as a number is stored in a portion of the microcontroller that functions as the cache memory management device 24.
[0029]
The route selection information management device 22 analyzes the received data, recognizes that the use request of the transmission line is a connection request from the node 100 to the node 200, and is provided in the route selection information management device 22. The transmission line use state flag and the wavelength in use are checked, and if it is available, the flag is set to the use state, a connection preparation request packet shown in FIG. 6 is created, and the parallel / serial converter 612 is used. And 614.
[0030]
This connection preparation packet also includes information provided from a part of the microcontroller that plays the role of the additional information management device 23 and wavelength information. These two connection preparation request packets are output to the node 100 and the node 200 using the optical signal of λ1 as in the case of the optical arbiter interface. Here, the four nodes use different wavelengths λ2 and λ3 for data communication, respectively, so that two-system communication can be simultaneously performed between one-to-one nodes.
[0031]
Subsequently, the cache information management device 24 sends the address 4100000h to the nodes 300 and 400 based on the information provided to itself, in order to maintain the consistency of the cache memory between the nodes. If the data is held in the cache memory, a cache memory invalidation packet as shown in FIG. 7 is created and written to the parallel / serial converters 616 and 618 to instruct to invalidate the data.
[0032]
The written packet is transmitted through the light emitting element 610 onto the cache maintenance optical signal path 35 using the wavelength λc, and is input to the star coupler 50 inside the concentrator 30. At this time, λc is different from the wavelengths λ2 and λ3 used for data transmission in order to prevent interference. The cache memory invalidation packet input to the star coupler 50 is equally branched to each node and output to each node through the optical fibers 31, 32, 33, and 34.
[0033]
The subsequent data transfer from the node 100 to the node 200 is omitted, and the cache memory coherency maintaining operation in the nodes 300 and 400 will be described by taking the operation in the node 300 as an example.
[0034]
FIG. 8 shows a configuration diagram of the cache maintenance interface 307 and the optical multiplexing device 308.
[0035]
The optical signal input by the fiber 33 is separated by the optical demultiplexer 176 into light (λ2 or λ3) directed to the connection path interface 303 and light (λc) directed to the cache maintenance interface 307. The light having the wavelength λc input to the cache maintenance interface 307 is converted into an electric signal by the light receiving element 169 and further converted into a parallel signal by the serial / parallel converter 170, and at the same time, the cache signal is received by the cache maintenance packet reception signal 171. The control microcontroller 172 is notified.
[0036]
When this notification is detected, the cache control microcontroller 172 of the node 300 uses the device select signal 173 and the data bus 175 from the serial / parallel converter 170 to read the cache memory invalidation packet shown in FIG. , Requesting permission to use the internal bus 305 toward the inside of the node. When the use permission of the internal bus is given, the cache control microcontroller 172 uses the cache maintenance request signal group 174 to send the 4-byte data of the address 4100000h to the connection path interface 303 based on the contents of the packet in FIG. Instructs the caching cache memory to invalidate the block.
[0037]
FIG. 9 shows an example of the connection path interface unit 303. Here, the address (4100000h) is instructed to the address driver 130 and the request to invalidate the cache memory block is instructed to the data transfer sequencer 131 by the cache maintenance request signal group 174 sent from the cache control microcontroller 172. In this case, specifically, writing of dummy data at address 4100000h onto internal bus 305 is instructed.
[0038]
The sequencer 131 instructs the address driver 130 to drive the address 4100000h by the signal 134, and then instructs the control driver 132 to transfer the transfer size (4 bytes) and the control signal of the read / write signal (write) to the bus. Through the signal line 136. Further, it instructs the data buffer 133 to drive the dummy data onto the bus through the signal line 137.
[0039]
The in-node processor 301 that has snooped the write processing of the dummy data checks the address tag of the cache memory of the internal cache memory 307, and invalidates the corresponding block if it exists.
[0040]
On the other hand, since there is no corresponding address in the memory 302, this write processing is ignored. The data transfer sequencer 131 instructs the control driver 132 after a certain delay to drive the acknowledge signal in order to prevent the bus from timing out.
[0041]
A similar operation is performed in node 400.
[0042]
As a result, a cache coherency maintaining operation accompanying the data write operation of the node 100 is realized.
[0043]
The same processing is performed in the transfer between other nodes.
[0044]
However, the source node 100 and the node 200 that is the destination of the data to be transferred receive the cache invalidation request packet shown in FIG. 7 and interpret the contents thereof, and determine the own node number in the packet. When the packet is found in the request source or destination field, any further processing that should be performed by the packet is ignored.
[0045]
In this embodiment, light having a wavelength of λ1 is used for the optical signals on the arbitration signal paths 110, 210, 310, and 410 in FIG. 1, and the optical signals on the connection paths 31, 32, 33, and 34 are used for the optical signals. Although wavelengths λ2 and λ3 (λ2 and λ3 are different wavelengths) are used, there is no problem in the configuration even when λ1 = λ2 and λ1 = λ3.
[0046]
Next, in the case of changing the data to the cacheable area inside the own node, specifically, the node 100 caches the data (4 bytes) of the address 010000000h in the own node RAM which can be cached by another node. This indicates how, when an attempt is made to change it, the operation of maintaining the coherency of the cache memory is performed accordingly.
[0047]
In FIG. 3, the external access detection signal 144 does not respond this time, and the control is passed to the program operating on the node arbitration control processor 141 only by the write request detection signal 150. At the same time, the address latched on the address signal line 153 is latched in the address latch register 142, and control information such as the number of transfer bytes (4 bytes) is latched in the control signal latch register 143.
[0048]
The node arbitration control processor 141 reads the address and control information latched by the address latch 142 and the control signal latch 143, creates a cache maintenance request packet as shown in FIG. The parallel / serial converter 161 converts the written information into serial data and outputs the serial data to the light emitting element 163. The light emitting element 163 photoelectrically converts the input signal, and outputs the signal to the arbiter 20 through the communication path 110 constituted by an optical fiber as an optical signal of the wavelength λ1.
[0049]
In FIG. 5, when a cache maintenance request packet arrives from the node 100 and is input to the serial / parallel converter 611, the serial / parallel converter 611 converts the input serial electric signal into a parallel signal, and at the same time 622. Is notified to the packet information management device 21 by the data reception detection signal.
[0050]
Upon receiving the data reception detection signal (1) 622, the packet information management device 21 selects the serial / parallel converter 611 according to the device select signal 619, and sends the cache transmitted from the node 100 via the data bus 620 to the internal register. Read the maintenance request packet. Then, the information such as the address in the packet, the number of transfer bytes, and the requesting node number is stored in a portion of the microcontroller that functions as the cache memory management device 24.
[0051]
If the cache information management device 24 holds the data of the address 01000000h in the cache memory for each node in order to maintain the consistency of the cache memory based on the information provided to itself, To instruct to invalidate the packet, a cache memory invalidation packet as shown in FIG. 7 is created and written to the parallel / serial converter 627. However, in this case, the connection destination node field of the packet includes a node number that does not actually exist. The written packet is converted into an optical signal by 628 and distributed to all nodes through the star coupler 50.
[0052]
Subsequent operations are the same as in the previous example, and will not be described.
[0053]
In the embodiment described so far, the arbitration signal paths 110 and 31, 210 and 32, 310 and 33, and 410 and 34 in FIG. 1 assume physically different signal paths. If these lines can be logically separated, they may physically pass through the same signal path (ie, the same optical fiber). However, in this case, it is necessary that λ1, λ2, λ3, and λc have different wavelengths in order to prevent interference in wavelength multiplexing.
[0054]
【The invention's effect】
As described above, according to the present invention, a plurality of sets of nodes can be simultaneously connected between a plurality of nodes, and a plurality of sets of nodes can be connected at the same time by a connection path that connects the lights of a plurality of wavelengths by optical wavelength multiplexing. Part or all of the information necessary for data transfer is transmitted from the node to the arbitration unit using the arbitration signal path, and part or all of the transmitted information is arranged on the connection path, Is distributed to each node through the data distributor and the connection path, and in each node, the distributed information is reflected in the cache memory inside the node, thereby maintaining the data consistency in the cache memory between the nodes. , It is possible to increase the computation efficiency of the node and the utilization efficiency of the connection path, thereby realizing a higher performance information processing system.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a functional configuration of an information processing system according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating an address map of the system according to the embodiment.
FIG. 3 is a diagram illustrating an arbiter interface of a node according to the embodiment;
FIG. 4 is a diagram showing a configuration of an arbitration request packet.
FIG. 5 is a diagram showing a configuration of an arbiter of the embodiment.
FIG. 6 is a diagram showing a configuration of a connection preparation request packet.
FIG. 7 is a diagram showing a configuration of a cache memory invalidation packet.
FIG. 8 is a diagram illustrating a configuration of a cache maintenance interface unit.
FIG. 9 is a diagram illustrating a configuration of a connection path interface unit.
FIG. 10 is a diagram showing a configuration of a cache maintenance request packet.
[Explanation of symbols]
Reference Signs List 20 arbiter 21 packet information management device 22 route selection information management device 23 additional information management device 24 cache information management device 30 concentrators 31, 32, 33, 34 optical fiber 35 cache maintenance optical signal path 50 star couplers 100, 200, 300, 400 Node 101, 201, 301, 401 CPU (processor)
102, 202, 302, 402 Memory 103, 203, 303, 403 Connection path interface circuit 104, 204, 304, 404 Arbiter interface circuit 105, 205, 305, 405 Node internal bus 106, 206, 306, 406 Cache memory 107, 207, 307, 407 Cache memory maintenance interface circuit 108, 208, 308, 408 Wavelength multiplexing device 110, 210, 310, 410 Arbitration signal path 130 Address driver 131 Data transfer sequencer 132 Control driver 133 Data buffer 134 Address drive signal 135 Acknowledge signal 136 control driver control signal 137 data buffer control signal 138 data reception signal 139 parallel Serial converter control signal 140 Address decoder 141 Node arbitration control processor 142 Address latch register 143 Control signal latch register 144 External access detection signal 145 Data signal line 146 Register select signal line 147 Device select signal line 148 Data reception signal 149 Data transmission / reception request signal Group 150 Write request detection signal 151 Internal bus data signal line 152 Internal bus control signal line 153 Internal bus address signal line 161, 165, 612, 614, 616, 618, 627 Parallel / serial converters 162, 166, 170 , 611, 613, 615, 617 Serial / parallel converters 163, 167, 602, 604, 606, 608, 628 Light emitting elements 164, 168, 16 9, 601, 603, 605, 607 Light receiving element 171 Cache maintenance packet reception signal 172 Cache control microcontroller 173 Device select signal 174 Cache maintenance request signal group 175 Data signal line 619 Device select signal 620 Data bus 621 Microcontroller 622, 623, 624, 625 Data detection signals 1, 2, 3, 4
626 control signal

Claims (8)

A plurality of nodes, each node having at least one processor, a cache memory, monitoring means for monitoring a signal on a bus inside the node, and an inside of the cache memory based on a monitoring result of the monitoring means. An information processing system comprising: an updating unit that updates a state,
By connecting by using light of a plurality of wavelengths and optical wavelength multiplexing, a plurality of connection paths can be simultaneously connected between the plurality of nodes,
Data distribution means arranged on the connection path and distributing data to each node through the connection path ;
Arbitration means for arbitrating the use request of the connection path;
An arbitration signal path for connecting between the arbitration means and each node;
Transmission means for transmitting a part or all of the information necessary for data transfer between nodes and between nodes, from the node to the arbitration means, using the arbitration signal path,
Distribution control means for controlling a part or all of the information transmitted by the transmission means to be distributed to each node through the data distribution means and the connection path from the arbitration means,
An information processing system, wherein each node is provided with reflection means for reflecting the information distributed by the distribution control means in a cache memory inside the node. The information processing system according to claim 1, wherein the data distribution unit includes a star coupler. 2. The information processing system according to claim 1, wherein the node-to-node connection path and the arbitration signal path are optically wavelength multiplexed and configured by a common optical fiber. 2. The information processing system according to claim 1, wherein the protocol for maintaining data consistency of the cache memory in the node is a write-through invalidation type protocol. A plurality of nodes, each node having at least one processor, a cache memory, monitoring means for monitoring a signal on a bus inside the node, and an inside of the cache memory based on a monitoring result of the monitoring means. An information processing system comprising: an updating unit that updates a state.
An arbitration step of arbitrating a request for use of a plurality of simultaneously connectable connection paths by an arbitration unit by connecting the plurality of nodes by using light of a plurality of wavelengths by optical wavelength multiplexing ,
Using an arbitration signal path that connects between the arbitration unit and each node, a part or all of information necessary for data transfer within a node and between nodes is transferred from the node to the arbitration unit, A transmission step of transmitting using an arbitration signal path;
A data distributor that distributes part or all of the information transmitted in the transmission step from the arbitration unit on the connection path and distributes data to each node, and a distribution step of distributing the data to each node through the connection path; When,
A reflection step of reflecting the information distributed to each node in the distribution step in a cache memory inside the node. The information processing method according to claim 5 , wherein the data distributor distributes data using a star coupler. 6. The information processing method according to claim 5 , wherein a common optical fiber is optically wavelength-multiplexed and used as the inter-node connection path and the arbitration signal path. 6. The information processing method according to claim 5 , wherein the node uses a write-through invalidation protocol to maintain data consistency of the cache memory.

Images