JP2557175B2 - Computer system - Google Patents

Description

Detailed Description of the Invention

[0001]

FIELD OF THE INVENTION This invention relates to computers and computer systems, and more particularly to parallel array processors. According to the present invention, a parallel array processor can be incorporated on a single semiconductor silicon chip. The chip is the basis for a system capable of massively parallel processing of complex scientific research and business applications.

[0002]

2. Description of the Related Art First, terms used in this specification will be described.

ALU ALU is the arithmetic logic circuit part of the processor.

Array An array refers to an array of elements in one or more dimensions. An array can include an ordered set of data items (array elements), but in languages such as FORTRAN, those data items are identified by a single name. In other languages, the name of an ordered set of data items refers to an ordered set of data elements that all have the same attributes. In program arrays, dimensions are generally specified by number or dimension attributes. In some languages, array declarators specify the size of each dimension of the array, and in some languages the array is an array of elements in a table. In a hardware sense, an array is a collection of structures (functional elements) that are the same in a massively parallel architecture as a whole. An array element in data parallel computer processing is an element to which an operation can be assigned and which can execute a required operation independently and in parallel in a parallel state. In general, an array can be thought of as a grid of processing elements. By assigning partition data to each section of the array, the partition data can be moved in a regular grid pattern. However, it is possible to index the data or assign the data to any location in the array.

Array Director The array director is a unit programmed as an array control program. The array director serves as a master control program for a group of functional elements arrayed as an array.

Array Processor Array processors mainly include multiple instruction multiple data scheme (MIMD) and single instruction multiple data scheme (SIM).
D). In a MIMD array processor, each processing element in the array uses its own data to execute its own unique instruction stream. SIM
In a D array processor, each processing element in the array is
Limited to the same instruction via a common instruction stream. However, the data associated with each processing element is unique. There are other features of the preferred array processor of this invention. This is referred to herein as an enhanced parallel array processor and uses the abbreviation APAP.

Asynchronous Asynchronous means that there is no regular time relationship. That is, the relationship between the executions of each function is unpredictable, and there is no regular or predictable time relationship between the executions of each function. In a control situation, the control program addresses the location to which control is passed when data is waiting for the idle element being addressed. This keeps the operations in order, even though they do not match the time of any event.

BOPS / GOPS BOPS or GOPS is an abbreviation for the same meaning of 1 billion operations per second. See GOPS.

Circuit Switching / Store-and-Switch These terms refer to two mechanisms for moving data packets through a network of nodes. Store-and-forward is a mechanism by which a data packet is received at each intermediate node, stored in its memory, and then forwarded towards its destination. Circuit switching directs an intermediate node to logically connect its input port to an output port so that data packets do not go into the intermediate node's memory but pass directly through the node to its destination. It is a mechanism that enables it.

Cluster A cluster is a control unit (cluster control device),
Hardware connected to it (terminals, functional units,
Or a virtual component) and a station (or functional unit). As used herein, a cluster is a processor memory element (PM), also referred to as a node array.
E) of the array. Normally, a cluster has 512 PM
It has an E element.

The entire PME node array of the present invention consists of a set of clusters each supported by one Cluster Controller (CC).

Cluster control device A cluster control device is a device that controls the input / output operations of a plurality of devices or functional units connected to it.
The cluster controller is typically under the control of a program stored in, and executing on, the unit, as in the IBM 3601 financial institution communication controller.
It is fully controllable in hardware, as in the 3272 controller.

Cluster synchronizer A cluster synchronizer manages the operation of all or a part of a certain cluster, maintains the synchronous operation of various elements, and enables each functional unit to maintain a specific time relationship with the execution of a program. It is a functional unit to make it.

Control Device A control device is a device that directs the transmission of data and commands over the links of an interconnection network. The operation of the control device is controlled by a program executed by a processor to which the control device is connected or a program executed in the control device.

CMOS CMOS is an abbreviation for complementary metal oxide semiconductor technology. It is widely used in the manufacture of dynamic random access memory (DRAM). NMOS
Is another technique used in the manufacture of dynamic random access memories. In the present invention, the complementary metal oxide semiconductor is used.
The technology used to manufacture the processor (APAP) does not limit the scope of semiconductor technology used.

Dotting refers to joining three or more lead wires by physical connection. Most backpanel buses use this connection method. This term is
Related to the OR DOTS of the past, it is used here to identify multiple data sources that can be coupled onto the bus by a very simple protocol.

The concept of I / O zippers in the present invention can be used to implement the concept that an input port entering a node can be driven by an output port exiting a node or data coming from the system bus.
Conversely, the data output from one node can be used as an input to another node and system bus. It should be noted that the data output to the system bus and another node are not executed at the same time but in another cycle.

Dotting is used in the discussion of H-DOT, by which the two-port PE or PME or picket can be used for arrays of various configurations. Several topologies have been discussed, including 2D and 3D meshes, base 2N cubes, sparse base 4N cubes, sparse base 8N cubes.

DRAM DRAM is an abbreviation for dynamic random access memory, which is a common storage device used by a computer as a main storage device. However, the term DRAM is also applicable to use as a cache or memory that is not main memory.

Floating-Point Floating-point numbers are represented in two parts: a fixed-point part, or fractional part, and an exponent part to a radix or base on the promise. The exponent indicates the actual position of the decimal point. In typical floating point notation, the real number 0.0001234 is represented as 0.1234-3. Here, 0.1234 is a decimal part, and -3 is an exponent. In this example, the floating point radix or base is 10, representing an implicit fixed integer base greater than one. The exponent represented explicitly in the floating-point representation, or represented in the exponent in the floating-point representation, is raised to the power of this base and then multiplied by the fractional part to obtain the real number represented. Numeric literals can be represented either in floating point notation or in real numbers.

FLOPS This term refers to the number of floating point instructions per second. Floating point operations include ADD (addition), SUB (subtraction),
It involves MPY (multiplication), DIV (division) and often many other operations. The parameter of floating point instructions per second is often calculated using add or multiply instructions and can generally be considered a 50/50 mix. The calculation includes generation of an exponent part and a decimal part, and normalization of a necessary decimal part. The present invention can handle 32-bit or 48-bit floating point formats (which may be longer, but such formats were not counted in the mix). Multiple instructions are required when performing floating point operations with fixed point instructions (regular or RISC). Some people use a 10: 1 ratio when calculating performance, and a ratio of 6: 1.
Some studies have shown that 25 is more appropriate. Different architectures have different ratios.

Functional Unit A functional unit is a hardware, software, or both entity that can achieve a certain purpose.

G bytes G bytes refer to 1 billion bytes. 1 Gbyte / sec
That's 1 billion bytes per second.

GIGAFLOPS 10 ⁹ floating point instructions per second

GOPS and PETAOPS GOPS or BOPS have the same meaning of 1 billion operations per second. PETAOPS means 1 trillion operations per second, which is the potential of current machines. In the APAP machine of the present invention, these terms are BIP / G which means 1 billion instructions per second.
It is almost the same as IP. While some machines can perform multiple operations (ie, both add and multiply) with a single instruction, the present invention does not. Also, it may take many instructions to perform one operation. For example, the present invention uses multiple instructions to perform 64-bit operations. However, when counting the calculation, the logarithmic calculation was not performed. GO to describe performance
The use of PS was preferred, but it was not used consistently. MIP / MOP, BIP / BOP as units above it, and MegaFLOPS / Giga
FLOPS / TeraFLOPS / PetaFLOPS
Is used.

ISA ISA means a Set Architecture (architecture setting) instruction.

Link A link is a physical or logical element. A physical link is a physical connection for connecting elements or units, while a link in computer programming is an instruction or address that exchanges control and parameters between different parts of a program. In multiple systems, a link is a connection between two systems specified by a program code that identifies the link identified by its real or virtual address. Thus, a link typically includes the physical medium, any protocol, and associated equipment and programming. That is, the link is both logical and physical.

MFLOPS MFLOPS means 10 ⁶ floating point instructions per second.

MIMD MIMD is a system in which each processor in the array has its own instruction stream, and thus has multiple instruction streams, with multiple data arranged, one per processing element.
Used to refer to the processor array architecture that executes a stream.

Module A module is a discrete and identifiable program unit, or a functional unit of hardware designed for use with other components. An aggregate of PEs included in a single electronic chip is also called a module.

Node In general, a node is a junction of links. In a general-purpose array of PEs, one PE can be a node.
A node can also include a collection of PEs called modules. In the present invention, a node is formed from an array of PMEs, and this set of PMEs is called a node.
The node is preferably 8 PMEs.

Node array A collection of modules composed of PMEs is sometimes called a node array. It is an array of nodes made up of modules. Node arrays are typically
Although there are more than a few PMEs, the term encompasses a plurality.

PDE PDE is a partial differential equation.

PDE relaxation solution process PDE relaxation solution process is PDE (partial differential equation)
Is a method of solving. Solving PDEs uses most of the computational power of supercomputers in the known field, and thus it exemplifies the mitigation process. PD
There are many ways to solve the E equation, and several numerical solutions include the relaxation process. For example, when solving a PDE with the finite element method, most of the time is spent computing the relaxation. Consider the example of the field of heat transfer. Given the hot gas inside the chimney and the cold breeze outside, what is the temperature gradient inside the brick of the chimney? Considering bricks as small segments and writing an equation that describes how heat flows between the segments as a function of temperature difference, the heat transfer PDE is transformed into a finite element problem. Now, assuming that all elements, except the inner and outer elements, are at room temperature and the boundary segments are the temperature of the hot gas and cold wind, a problem is created to initiate the relaxation.
The computer program then models the time by updating the temperature variable within each segment based on the amount of heat flowing into or out of the segment. In order to relax a set of temperature variables in the chimney to represent the actual temperature distribution that occurs in the physical chimney, it must take many cycles to process all the segments in the model. If the goal is to model gas cooling in a chimney, the elements must be extended to the gas equation, then the inner boundary conditions are linked with another finite element model and the process continues. Note that the heat flow depends on the temperature difference between adjacent segments. Therefore, the temperature variable is distributed using the communication path between PEs. It is this near-neighbor communication pattern or characteristic that makes the PDE relationship well applicable to parallel computing.

Picket This is an element within the array of elements that make up the array processor. This element is a data flow (ALU
REGS), memory, controls, and the parts of the communication matrix associated with this element. This unit refers to 1 / n of an array processor that consists of parallel processor elements and memory elements and their control and part of the array intercommunication mechanism. Pickets are a form of processor memory element (PME). The PME chip design processor logic of the present invention may implement the picket logic described in the related application or have the logic for a processor array formed as a node.
The term picket is similar to the commonly used array term PE for processing elements, and is preferably a combination of processing elements and local memory for processing bit parallel bytes of information in a clock cycle. Consists of,
It is an element of the processing array. The preferred embodiment consists of a byte wide data flow processor, 32 bytes or more of memory, a primitive control mechanism, and a mechanism for communicating with other pickets.

The term "picket" comes from Tom Sawyer and his white fence. However, it will also be understood that, functionally, it is similar to the army's picket line.

Picket Chip A picket chip contains multiple pickets on a single silicon chip.

Picket Processor System (or Subsystem) A picket processor consists of an array of pickets, a communication network, an input / output system, a microprocessor, a canned routine processor, and a microcontroller executing the array. It is a total system including a SIMD control device.

Picket Architecture The picket architecture is a preferred embodiment of the SIMD architecture and has features that can address a variety of different issues, including: -Set associative processing-Parallel numerical central processing-Image-like physical array processing

Picket Array A picket array is an ordered array of pickets arranged in a geometric order.

PME or Processor Memory Element PME stands for Processor Memory Element. The term PME is used herein to refer to a single processor, memory, and I / O capable system element or unit forming one of the parallel array processors of the present invention. PME is a term that encompasses pickets. A PME is a processor, memory associated with it,
1 / n of the processor array that consists of the control interface and part of the array communication network facility. This element can comprise a PME with regular array connectivity, such as in a picket processor, or a PME as part of a sub-array, such as in the multiple PME node described above.

Routing The routing is the allocation of a physical route for delivering a message to a destination. A source and a destination are required for route allocation. These elements or addresses are
Has a temporary relationship or affinity. Routing of messages is often based on keys obtained by consulting a table of assignments. Within a network, a destination is any station or addressable unit of a network that is addressed as the destination of information to be transmitted by a routing address that identifies a link. The destination field identifies the destination with the message header destination code.

A processor array architecture in which all processors in a SIMD array are instructed to execute multiple data streams arranged one per processing element from a single instruction stream.

SIMD MIMD or SIMD / MIMD SIMD MIMD or SIMD / MIMD is a term that refers to a machine that has the dual function of switching from MIMD to SIMD for a certain amount of time to process complex instructions, and thus has two modes. is there. Placing a Connection Machine Model CM-2 from Thinking Machines, Inc as the front end or back end of a MIMD machine allows the programmer to configure multiple modes, also known as dual mode. I was able to run different parts of the problem that were running. These machines have been around since ILLIAC and use a bus to interconnect a master CPU with other processors. The master control processor has the ability to interrupt the processing of other CPUs. Other CPUs can execute independent program code. During the interrupt, something needs to be done for the checkpoint function (close and save the current state of the controlled processor).

SIMIMD SIMIMD is a processor array architecture in which all processors in the array are commanded by a single instruction stream to execute multiple data streams arranged one per processing element. Is.
Within this configuration, the data dependent operations within each picket that mimic instruction execution are controlled by the SIMD instruction stream.

This uses the SIMD instruction stream to order multiple instruction streams (one per picket) and multiple data streams (one per picket).
A single instruction stream machine capable of executing SIMIMD can be performed by the PME system.

SISD SISD is an abbreviation for single instruction single data.

Swapping Swapping refers to the interchange of the data content of one storage area with the data content of another storage area.

Synchronous operation Synchronous operation in MIMD machines is a mode of operation in which each action is associated with some event (usually a clock). This event can be a specified event that occurs regularly in the program sequence.
Operations are dispatched to a number of processing elements, each of which independently performs a function. Control is not returned to the controller until the operation is complete.

If the request is for an array of functional units, a request is made by the controller to an element in the array, which element must complete operation before control is returned to the controller.

TERAFLOPS TERAFLOPS means 10 ¹² floating point instructions per second.

VLSI VLSI is an abbreviation for very large scale integration (applied to integrated circuits).

Zipper A zipper is a newly provided function for establishing a link from a device outside the normal interconnection of an array configuration.

The prior art as the background of the present invention will be described below. Engineers are constantly pursuing faster computers, linking hundreds and sometimes thousands of low-cost microprocessors in parallel
They are building supercomputers to solve complex problems that are out of hand with today's machines. Such machines are called massively parallel machines. The inventors have developed a new method for building a massively parallel system. The numerous improvements we have made should be considered against the background of numerous studies by others.

Multiple computers operating in parallel have existed for decades. Early parallel machines included the ILLIAC, which started in the 1960s. The ILLIAC IV was built in the 1970s. Other multiprocessors (see abstract US Pat. No. 4,975,834) include Cedar, Sigma-1 and the Butterfly and the Mo.
narch), Intel ipsc (the Intel ipsc), Connection Machines (The Connection Machines), Caltech COSMIC (the Caltech COSMIC), N cube (the N Cube), RP3 of IBM, GF1 of IBM
1. NYU Ultra Computer (theNYU Ultra Com
puter) and the Intel Delta and Touchstone.

Large multiprocessors starting with ILLIAC are considered supercomputers. The most commercially successful supercomputers are based on multiple vector processors,
Cray Research Y-MP System, IBM 3090, and Amdah
Machines from other manufacturers such as l), Hitachi, Fujitsu, and NEC are representative.

Massively parallel processors (MPP) are currently
It is considered capable of becoming a supercomputer. These computer systems assemble a number of microprocessors through an interconnect network and program these microprocessors to operate in parallel. There are two modes of operation for these computers. That is, there are MIMD mode machines and SIMD mode machines. The most commercially successful of these machines is Connection Machines Series 1 and 2 from Thinking Machines.
Is. This is basically a SIMD machine. Many large parallel machines use parallel interconnected microprocessors to achieve concurrency, or parallel operating capability. Intel microprocessors such as the i860 have been used by Intel and others. N-Cube has built a massively parallel machine using the Intel ^ 386 microprocessor. Other machines are built using so-called "transputer" chips. Inmos Transput
er) IMS T800 is one example. The Inmos Transputer T800 is a 32-bit device with an integrated high speed floating point processor.

To give an example of the type of system to be constructed, for example, a plurality of inmos transputers T8
00 chips each have 32 communication link inputs and 32
Have link outputs. Each chip has a single processor, small memory, a communication link to local memory, and a communication link to an external interface. Furthermore, in order to complete the system, IMS C01
A communication link adapter such as 1 or C012 is connected. In addition, switches such as the IMS C004 can be used, for example, as a crossbar switch between 32 link inputs and 32 link outputs to provide additional transputer
Make a point-to-point connection between chips. In addition, special circuits and interface chips for transputers are used and adapted for special purposes tailored to the requirements of special devices, graphics controllers, or disk controllers. Inmos IMS
The M212 is a 16-bit processor with one on-chip memory and multiple communication links.
The processor contains the hardware and logic to control the disk drive and can be used as a programmable disk controller or general purpose interface. Because it uses concurrency (parallel operation),
Inmos uses O, a special language for transputers.
ccam was developed. The programmer needs to describe the network of transputers directly in the Occam program.

Some of these massively parallel machines are
It uses a parallel processor array consisting of processor chips interconnected in various topologies. The transputer adds an IMS C004 chip to form a crossbar network. Other systems use hypercube connections. Some systems use a bus or mesh to connect the microprocessor and its associated circuitry. Some systems are interconnected by circuit switched processors and use the switch as a processor addressable network.
Last fall, Lawrence Livermor
In e), the processor-addressable network is to be considered as a coarse multiprocessor, as in the case of 14 RISC / 6000 interconnected by wiring the machines together.

Several large machines have been built by Intel, N-Cube and others to address the so-called "far-reaching challenges" in data processing. However, these computers are extremely expensive. Computers funded for development by the US Government to address "far-reaching challenges" have recently estimated costs of about $ 3,000-75 million (teracomputers)
Is. These "far-reaching challenges" include climate modeling, turbulence, pollution dispersion, human gene and ocean current mapping, quantum chromodynamics, semiconductor and supercomputer modeling, combustion systems, and visual recognition.

To add to the background of the present invention,
One should be aware of one of the earliest massively parallel machines developed by IBM. The term PME rather than "transputer" is used herein to describe one of the eight or more memory units with processors and I / O functions that make up an array or node of PMEs within a chip. use. Prior art "transputer" referenced
Has one processor on a chip, a FORTRAN auxiliary processor, and a small memory and input / output interface. The PME of the present invention is generally applicable to transputers and RP3 PMEs. However, as will be seen below, the small chip of the present invention differs in many respects. The small chip of the present invention has many functions described below. However, we are aware that the term PME was coined to refer to another PME that became more typical nowadays, which was the basis of the massively parallel machine known as RP3. are doing. RP3 (IB
The M research parallel processing prototype) was an experimental parallel processor based on a multiple instruction multiple data (MIMD) architecture. RP3 is a T.M. J. Designed and built at the Watson Research Center in collaboration with the New York University Ultracomputer Project. This work was supported in part by the US Department of Defense Advanced Research Projects Agency. RP3
Consisted of 64 processor memory elements (PMEs) interconnected with a high speed Omega network. Each PME has a 32-bit IBM "PC Scientifi
It had a c "microprocessor, 32 KB cache, 4 MB segment system memory, and I / O ports. PME I / O port hardware and software provided initialization, status acquisition, and shared I / O support processor (ISP). ) Supports communication between the memory and the processor. Each input / output support processor supports eight PMEs independently of the system network by an expansion input / output adapter (ETIO). , IBM S
/ 370 channel and IBM Token Ring network interface with operator monitor
Provide services. Each expansion I / O adapter was connected as a device to a PME ROMP storage channel (RSC) and provided programmable PME control / status signal I / O via an ETIO channel. The ETIO channel is
A 32-bit bus that interconnects the ISP to eight adapters. The ETIO channel used a custom interface protocol supported by hardware on the ETIO adapter and software on the I / O support processor.

[0062]

The machine referred to herein as an advanced parallel array processor (APAP) is
The inventors believe that it is a dense parallel processor and that it is needed to address the problems of conventional designs. As mentioned above, many dense (and sparse) processors are built from point designs and off-the-shelf processors that use dedicated and shared memory and one of many possible interconnection schemes. . To date, all of these approaches have some design and performance limitations. The "solutions" have different goals, but each has its own problems. Existing parallel machines are difficult to program. Each parallel machine generally cannot fit into machines of varying sizes compatible with a set of applications. Each parallel machine is limited in its design by physical design, interconnects, and architectural issues.

Physical Problem: There are approaches to using separate chip designs for each of the various functions required for horizontal structures.
These approaches have limited performance due to chip crossing delay.

There is also a technique for vertically integrating various functions on a single chip. These approaches are physically limited by the number of logic gates that can be integrated on a chip that can be produced.
Its performance is limited.

Interconnection Problem: For dense parallel processors, a network interconnecting various processing functions is important. Processor designs using buses, meshes, and hypercubes have all been developed. Each of these networks has its own limitations on processing power. Bus designs limit both the number of physically interconnectable processors and network performance. The mesh design limits the performance of the network due to the large diameter of the network. Hypercube designs require a large number of interconnect ports on each node. Therefore, the number of processors that can be interconnected is
Limited by the physical I / O pins at the node. The hypercube structure is considered to be significantly better in performance than conventional bus and mesh structures.

Architectural problem: Processes suitable for densely parallel processors can be divided into two types. A process that can be functionally divided is a multi-instruction multi-data (MI
The MD) architecture tends to perform better. A process that is not functionally distinguishable but has multiple data streams is a single instruction multiple data (SI
The MD) architecture tends to perform better. Every application contains several processes of both kinds. System trade-offs are required to choose the architecture that best fits a particular application, but no single solution has yielded satisfactory results.

[0067]

The present invention develops a new method of building massively parallel processors and other computer systems by creating new "chips" and systems designed according to the new concept. did. The present invention is directed to such a system. The components described herein can be combined in a system of the invention to create a new system. These components can also be combined with existing technology.

Small CMOS about 14 × 14 mm of the present invention
DRAMs can be constructed similar to brick building walls and paving roads. This chip provides the structure necessary to build a "house" or complex computer system by connecting replicas.

In overview of the invention, four identical small chips, each with internal array functionality and external input / output BCI, with eight or more processors embedded in the memory, memory of 36 or more complex computers and Processing power is provided. All of these chips can be sized as a wristwatch due to compact hybrid packaging, and each chip dissipates only about 2 W, allowing it to operate at very low power. The inventors have developed a number of new concepts using this chip, and the concepts considered unique inventions are described in detail in the examples and claims. Systems that can be built using the computer system of the present invention range from small devices to large machines with PETAOP capabilities. Details regarding such chips can be found in the related patent applications. Some of the features described herein are also applied to the multiprocessor memory element (PME) parallel processor, with particular reference to the multiprocessor of the present invention in terms of chip design, and to less compact processing elements and pickets. Some features that apply are described.

By this design, the processor is
The trade-off between SIMD and MIMD is not necessary in this system as it allows dynamic switching between modes and SIMD modes. This eliminates many of the problems that application programmers on "hybrid" machines will run into. Moreover, this design allows some processors to be in SIMD or MIMD mode.

Extended Parallel Array Processor (APAP)
Is a dense parallel processor. APAP is composed of partitionable control and processing sections so that a configuration suitable for supercomputer processing by personal computing applications is satisfied. In most configurations, it connects to the host processor and supports offloading the host workload to each segment. Since the APAP array processing element is a general purpose computer, the type of workload offloaded depends on the capabilities of the host. For example, the APAP of the present invention may be a module of the IBM 3090 Vector Processor Mainframe. When connecting to a mainframe with high performance vector floating point capabilities, the offloaded task may be a sparse to dense matrix transformation. When connecting to a personal computer, the offloaded task may be a three-dimensional graphic processing centered on numerical calculation.

[Parallel Associative Processor Syste
In US patent application Ser. No. 07 / 611,594 entitled "m",
The idea of integrating computer memory and control logic into a single chip and replicating the combination within the chip to build a processor system from a single chip replication is described and referenced by necessity. I want to. This approach is continued and extended by the present invention to develop and manufacture only one type of chip to perform massively parallel processing functions and reduce performance due to reduced chip boundary crossings and shorter line lengths. An improved system results.

In US patent application Ser. No. 07 / 611,594 filed on Nov. 13, 1990, a one-dimensional input / output structure (basically linear input / output) is provided in a plurality of SIMD P in a chip.
It is described that the ME is attached to the structure and used, and reference should be made when necessary. In this embodiment, these concepts are extended to two dimensions or more. Next, a four-dimensional input / output structure including eight SIMD / MIMD PMEs per chip will be described. However, FIG.
0, FIG. 11, FIG. 17, and FIG. 18, it is possible to increase the number of dimensions or the PME per dimension more than this. The processing element of the present invention comprises a complete I / O system including data transfer interrupts and program interrupts. In the description of the preferred embodiment, mainly 8 SIMD / MIMD Ps per chip are used.
Take a preferred four-dimensional input / output structure with ME.
In our opinion, this structure is currently particularly advantageous. However, as described herein, this dimensionality,
Alternatively, the PME per dimension can be increased beyond this. In addition, most applications have prioritized and made inventions in higher dimensional areas with hypercube interconnects, particularly modified hypercubes described below. However, depending on the application, the two-dimensional mesh interconnection of chips can be applied to the task at hand. For example, for some military computers, a two-dimensional mesh is suitable and cost effective.

This specification and related patent applications describe in detail the picket processor and what is referred to herein as the Advanced Parallel Array Processor (APAP). The picket processor is a processor memory element (PM
Note that E) can be used. Picket processors are particularly useful in military applications where a very compact array processor is desired. In this regard, the picket processor differs somewhat from the preferred embodiment associated with the APAP or enhanced parallel array processor. However, there are similarities and the aspects and features provided by the present invention can be used on different machines.

The term picket refers to the 1 / nth element of an array processor consisting of the processor and memory and the communication elements contained therein for array intercommunication.

The picket concept also applies to 1 / n of the APAP processing array.

The picket concept can differ from APAP in terms of data width, memory size and number of registers. In a massively parallel implementation, which is an alternative to APAP,
The difference is that the PME in the APAP is part of the sub-array, while it is configured to have connectivity for 1 / n of the regular array. Both systems are SIM
DIM can be executed, but the picket processor is configured as a SIMD machine with MIMD in PE, so SIMIMD can be executed directly.
The D APAP configuration will perform SIMIMD using MIMD PEs that are controlled to emulate SIMD. Both machines use PME.

Either system can be configured as a parallel array processor containing an array processing unit for an array having N elements interconnected by an array communication network, with one of the processor arrays being
/ N is part of the processing element, its associated memory, control bus interface, and array communication network.

The parallel array processor has the capability of a dual mode of operation and can instruct the processing unit to operate in either or both modes, the processing unit having two for SIMD and MIMD operations. It is possible to move freely between the two modes, and when the SIMD is the mode of its organization, the processing unit can instruct each element to execute its own instructions in SIMIMD mode, and the MIMD When in the enforcement mode of organization, selected elements of the array can be synchronized to simulate MIMD execution. This is called MIMD-SIMD.

The parallel array processors of both systems provide an array communication network with paths for exchanging information between the elements of the array. Information movement can be directed in one of two ways:
In this method, the array controller dictates that all messages move in the same direction at the same time, so the moving data does not define its destination. In the second method, each message is self-routed and the header at the beginning of the message defines its destination.

A segment of a parallel array processor array has multiple copies of a processing unit provided on a single semiconductor chip, each copy being a part of the array and its part of the array communication network. Includes a portion associated with a segment and buffers, drivers, multiplexers, and control functions, allowing portions of that segment of the array to be seamlessly connected to other segments of the array to extend the array communication network .

A control bus or control path from the controller is provided for each processing unit and extends to each element of the array to control its activity.

Each processing element segment of the parallel array contains multiple copies of the processor memory elements. A processor memory element is contained within a single semiconductor chip, has one segment of the array, includes a portion of the array control bus and a register buffer, and controls the array segment contained within the chip. Support communication.

Both can perform mesh movements or routing movements. APAP typically implements a dual interconnect structure in which eight elements on a chip are related in one way and chips are related in another way. Programmable routing on the chip establishes links between PMEs as described above, but nodes can, and usually do, associate in different ways. On chip, basically a typical APAP configuration is a 2x4 mesh and the node interconnects can be routed sparse octal N-cubes. Both systems are PE (PM
There is an inter-PE communication path between E) and the matrix is 2
It can be composed of point-to-point paths.

With this background and perspective, the features and aspects of the present invention that are related to preferred embodiments of the invention will now be described in detail with reference to the drawings.

[0086]

The present invention will be described in detail below. 1 and 2
Is exemplified by Transputer T800 chip, Touchstone Delta (i860), N cube (^ 38
6) shows existing technology levels representing similar chips for machines such as 6). Comparing FIGS. 1 and 2 with the system developed in the present invention, by using the present invention,
It will be appreciated that not only can a system like a conventional system be greatly improved, but also a new and powerful system can be constructed, as will be described later. The conventional current microprocessor technology of FIGS. 1 and 2 makes heavy use of pins and memory. Bandwidth is limited and chip-to-chip communication reduces system performance.

This innovative new technique, illustrated in FIG. 3, combines a processor, memory, and I / O mechanism to combine multiple PMEs (each with a memory access delay) formed on a single low-power CMOS DRAM chip. Each uses all pins for networking,
8 or more 16-bit processors). This system may take advantage of the disclosed concepts referenced above and the inventive concepts described separately in the co-pending application of the present invention and applicable to the systems described herein.
Accordingly, these disclosures and applications are incorporated herein by reference. Grouping, autonomy, transparency, zipper interaction, asynchronous SIMD, SIMIMD, or SI
All of the inventive concepts such as MD / MIMD can be used with this new technology. It may also be used in prior art systems, but with less benefit, in combination with our conventional multiple picket processor.

This processor can be used in the picket system of the present invention. The basic idea of the present invention is that a memory processor having an embedded processor, a router, and an input / output mechanism.
Providing a new basic building block, a replicable brick, for a system, which is a unit, with the novel memory processor of the present invention. This basic building block is scalable. The basic system embodying the invention uses a 4 megabit CMOS DRAM. This system can be expanded to 16 Mbit DR
It can be used for larger memory configurations with AMS and 64 Mbit chips. Each processor is a gate
It is an array. Increasing the adhesion density allows more clock speed processors to be placed on the same chip, and the use of gates and additional memory results in each PME
Performance is improved. Scaling the one-part type provides a system framework and architecture that can perform well in the PETAOP range.

FIG. 3 illustrates a memory processor according to the preferred embodiment, referred to herein as a PME or processor memory element. This processor has eight or more processors. In the illustrated embodiment, there are eight processors. More chips can be added by expanding the chip (horizontally). The chip can hold the logic circuit and add cells to the DRAM
The memory can, and preferably does, grow linearly (vertically). In the figure, 16-bit wide data
CMOS with eight replicas of the Flow Processor
DR surrounding a field with a gate array gate
Sixteen 32-kilobit by 9-bit sections of AM memory are shown.

In this processor, IBM CMOS low power sub-micron IBM CMOS on silicon deposition technology is used, using specific silicon with trenches,
Large storage is provided on the surface of the small chip. IB
With the advanced semiconductor chip manufacturing technology of M, interconnections organized from memories and a plurality of processors are performed. However, it should be understood that the small chip described herein has about 4 megabits of memory. The small chip of the present invention can be used in larger memory sizes, each 9 bits wide, without altering the logic circuitry when 16 Mbit memory technology is stabilized, yields are improved, and ways to address defects are established. Designed for migration. With the development of photolithography and X-ray lithography,
The minimum feature size is well below 0.5 micron. Further advances are envisioned in the design of the present invention. These advances will allow the placement of very large memories with processing capabilities on a single silicon chip.

The device of the present invention is a 4 mega CMOS DRA.
M, which is considered the first general-purpose memory chip with large space for logic circuits. 32 kilobits x
A memory array is constructed with 16 copies of a 9-bit DRAM macro. This DRAM has a large surface area allocated for on-chip application logic and has 120K cells with triple level metal wiring. The processor logic cells are preferably gate array cells. The DRAM access time is 35 nanoseconds or less, which matches the cycle time of the processor. This CMO
In the S embodiment, a very effective processing element (picket)
Is provided, and the power dissipation of the logic circuit at that time is 1.3W. Each separate memory section of the chip is 32 kilobits x 9 bits (expandable without changing the logic circuit), CMOS gate
It surrounds the array gate field. This field represents a 120K cell and has the logic described with respect to the other figures. The memory is barriered to dissipate 9W of power with a separate power supply. By combining a large logic circuit and a large scale memory on the same silicon substrate, the problems associated with the electrical noise inconsistency between the logic circuit and the DRAM have been solved. Logic circuits tend to be noisy,
In memory, on the other hand, the noise must be fairly low in order to detect millivolt scale signals caused by reading the cells of the DRAM. In the present invention, with trench 3
Heavy metal layer silicon deposition is used to dedicate each separate barriered portion of the memory chip to memory and processor logic circuitry, providing separate power distribution and barriers to ensure consistency between logic circuitry and DRAM. To achieve.

Overview of the APAP System of the Preferred Embodiment: This new technique is introduced here in the following order: 1. Technology 2. Explanation of chip hardware 3. Networking and system construction 4. Software 5. application

In the first few sections, 4 mega DRAM low power C
The MOS chip is on the PMEDRAM chip after manufacturing,
Also, how to provide, as part of the chip, eight processors each of which supports the following functions will be described. 1.16 bits, 5 MIP data flow Independent instruction stream and interrupt handling 3.8-bit (and parity and control) wide external port and interconnection with three other on-chip processors

The present invention provides multiple functions integrated into a single chip design. This chip provides powerful and flexible PME functions sufficient for a chip with scaling capabilities to effectively perform processing, routing, storage, and three types of I / O. In this chip, memory and control logic are integrated in a single chip to form a PME, and this combination is duplicated in the chip. The processor system is built from a single chip replica.

In this method, a low output CMOS DRAM is used.
Divide. Low-power CMOS DRAMs are formed as sections of multiple word length (16) bits by 32 kilobits, with each section associated with a processor (the term PME refers to a single processor, memory, and input / output system unit. Point). This partition makes each DRAM an 8-way "cube-connected" MIMD parallel processor with independent interconnect ports that are 8 bytes wide (replicating dense parallel technology replicating, showing the potential for replication and ring torus). See FIG. 7 for an example).

The software description addresses several different program types. At the lowest level, the process adapts the user program (or the service called by the application) to the detailed hardware requirements. This level contains the tasks necessary to manage I / O and interprocessor synchronization and can be called a microprogram for MPP. For intermediate level services, MPP mapping applications (developed using vector and matrix operations)
Control functions, synchronization functions, startup functions, and diagnostic functions can be used. At the host level, high-level languages are supported by library functions that support vectorization programs by simple automatic data allocation to the MPP or user-coordinated data allocation. Multi-level software techniques allow applications to take advantage of varying degrees of control and optimization within a single program. Thus, the user can code the application program without understanding the architectural details, and the optimizer will only tune the small, heavily used kernel of the program at the microcode level.

1024 Element 5G IPS Device and 327
Some sections describing 68-element 164GIPS devices show the range of possible systems. However, they are not limiting and smaller and larger devices are feasible. These particular sizes were chosen as examples for small devices such as microprocessors (accelerators), personal computers, workstations, and military applications (of course using different packaging technologies).
Suitable for large scale devices to demonstrate that it is an example of a mainframe application as a module or a complete supercomputer system. The software description provides examples of other challenging tasks that can be effectively programmed with each example system.

PME DRAM CMOS-Multiprocessor PME Basics: FIG. 3 shows an improvement of the invention at the chip technology level. This expandable computer organization uses only one type of chip,
Very cost and performance efficient over a wide range of system sizes. The combination of memory and processing on a single chip eliminates the need for pins dedicated to the memory bus, and thus the reliability and performance penalties associated with such pins. Duplicating the inventive design in a chip makes custom logic design for the processor subsection economically feasible. Since the chip is duplicated in the system, the manufacturing cost can be significantly reduced. Finally, CMOS technology requires lower power per MIP, thus minimizing power supply and cooling needs. Because the chip architecture can be programmed for multiple word lengths, operations that would otherwise require a much longer processor can be performed on this system. Together these attributes enable a wide range of system performance.

The new technique of the present invention can be compared with a case where the old technique partially common therewith is expanded. Obviously, feature miniaturization has been used by processor designers to increase the complexity of chips and memory designers to expand the replication of simple elements. If this trend continues, it is expected that the memory will be quadrupled, the processor density will be improved, and the following can be realized. 1. It has multiple execution units with instruction routers. 2. Increase cache size and related functionality. 3. Increase instruction look-ahead and improve computing capabilities.

However, even if these techniques are tried by the conventional technique shown in FIG. Duplicating processors increases the number of pins required proportionately, but the number of pins per chip remains fixed.
Improving cache behavior only improves the application's data reuse pattern. Above that, memory bandwidth is the limit. Application data dependencies and branching limit the potential benefits of read-ahead schemes. Also, it is not clear whether MPP applications with dense parallelism require 1 megaword memory, 4 megaword memory, or 16 megaword memory per processor. When trying to share such a large amount of memory among multiple processors, the limitation due to the memory bandwidth becomes severe.

There is no deadlock with the novel method of the present invention. As shown in FIG. 3 and subsequent figures, the present invention combines a large scale memory and input / output mechanism to form a single chip. The novel method of the present invention reduces the number of parts required,
There is no delay associated with chip crossing. More importantly, the novel method of the present invention allows the I / O pins of all chips to be dedicated to interprocessor communication to maximize network bandwidth.

To implement the preferred embodiment shown in FIG. 3, IBM low power CMOS technology is used, using currently available processes. This embodiment can be implemented in complementary metal oxide semiconductor (CMOS) with CMOS DRAM density and can be implemented with denser CMOS. In this embodiment, as the density of CMOS increases, the 32 kilobit memory of each of the 8 PMEs on the chip can be increased. In this embodiment, 4 mega CM
This is extended by using real estate and process technology for OS DRAM and associating a replica of the processor with 32K memory in the chip itself. It will be appreciated that in each chip package of the cluster shown in FIG. 4, the chip has a processor, memory, and I / O. Within each package is a memory, embedded processor element, router, and I / O, all of which are considered the first general-purpose memory chip with plenty of space for logic circuits, a 4M CM
It is in OS DRAM. This chip uses specific silicon with trenches to provide large storage areas on a small chip surface. Alternatively, each processor of the present design is constructed from 16 replicas of a 32 kilobit by 9 bit DRAM macro (35/80 nanoseconds) and a memory array is constructed using 0.87 micron CMOS logic. It can also be configured. This device
It is unique in that it allocates surface area for 120K cells of application logic on the chip and supports it with the function of triple level metallization. On the left side of FIG. 4, a prior art card is shown with an X.

The replicable building block brick technology of the present invention is the answer to the old technology. "X" on the left side of Fig. 4
A review of the marked technologies shows that there are too many chips and cards, which is a waste. For example, the teraflop machine proposed by other inventors today is
It literally has over a million chips. In today's other technologies, only a few percent of these chips actually work, and the rest are "overhead" (usually memory, network interfaces, etc.).

It will be appreciated that it is not possible to package such a large number of such chips into ones that need to operate in a physical size constrained environment (how many mounts in a small cockpit). Can you do that?). Furthermore, although the teraflop machine proposed by another inventor is already large, it reaches 1000 times scale to reach the petaflop range.
I have to get up. We have a solution that dramatically reduces the proportion of non-working chips. The present invention provides this within a reasonable network dimensionality. With this brick technology, the memory becomes the operator, the network is used for control exchanges, and the number of working chips is significantly increased. In addition, the upgrade dramatically reduces the number of chip types. The system of the present invention is designed to scale up without special packaging, cooling, power, or environmental constraints.

In the brick technology of the present invention, the processor,
Instead of having separate memory with embedded processor and network functionality, the configuration shown in FIG. 4 is used. This configuration represents a card with a chip that is pin compatible at the connector level with current 4 Mbit DRAM cards. With such a card, a basic 40 MIPS design point per chip performance level could hold 32 chips, enabling 1280 MIPS. Four such cards will provide 5G IPS. The workstation configuration shown is such a PE memory
IBM RISC with sufficient performance to execute and monitor array processor applications developed on arrays, cluster controllers, and workstations
It is preferable to have a System / 6000.

In the processor part, a processor having a very high gate efficiency can be used. Such designs have been used for processors, but never in memory. Further, in the present invention, the MIMD basic operation and SIMD
It provides the ability to mix basic movements. The chip of the present invention provides a "broadcast bus" that provides an alternate path for the instruction buffer of each CPU. The cluster control device of the present invention issues a command to each processing element in the PME. By storing these commands in the PME, the operation of the processing element can be controlled in multiple modes. Each PME does not have to store the entire program, only the portion that applies to a given task at various times during the processing of an application.

Given a base device, one processor and memory combination can be developed.
Alternatively, a design for making 2, 4, 8, or 16 PME replicas is also possible by using a simpler subset of processors and memory macros. Adjusting the data flow bandwidth or replacing the function accelerator with processor cycles can further simplify the PME. In most embodiments, it is preferable to duplicate the basic PME described above eight times.

According to an application study by the inventors, the most preferable method at present is a 16-bit wide data flow and 32K words of memory.
It is to duplicate twice. The reasons for this conclusion are as follows. The 1.16 bit word allows fetching instructions and addresses in a single cycle. If each of the 2.8 PMEs has an external port,
Four-dimensional torus interconnection is possible. 4 on each ring
Using 8 or 8 PMEs will result in modules suitable for the range of targeted system performance. 3. Approximately 50% of chip pins in 8 external ports
And the remaining chip pins are sufficient for power, ground, and common control signals. Implementing 4.8 processors in 64 KB of main memory: a. A register-based architecture is available instead of a memory mapped architecture. b. Some accelerators that are preferable but not necessary,
It can be enforced by multiple processor cycles.

This last attribute is important because it can increase the logic density under development. The novel accelerator of the present invention (eg, floating point arithmetic for PMEs) is added as chip hardware without affecting system design, pins and cables, or application code.

The resulting chip layout and size (14.59 × 14.63 mm) is shown in FIG. FIG. 4 shows a cluster of such chips. This chip is a system for stand-alone devices, a workstation
A workstation, AW placed adjacent to the host
It can be packaged in AC applications and supercomputers. This chip technology
It offers several advantages at the system level. This makes it possible to develop an MPP that can be scaled by a one-part type basic replication. DRAM per processor
Two macros provide sufficient storage for both data and programs. SRAMs of equal size can consume ten times more power. This advantage allows the MIMD machine model to be used rather than the more restrictive SIMD model that is typical of machines with single chip processor / memory designs. DRAM access times of 35 nanoseconds or less are consistent with expected processor cycle times. CMOS logic circuit
The PME provides a very effective logic density and the power dissipated therein is only 1.3 W (total chip power 1.3 + 0.9 (memory) = 2.2 W). These features allow the chip to be used in military applications where conduction cooling is required (air cooling in non-military applications is much easier). However, air-cooled embodiments may be used for workstations and other environments. Stand-alone
The processor can be configured with an 80A-5V power supply.

Enhanced Parallel Array Processor (APAP)
5 and 6 are shown in FIGS. FIG. 5 shows the APAP
FIG. Multiple application interfaces 150, 160, 170, 180
Exist for the application processor 100 or the processors 110, 120, 130. Figure 6
Shows the basic building blocks that can be configured into various system block diagrams. APAP can incorporate 32768 identical PMEs in a maximum configuration. APAP
Is an application processor interface 260 for a PME array 280, 290, 300, 310, array director 250, and application processor 200 or processor 220, 230.
Consists of The array director 250 consists of three functional units: an application processor interface 260, a cluster synchronizer 270, and a cluster controller 270. The array director 250 implements the functions of the array controller of our previous linear picket system for SIMD operation with MIMD functionality. Cluster controller 27
0 is a set of 64 array clusters 280, 290,
Together with 300, 310 (ie, a cluster of 512 PMEs), they are the basic building blocks of the APAP computer system. The elements of array director 250 allow systems with extensive cluster replication to be configured. This modularity, which is based on exact replication of processing and control elements, is a unique property of this massively parallel computer system. In addition, application processor interface 260 supports test / debug device 240, which performs important design, debug, and monitor functions.

Controllers, including controllers with i860, are assembled with well-defined interfaces such as the IBM Microchannel used in other systems today. The field programmable gate array adds functionality to the controller that can be modified to meet the requirements of a particular configuration (number of PMEs present, their combination, etc.).

PME arrays 280, 290, 300, 3
10 has the functions necessary to operate as a SIMD device or a MIMD device. These arrays also provide the ability to partition an entire set of PMEs into 1 to 256 different subsets. When dividing into subsets, the array director 250 interleaves between the subsets. The order of the interleaving processes and the degree of control exerted on each subset is programmatically controlled. Different subsets of array with different programs in one mode, MIM
This ability to operate in the D mode and the other set in the tightly synchronized SIMD mode under the control of the array detector is an advance in the art. The following examples demonstrate the benefits of this concept.

Array Architecture: A set of nodes forming an array are connected as an n-dimensional modified hypercube. In this interconnection scheme, each node is
It is directly connected to n nodes. The connections can be simplex, half-duplex, or full-duplex paths. In any dimension that has more dimensions than three dimensions, modified hypercubes are a new concept in interconnect technology. Generate a folded back-to-back grid on the surface).

An inductive description is necessary to describe the interconnection method when the number of dimensions is more than three. 1 set m ₁
Individual nodes can be interconnected as a ring (rings can have "simple connections", "braids", "cross connections", "full connections", etc. Although required, the modified hypercube structure is not affected by such increased complexity).
By connecting the equivalent nodes in a set of m ₂ rings, m ₂ rings can be linked together. The torus is obtained at this point. To construct an i + 1 dimensional modified hypercube from an i dimensional modified hypercube, assume m _{i + 1} sets of i dimensional modified hypercubes,
All equivalent mi-level nodes are interconnected to form a ring.

Using m _i = 8 (i = 1.4), 4
Figure 7 illustrates this process in the dimensionally modified hypercube.
Shown in This explanation under node topology,
Compare FIG. 7, FIG. 10, FIG. 11, FIG. 17, and FIG.

FIG. 7 shows a single processor element 300 consisting of a 32K 16-bit word memory and a 16-bit processor, with eight processors 312 and associated memory 311, the complete distribution associated with the latter. Type input / output router 313 and signal input / output port 31
4, a dense parallel technology path leading to a network node 310 consisting of 4,315, and further through a group of nodes represented by a cluster 320 to a cluster configuration 360 and various applications 330, 340, 350, 370. Two-dimensional level structure is cluster 32
0, and 64 clusters are integrated, 32768
4D modified hypercube 36 consisting of individual processing elements
0 is formed.

Preferred Embodiment of Processor Memory Element (PME): As shown in FIGS. 3 and 12, the preferred APAP has a basic building block consisting of one chip node. Each node has eight identical processor memory elements (PMEs) and one broadcast / control interface (BCI). Although some of the present invention can be implemented without all the functions on the same chip, from the point of view of performance and cost reduction, using the above-mentioned advanced technology that can be implemented now, eight It is important to form the chip as one chip node with a PME.

The preferred embodiment of the PME is 64 KB of main memory, 16 16-bit general purpose registers for each of the 8 program interrupt levels, a full-function logic unit (ALU), working registers, status registers,
And four programmable bidirectional input / output ports. In addition, this preferred embodiment provides a SIMD mode broadcast interface with a broadcast / control interface (BCI). This interface allows external controllers (see the original application of the present invention and the description of the presently preferred embodiment of a nodal array and system with clusters) to decode PME instructions, memory addresses, and ALU data inputs. To be able to drive. This chip can perform the functions of a microcomputer that allows it to perform multiple parallel operations within it, and can be coupled to other chips within a multi-node system. In this case, the connection method can be an interconnection network, mesh network, or hypercube.
It may be a network, advanced and scalable, any of the preferred embodiments of the invention.

The PMEs can be interconnected as a series of rings or toruses in the preferred embodiment of the invention, which is scalable. Depending on the application, the nodes can also be interconnected as a mesh. In the preferred embodiment of the invention, each node has a P for each of the four tori.
It has two MEs each. The torus is indicated by W, X, Y, and Z (see FIG. 7). FIG. 12 shows the interconnection of PMEs within a node. 2 in each torus
Each PME has its external input / output port (+ W, -W, +
X, -X, + Y, -Y, + Z, -Z).
Within the node there are also two rings interconnecting 4 + n and 4-n PMEs. These inner rings are
Acts as a route for moving messages between external tori. In the preferred embodiment of the invention, the APAP can be a four-dimensional orthogonal array, so that the inner ring allows messages to be moved in all dimensions across the array.

The PME is a self-contained program memory microcomputer having a main memory device, a local memory device, an instruction decoder, a logical operation unit (ALU), a work register, and an input / output port. PME is M
In the IMD operation, the stored instruction is fetched from its own main memory and executed.
It has the ability to retrieve and execute commands via the CI interface. This interface allows intercommunication between controllers, PMEs, and other PMEs in a multi-chip system.

The BCI is the node's interface to external array controller elements and array directors. The BCI provides common node functions such as timers and clocks. It also provides masking of broadcast function for each node PME, physical interface and buffering for data transfer between broadcast bus and PME, and system status and nodal interface with monitor and debug elements. .

Each PME has a separate interrupt level supporting its respective point-to-point and broadcast interfaces. Data is stored directly in memory
Under the access (DMA) control mechanism, the data is input to or output from the PME main storage device. "i
The nput transfer complete "interrupt can be used by each interface to send a signal to the PME software indicating the presence of data. The status information can be used by the PME software to determine the completion of a data output operation.

Input / output “circuit switching mode” for each PME
And one of the four input ports can be directly switched to any of the four output ports without inputting data to the PME main memory. The selection of the "circuit switched" source and destination follows the control of the software running on the PME. The other three input ports continue to have access to PME main memory functions and the fourth input is switched to the output port.

Another type of I / O has data that must be broadcast to and collected from all PMEs and data that is too special to fit on the standard bus. The broadcast data includes a SIMD command, M
It contains IMD programs and SIMD data.
The data collected is primarily status and monitoring functions. Diagnostic and test functions are special data elements. Each node has one BCI in addition to a set of embedded PMEs. During operation, the BCI section monitors the BCI, sends broadcast data to the addressed PMEs, and collects broadcast data from the PMEs. The BCI uses a combination of enable masks and addressing tags to determine which broadcast information is what PM.
It is determined whether E is the target.

In the preferred embodiment of the present invention, each PME is capable of operating in SIMD or MIMD mode. S
In IMD mode, each command is sent from the broadcast bus to the PME via the BCI. The BCI continues to buffer each broadcast data word until all selected node PMEs have finished using the interface. This synchronization allows the data timing dependencies associated with the execution of SIMD commands to be addressed and the asynchronous operation to P
It can be executed by ME. In MIMD mode, each PME executes its own program from its own main memory. The PME is in SIMD mode by default. For MIMD operation, the external controller is typically P
When the ME is in SIMD mode, it broadcasts the program to each element and then instructs the PME to switch to MIMD mode and begin execution. Masking or tagging broadcast information allows different sets of PMEs to have different MIMs
It allows D programs to be included and / or a particular set of PMEs to run in MIMD mode while other sets of PMEs run in SIMD mode, or both. In various software clusters or partitions, each of these functions can operate independently of its operation in other clusters or partitions.

PME instruction set architecture (I
The operation of SA) depends on whether the PME is in the SIMD mode or not.
Slightly different depending on whether it is MD mode. Most ISA instructions perform the same operation regardless of mode. However, the PME does not perform branching or other control functions in SIMD mode, so some code points dedicated to those MIMD instructions are reinterpreted in SIMD mode,
Special operations such as searching for data that matches the broadcast data value in the main memory or switching to MIMD mode are performed.
The ME is ready to execute. Therefore, the system flexibility of the array is further expanded.

PME Architecture: Basically, the preferred architecture of the present invention is a 16-bit wide data flow, 32K 16-bit memory, special I / O ports and I / O switching paths, and the instruction set of the present invention.
It includes a PME with the control logic necessary to allow each PME to fetch, decode and execute the 16-bit instruction set provided by the architecture. The preferred PME performs the functions of a virtual router,
It therefore performs both processing and data router functions. In this memory organization, mutual addressing of memory between PMEs allows access to large random access memory and direct memory for PMEs. All individual PME memories can be local or can be divided into a local area and a shared global area by a program. The special controls and functions described herein allow for quick task switching and
It becomes possible to retain the program state information at the ME interrupt execution level. Although some of the functions provided by the present invention existed in other processors, there is no other example that is applied to management of interprocessor input / output in a massively parallel machine.
An example is the integration of the message router function into the PME itself. This eliminates the need to develop special router chips and special VLSI routers. It should also be noted that the present invention allows the functionality provided on a single chip to be distributed on multiple chips interconnected by metallization layers or the like to improve a massively parallel machine. . Furthermore, the architecture of the present invention allows a single node to run on a massively parallel supercomputer.
It is possible to utilize some of the concepts of the present invention at various levels, as it can be scaled to machine levels. For example, as shown below, the PME data flow of the present invention is very powerful, but works for a scalable design to work.

The processing memory elements (PMEs) form a fully distributed architecture for each of the PMEs of a node. All PMEs are 16 bit
Data flow processing function, 64 KB local storage, store and switch / circuit switch logic, PME-to-PME communication, S
It consists of an IMD / MIMD switching function, programmable routing, and dedicated floating point assist logic. These functions can be independently manipulated by the PME and can be integrated with other PMEs within the same chip to minimize chip crossover delay.
The PME data flow is shown in FIGS. PM
E is a 16-bit wide data flow 425, 435,
445, 455, 465, 32 kbit x 16 bit memory 420, special input / output ports 400, 410, 4
80, 490, and input / output switching paths 425, 16
The PME retrieves and decodes the bit reduced instruction set,
Control logic 430 necessary to enable execution,
It is composed of 440, 450 and 460. Also, the special logic function allows the PME to run as both the processing unit 460 and the data router. Special control mechanism 40
5, 406, 407, 408 and functions are incorporated to allow quick task switching and hold program information instructions at each interrupt execution level of the PME. Although such a function was built into other processors, there is no other example where it is applied to management of input / output between processors in a massively parallel machine. Specifically,
This allows the router function to be integrated into the PME without special chips or VLSI development macros.

16-bit internal data flow and control: Figure 8 illustrates a significant portion of the internal data flow of processing elements.
Shown in FIG. 8 illustrates the internal data flow of processing elements. This processing element handles all 16-bit wide internal data
It has flows 425, 435, 445, 455, 465. The key pathways for these internal data flows are:
P register 450, M register 440, WR register 4
70, program counter PC register 430, etc. 1
It uses two nanosecond hard registers. Fully distributed AL from these registers in all operations
Data flows to the U460 and I / O router registers and special control mechanisms 405, 406, 407, 408.
With current VLSI technology, the processor is 25 MH
Memory operations and instruction steps can be performed in z, and important elements such as OP register 450, M register 440, WR register 470, program counter PC register 430, etc. can be constructed with 12 nanosecond hard registers. The other required registers are mapped into memory locations.

As shown in FIG. 9, internal data of PME
The flow has 32 kilobits by 16 bits of main memory in the form of two DRAM macros. The rest of the data flow consists of CMOS gate array macros. All memories are low power CMOS DR
It is integrally formed with the logic circuit by AM attachment technology to form a very large scale integrated PME chip node. In the preferred embodiment of the node chip, the PME is replicated 8 times.
The PME data flow is a 16 word by 16 bit general purpose register stack, a multifunctional logic unit (ALU) for buffering memory addresses, working registers, memory output registers, ALU output registers, arithmetic / operation
It consists of a command input / output register and a multiplexer for selecting the inputs to the ALU and registers. Using the current CMOS VLSI technology for 4MB DRAM memory and the logic circuit of the present invention, PME
Will be able to execute instruction steps at 25 MHz.
In the present invention, the OP register 450, the M register 440,
The WR register 470 and the general purpose register stack are formed of 12 nanosecond hard registers. The other required registers are mapped to memory locations within the PME.

The PME data flow is designed as a 16-bit integer arithmetic processor. A special multiplexer path has been added to optimize the subroutine emulation of nx16 bit floating point operations (n≥1). This 16-bit data flow enables effective emulation of floating point operations. Special paths in the data flow are built in to allow floating point operations in 10 cycles. The ISA has special code points that enable subroutines for extended (longer than 16 bits) operand operations. Floating point performance thereafter is about one-twentieth of the fixed floating point performance. This performance is
It suffices to eliminate the special floating point chip that assists the PME, which is typical of other massively parallel machines. Some other processors have a special floating point processor on the same chip as the single processor (see Figure 1). A special floating point hardware processor could be used on a chip with a PME, but this would require additional cells in addition to those currently required in the preferred embodiment. (For floating point arithmetic, refer to the above-mentioned "floating point Implementation on a SI" regarding the improvement of the IEEE standard.
Please refer to the US patent application entitled "MD Machine" if necessary.)

The technique developed in the present invention can directly use the usual increase in VLSI technical performance. As circuits become smaller and package density increases, it becomes possible to move data elements such as base registers and index registers currently mapped in memory into hardware. Similarly, the addition of hardware accelerates the floating point substeps. this is,
Reliable density level will increase, so CMO under development
Preferred for SDRAM technology. Very importantly, this hardware approach does not affect software.

The PME is initially set to SIMD mode with interrupts disabled. The command is BCI to PME
It is sent to the instruction decoding buffer. Each time the instruction operation is completed, the PME requests a new command from the BCI. Similarly, immediate data is requested from the BCI at the appropriate point in the instruction execution cycle. Most ISA instructions perform the same operation whether the PME is in SIMD mode or MIMD mode. However, this is not the case when the SIMD instruction and immediate data are taken out from the BCI. MI
In the MD mode, the PME uses the program counter (P
C) and use it as an address in its own memory to fetch the 16-bit instruction. program·
Instructions such as "Branch" that explicitly address the counter have no meaning in SIMD mode, and some of their code points are reinterpreted to allow special SIMD functions such as comparing immediate data with areas of main memory. Is executed.

SIMD by PME instruction decoding logic circuit
Either operating mode or MIMD operating mode is enabled and the PME can dynamically transition between modes. S
In IMD mode, the PME receives the decoded instruction information and executes that data on the next clock cycle.
In MIMD mode, the PME maintains a program counter (PC) address and uses it as an address in its own memory to fetch 16-bit instructions. Instruction decoding and execution proceeds as in most other RISC-type machines. The SIMD mode PME enters the MIMD mode when given the information representing the decryption branch. M
A PME in IMD mode goes into SIMD mode when it executes a specific instruction for migration.

When the PME makes a dynamic transition between SIMD mode and MIMD mode, SIMD "write c"
Executing an "control register" (control register read) instruction enters MIMD mode and sets the relevant control bit to "1." Upon completion of the SIMD instruction, the PME enters MIMD mode to enable interrupts and its general purpose. The MIMD instruction is fetched from the main memory location specified by the register R0 and execution is started.
The interrupt is masked or unmasked depending on the state of the interrupt mask when the IMD control bit is set. P
ME is initialized from the outside or MIM
D "write control register"
When the (control register write) instruction is executed and the relevant control bit is set to 0, the SIMD mode is returned to.

Data Communication Paths and Controls: Returning to FIG. 8, each PME has three input ports 400 and three output ports 480 for on-chip communication and one input / output port 410, 490 for off-chip communication. Have. Existing technologies other than this concept require the off-chip port to be byte-width half-duplex. The input port receives data from the input to the memory or the input AR register 40.
5 is routed directly to the output register 408 via the 16-bit data path 425. The memory becomes the data sink for messages destined for the PME or moved in "store-and-forward" mode. Messages that are not destined for a particular PME are sent directly to the desired output port, initiating "circuit switched" mode when blocking is not occurring. The PME software is responsible for performing routing and determining the selected transmission mode. This allows for dynamic selection between "circuit switched" and "store and switch" modes. This is another unique feature of PME designs.

Thus, the preferred node of the present invention is 8
Have four PMEs, each PME having four output ports (left,
Right, vertical, and external). 3 out of input ports
Three of the output ports are 16 bit wide full duplex point-to-point connections to other PMEs on the chip. The fourth port, in the preferred embodiment, is combined to provide a half-duplex point-to-point connection to the off-chip PME. Due to the pin and power supply constraints imposed to utilize low density CMOS in the present invention, the actual off-chip interface is a byte width used to multiplex two halfwords of data words between PMEs. It is a route. Dynamically, temporarily, and logically destroy the inter-mode ring, allowing data to enter and leave the array,
When using special "zipper" circuits, these external PMEs
The port provides the APAP external I / O array function.

For data routed to PME memory, regular DMA so that the PME instruction stream need only participate in I / O processing at the beginning and end of the message.
Is supported. Finally, the data that is circuit switched to the internal output port is transferred without clocking. This allows a single cycle of data transfer within the chip and detects when a chip crossing occurs that allows for the fastest but still reliable communication. Forward data paths and reverse control paths are used for high speed transfers, and all transfers are in transparent mode. In short, the originator goes through several stages before being acknowledged by the PME performing the DMA or on-chip transfer.

As can be seen from FIGS. 8 and 9, the PME
The data on the input port can be destined for the local PME or for the PME further down the ring. Data destined for the PME further down the ring is transferred to the local PME.
After being stored in the main storage device, it can be transferred from the local PME to the target PME (store-and-switch), or the local input port can be logically connected to a specific local output port (circuit-switched) so that the data is stored in the local PME. Can also be passed "transparently" to the target PME. The local PME software provides the local PME for any of the four inputs and four outputs.
To dynamically control whether the mode is "store and switch" mode or "circuit switch" mode. In circuit switched mode, PME
Handles all functions at the same time, except for the I / O associated with circuit switching. In store-and-forward mode, the PME suspends all other processing functions and starts the I / O transfer process.

Data can be stored in shared memory or DASD external to the array (by an external controller), but P
It can also be stored somewhere in the memory provided by the ME. Input data destined for the local PME or buffered in the local PME during a "store and forward" operation is stored in the local PME main memory via a direct memory access (address) mechanism associated with each input port. To be done. A program interrupt can indicate that the message has been loaded into PME main memory. The local PME program is a message that interprets the header data and determines whether the data addressed to the local PME is a control message that can be used to set a circuit-switched path to another PME, or transfers it to another PME. Or not. The circuit switched path is controlled by the local PME software. The circuit switched path logically directly couples the PME input path with the output path without passing through an intervening buffer store. PME on the same chip
Since there is no intervening buffer storage in the output path between, data can be put into the chip, passed through multiple PMEs on the chip, and loaded into main memory of the target PME in a single clock cycle. it can. Intermediate buffer storage is only needed when the circuit switched coupling leaves the chip. This reduces the effective diameter of the APAP array by the number of unbuffered circuit switched paths. as a result,
Data can be sent from the PME to the target PME in as few clock cycles as there are intervening chips, regardless of the number of PMEs in the path. This type of routing can be compared to a switching environment where each node cycle requires several cycles to transfer data to the next node. Each node of the present invention has 8 PMEs.

Memory and Interrupt Level: The PME stores 32 kilobits by 16 bit words in memory 420. This storage is completely general purpose and can contain both data and programs. In SIMD operation, all memory can be data. This is characteristic of other SIMD massively parallel machines. M
In IMD mode, the memory is quite normal, but unlike most massively parallel MIMD machines, PM
Since it is on the same chip as E, it can be used immediately. This eliminates the cache operation and cache coherency techniques typical of other massively parallel MIMD machines. For Inmos chips, only 4K resides on the chip and requires an external memory interface bus and pins. In the present invention, these are unnecessary.

The lowest storage location is used to provide a set of general purpose registers for each interrupt level. PME
Certain ISAs developed for use short address fields for these register references. Interrupts are used to manage processing, I / O activity, and software-specified functions (ie, PME switches to interrupt level when incoming I / O starts during normal processing).
If the interrupt level is not masked, this switch is performed by changing the hardware pointer so that the register is accessed from a new section of lowest memory and swapping a single PC value. This technique allows fast level switching, allowing the software to avoid normal register save operations and save the situation in the interrupt level register.

The PME processor operates on one of eight program interrupt levels. Memory addressing allows the lower 576 words of memory to be partitioned into eight levels of interrupt. Sixty-four of the 576 words of this memory are directly addressable by the program executing at any of the eight levels. The other 51
The two words are divided into eight 64-word segments. Only the program running at the interrupt level associated with it can directly access each 64-word segment. The use of direct addressing techniques allows all programs to access the entire 32K word of PME memory.

Interrupt levels are assigned to input ports, BCIs, and error handling mechanisms. There is a "normal" level, but no "privilege" or "supervisor" level. Program interrupts cause context switching to occur, PC program counters, status / control registers,
And the contents of certain general purpose registers are stored in the specified main storage location, and the new values of these registers are
Fetched from another designated main storage location.

The PME described with reference to FIGS. 8 and 9.
The data flow can be extended with reference to the following sections. In complex systems, the PME data flow uses a combination of chips as array nodes and memory, processors, and I / O facilities. The input / output mechanism exchanges messages by using the BCI that is duplicated as a basic unit of the MMP constructed by the APAP of the present invention. MMPs can handle many word lengths.

PME Plural Data Flow Processing: The system described herein is capable of performing operations processed by current processors with a 16-bit wide data flow within the PME. To do so, an operation is performed on the data length, which is a multiple of 16 bits. Therefore, the operation is performed as a 16-bit fragment. It may be necessary to know the result of each fragment (ie whether the result was 0 or the high order bit of the sum was rounded up).

An example of a data flow is the addition of two 48-bit numbers. In this example, two 48-bit numbers (a (0-47) and b (0-4) are executed by performing the following operations in hardware.
7)) is added.

A (32-47) + b (32-47)-> ans (32-47) -Step 1 1) Save the execution result of the upper bits of the total. 2) Memorize whether the partial result was 0.

A (16-31) + b (16-31) + save carry-> ans
(16-31)-Step 2 1) Save the execution result of the upper bits of the total. 2) Memorize this result and whether the partial result was 0 in the previous step. If both are 0, 0 is stored. If either is non-zero, non-zero is stored.

A (0-15) + b (0-15) + saved carry-> ans (0
-15)-Final step 1) If this fragment is 0 and the last fragment is 0, the answer is 0
Is. 2) If this fragment is 0 and the last fragment is non-zero, the answer is non-zero. 3) If this fragment is non-zero, the answer will be positive or negative (assuming no overflow) depending on the sign of the sum. 4) If the sign of the executed answer is not equal to the sign of the executed answer, then the answer has the wrong sign and the result is an overflow (cannot be properly represented in the available bits).

The length can be extended by repeating the intermediate second step as many times as necessary. If the length is 32,
The second step is not executed. If the length is greater than 48, step 2 is executed multiple times. Exactly 1 in length
In the case of 6, the operation in step 1 is executed under the conditions 3 and 4 of the final step. Extending the length of an operand to multiple times the length of the data flow
If the width of the flow is narrow, it usually takes a long time to execute the instruction. That is, when adding 32 bits in a 32-bit data flow, the adder logic circuit is set to 1
Only one pass is required, but if the 16-bit data flow also adds 32 bits, it will need to pass through the adder logic twice.

It is interesting to note that in the current implementation of the machine, add / subtract / compare / move operations are performed on operands of length 1-8 words (length is defined as part of the instruction). There is a single instruction that can be executed. The individual instructions available to the programmer perform the same types of operations as those shown in steps 1, 2, and the final step (although the operand length is longer for the programmer, ie 16-128 bits). At the basic hardware level, it works on 16 bits at a time, but programmers think they are processing 16 to 128 bits at a time.

When these instructions are used in combination,
The programmer can handle operands of arbitrary length. That is, two instructions can be used to add two numbers up to 256 bits in length.

PME Processor: The PME processor of the present invention differs from the current microprocessors currently used in MPP applications. The differences between the processors are as follows.

1. The processor is a fully programmable hardwired computer (see ISA description for instruction set overview) • Has complete memory module shown in upper left corner (see Figure 9). -Has hardware registers with the controls needed to emulate a separate register set for each interrupt level (shown in the upper left corner). The logic unit has the registers and controls necessary to enable efficient multi-cycle integer and floating point arithmetic.・ PME interconnected by a point-to-point link shown in the upper right corner
It has the necessary input / output switching paths to support packet or circuit switched data movement between.

2. This is the smallest technique for processor design that can make 8 duplicates of PME per chip by CMOS DRAM technology.

3. This processor portion of the PME is approximately the minimum required to code the high speed instruction set architecture (ISA) (see table) required to enable the efficient MIMD or SIMD operation of the MMP of the present invention. Provides the data flow width of the.

PME Resident Software: PME is the smallest element of APAP that can execute stored programs. The PME resides in certain external control elements and the SI
Broadcast / control interface (BCI) in MD mode
It is possible to execute a program sent to the PME by means of, or to execute a program resident in its own main memory (MIMD mode). PME is SIM
It is possible to switch dynamically between D mode and MIMD mode. This is a SIMD / MIMD mode dual function, and the system can execute this dual function simultaneously (SIMIMD mode). A particular PME can do this dynamic switching simply by setting or resetting a bit in the control register. The SIMD PME software actually resides on an external control element, so more details about this can be found in the discussion regarding array directors and related applications.

In MIMD software, PME is SIM
In the D mode, it is stored in the PME main memory. This is possible because many PMEs process the same data asynchronously and therefore have the same program. It is pointed out here that these programs are not fixed and can be modified by loading the MIMD program from an external source while processing other operations.

Since the PME instruction set architecture shown in the table is a microcomputer architecture, there are few restrictions on the functions that the PME can perform in this architecture. Basically, each PME can function like a RISC microprocessor. Typical MIM
The D PME software routines are listed below.

1. Basic control program for dispatching various resident routines and prioritizing them.

2. Communication software for exchanging data and control messages between PMEs. This software determines when a particular PME enters and exits "circuit switched" mode. The communication software suitably performs a "store and forward" function. It also initiates, sends, receives, and ends messages between its own main memory and another PME's main memory.

3. The interrupt handling software completes the context switch and responds to the event that caused the interrupt. These software include fail safe routines and rerouting or reassignment of PMEs to the array.

4. Routines that implement the extended instruction set architecture described below. These reroutes execute macro-level instructions such as extended precision fixed point arithmetic, floating point arithmetic, vector arithmetic. Thus, not only can complex operations be handled, but also image processing activities for displaying image data in multidimensional (two-dimensional or three-dimensional images) and multi-media processes.

5. Standard math library functions can be included. These functions preferably include LINPAK and VPSS routines. Each processor memory element can operate on different elements of a vector or matrix, so that the various PMEs all
Different routines or different parts of the same matrix can be executed at one time.

6. Special routines are provided for performing scatter / gather or sort functions that utilize the APAP node interconnect structure and allow for dynamic multidimensional routing. These routines allow asynchronous operation to continue while effectively utilizing the degree of synchronization achieved between the various PMEs. For classification, there is a classification routine. APAP is well suited for batcher classification. Because such a classification requires a large amount of computation to determine the particular element to compare with a very short comparison cycle. Program synchronization is managed by I / O statements. Using this program, 1
Multiple data elements per PME are possible and very large parallel classifications can be done in a very simple way.

Although each PME has its own resident software, a system built from these microcomputers can execute higher level language processes designed for scalar parallel machines. That is, this system is FORTRAN, C, C ++, FORTRA
A high-level language such as ND can execute application programs written for UNIX machines or application programs of other operating systems.

It is interesting that the processor concept of the present invention uses a very old approach to processor design. I
A similar instruction set architecture (ISA) design has probably been used for 30 years in BM's military processors. Recognizing that this type of design, when combined with the overall PME design of the present invention, will find a way into the current microprocessor design that has been stalled, opening up new avenues for use in the next century. This is the first time that the present inventors have done so.

Although the design features of this processor are quite different from other current microprocessors, similar gate-constrained military and aerospace processors have been using this design since the 1960s. The processor provides enough instructions and registers for the development of a simple compiler, and both general purpose and signal processing applications run effectively with this design. The design of the present invention has minimal gate requirements, and IBM has implemented a similar concept for several years when embedded chip designs were needed for general purpose processing. Since we have adopted some of the older ISA designs this time, many programmers have existing bases and knowledge of the design concept, and therefore a number of utilities and other utilities that allow us to quickly adopt our system. Software vehicles can be used.

PME input / output: PME is the route BC of FIG.
Interconnect to the bus by reading data from the Broadcast / Control Interface (BCI) bus into the logic unit via I or by fetching instructions directly from the bus into a decode logic circuit (not shown). PME is SIM
Power up in D mode, read, decode and execute instructions in that mode until a branch is hit. SIM
The D-mode broadcast communication command causes the transition to MIMD, and causes the local side to fetch the command. Broadcast communication PM
The E instruction ^ INTERNAL DIOW ^ reverses the state.

PME input / output can be data transmission, delivery, or reception. The PME sets the CTL register when transmitting data and sets XMIT to L, R,
Connect to either V or X. The hardware service then delivers a block of data from memory to the target via the ALU multiplexer and XMIT register. This process interleaves with normal instruction operations. Depending on the requirements of the application, the block of data to be transmitted may contain raw data for the defined PMEs and / or commands to establish a path. Upon receiving the data, the PME stores the input in memory and interrupts the active subprocess. Interpretation tasks at the interrupt level can use this interrupt event to perform task synchronization or initiate transparent I / O operations (when data is addressed elsewhere). The PME is free to continue execution during transparent I / O operations. The PME's CTL register makes the PME a bridge. Data passes through the PME without gating, and the PME remains in that mode until the CTL register is reset by the instruction or data stream. The PME cannot be the data source during the passing of data, but can be the data sink for another message.

PME Broadcast Section: This is the interface between the chip and the common control device. This interface can be used by devices that command I / O or act as controllers to test and diagnose complete chips.

The input is a word sequence (instruction or data) available to a subset of PMEs. Associated with each word is a code that indicates which PME uses that word. The BCI uses words to limit access to the interface and ensure that all required PMEs receive the data. This helps to tune the BCI for asynchronous PME operation (PME is asynchronous for I / O and interrupt handling even when in SIMD mode). This mechanism allows the PMEs to be formed into groups controlled by an interleaved set of command / data words received via the BCI.

In addition to passing data to the PME, the BCI also accepts request codes from the PME, combines those codes and sends out an integrated request. This mechanism can be used in several ways. MIMD processing can start in a group of processors all ending with an output signal. When the signal is ANDed, the controller starts a new process. Often the application is PM
Not all required software can be loaded into E-memory. Using coded requests to the controller, perhaps software from the host storage system.
Take out the overlay.

The controller uses a serial scan loop through multiple chips to retrieve information on individual chips or PMEs. These loops are initially interconnected with the BCI but can bridge to individual PMEs at the interface.

BCI: The Broadcast / Control Interface (BCI) on each chip implements a parallel input interface that allows data or instructions to be sent to the nodes. Incoming data is tagged with a subset identifier. The BCI provides the functionality necessary to ensure that all PMEs in a node operating in the subset are provided with data or instructions. The BCI's parallel interface acts both as a port that allows data to be broadcast to all PMEs and as an instruction interface during SIMD operation. The ability to satisfy both requirements and extend those requirements to support subset operation is unprecedented, except for the design approach of the present invention.

The BCI parallel input interface of the present invention allows data or instructions to be sent from a control element external to the node. The BCI has a "group allocation" register associated with each PME (for the concept of grouping, see Grouping of SIMD in the co-pending application).
See US patent application entitled pickets). Incoming data words are tagged with a group identifier. B
The CI provides the functionality necessary to ensure that data or instructions are provided to all PMEs in the node assigned to the dedicated group. The BCI parallel interface acts both as a port to allow data to be broadcast to the PME during MIMD operation and as an instruction / immediate operand interface during SIMD operation.

The BCI also has two serial interfaces. The high speed serial port gives each PME the ability to output a limited amount of status information. The purpose of this data is to:

1. The array director 61 signals that the PME, eg 500, has data that needs to be read, or that the PME has completed some operation.
Send to 0. Array director 610 passes the message to the external control element it represents. 2. It provides activity status so that external test and monitor elements can show the status of the entire system.

The standard serial port allows external control elements to selectively access a particular PME for monitoring and control purposes. Data passed through this interface can either be sent from the BCI parallel interface to a particular PME register or selected from a particular PME register and routed to the high speed serial port. These control points are set by the external control elements during the initial power-up and diagnostic phases to the individual P
Allows ME to be monitored and controlled. This allows the array director to input control data such that the output of the port is to a particular PME and node internal register and access point. These registers provide a path for the node's PME to output data to the array director, and allow the array director to input data to the device during the initial power up and diagnostic phases. Data input to the access points can be used to control test and diagnostic operations, such as execution of a single instruction step, stop on compare, breakpoint, and so on.

Node Topology: The modified hypercube topology connection of the present invention is most effective for massively parallel systems, but other poorly performing connections can be used with the basic PME of the present invention. VL by the present inventor
In the initial implementation of the SI chip, eight PMEs and fully distributed PME internal hardware connections are used. The internal PME-to-PME chip configuration is two rings of four PMEs, each PME having one connection to the PMEs of yet another ring. 8 PM on VLSI chip
If there is E, then it is a three-dimensional binary hypercube. However, our approach generally does not use hypercube organization within the chip. In addition, each PME
Allows the escape of one bus. In the initial embodiment, the bus escaped from one ring is + X,
Called + Y, + W, and + Z, the rings escaped from the other ring are similarly labeled- (minus).

A particular chip organization is called a node of the array, and nodes can be put into clusters of the array. The nodes are connected as an array using + -X and + -Y to form a cluster. The dimensionality of the array is arbitrary and is generally greater than the requirement of 2 for the development of binary hypercubes. The clusters are further connected using + -W, + -Z to form an array of clusters. Again, the dimensionality of the array is arbitrary. As a result, 4 nodes
A three-dimensional hypercube is obtained. To expand to a five-dimensional hypercube, ten PME nodes are required, and two additional buses, eg, + -E1, are used to connect the four-dimensional hypercube into a hypercube vector. The present invention further showed a pattern of extension to odd or even radix hypercubes. This modified topology limits the cluster to cluster wiring, but retains the benefits of hypercube connectivity.

The present invention's routability and topological configuration for massively parallel machines allows the X and Y dimensions to be maintained within the cluster level packaging of the present invention, with W bus connections to all adjacent clusters. And the Z bus connection can be distributed. After implementing the techniques described above, the product can be routed and manufactured while maintaining the unique characteristics of the defined topology.

A node consists of K ^* n PMEs and a Broadcast / Control Interface (BCI) section. Here, "n" represents the number of dimensions or rings that characterize the modified hypercube, and "k" represents the number of rings that characterize the node. A node can have k rings, but only two of those rings have
Providing an escape bus is a feature of this concept. In the preferred embodiment, the physical chip package limits "n" and "k" to N = 4 and k = 2. This limitation is physical and the use of different chip sets can increase the dimensionality of the array. The preferred embodiment of the present invention allows grouping of PMEs that are not only part of the physical chip package, but interconnect a set of rings in a modified hypercube. Each node has a PME architecture,
There are eight PMEs that can perform processing and data router functions. Therefore, n is the number of dimensions of the modified hypercube (see the next section). That is, the number of node elements of the 4-dimensional modified hypercube is 8 PMEs, and 5
The dimension modified hypercube has 10 PMEs. See FIG. 7 for nodes that can be used in the present invention, FIGS. 10 and 11 for interconnections, and FIG. 12 for a block diagram of each node. 17 and 18 show details of possible APAP interconnections.

“Metho, filed on May 13, 1991
d for Interconnecting and Systemof Interconnected
US Patent Application No. 07/6 entitled "Processing Elements"
No. 98866 describes modified hypercube criteria preferably used for the APAP MMPs of the present invention, which is hereby incorporated by reference. The above application describes a method of interconnecting processing elements so that the number of connections per element and the network diameter (worst case path length) can be balanced. To do so, one creates a topology with many of the well-known and preferred topological properties of a hypercube, while at the same time increasing the flexibility of the topology by enumerating the nodes of the network in a number system whose bases can change. Using a base-2 number system in this way yields a hypercube topology. The present invention has fewer interconnects than uniform connections in the hypercube and maintains the properties of the hypercube. There are the following three characteristics. 1) There are many alternative routes. 2) The total bandwidth is extremely large. 3) Other common problem topologies can be mapped with the topology of the network, using well-known existing methods. The result is a less dense non-binary hypercube. Note that although the present invention prioritizes the modified hypercube approach, some applications can use conventional hypercubes. Other topological approaches can be used to connect the nodes. However, the approach described herein is considered novel and sophisticated and will be prioritized.

An interconnect method for a modified hypercube topology for interconnecting multiple nodes in a PME's network is described below.

1.1 set of integers e1, e2, e3. . . The set of is defined as follows. The product of all elements equals the number M of PMEs in the network, while e1 and e
The product of all elements of the set, except 2, is the number N of nodes, and the number m of elements of the set defines the dimensionality of the network by the relation n = m-2.

2. 1 set of indexes a1, a2. . .
Address the PME located by am. Where each index is a PME position at an equivalent expansion level and the index ai is the formula (.... (a (m) ^* e (m-
1) + a (m-2)) e (m-1) ... a (2) ^* e (1)) + a (1)
Is 1, 2 ,. . . When m, it falls within the range of 0 to ei-1. In this formula, the notation a (i) means, as usual, that it is the i-th in the list a of elements. The same applies to e.

3. Two PMEs (addresses f and g) only if either of the following two conditions are met:
Connect. a. The integer part of r / (e1 ^* e2) is equal to the integer part of s / (e1 ^* e2). 1) The remainder of r / e1 differs from the remainder of s / e1 by 1.
Or 2) The remainder of r / e2 differs from the remainder of s / e2 by 1 or e2-1. b. The remainder of r / ei and the remainder of s / ei are i = 3,
4,. . . Unlike in the range of m, the remainder part of r / e1 is equal to the remainder part of s / e2 equal to i-3, and the remainder part of r / e2 differs from the remainder part of s / e2 by e2-1.

As a result, the computer system node is a non-binary
Form a hypercube. The node supports a PM that supports 2 ^* n ports such that the ports it provides match the dimensionality requirements of the modified hypercube.
Defined as an array of E. A set of integers e3, e that define a particular extent of each dimension of a particular modified hypercube
4,. . . consider that all ems are equal, eg b,
Assuming that e1 and e2 are a1, the immediately preceding formulas for addressability and connection are:

1. N = b ^** n

2. Here, PMEs are addressed with numbers that represent the base b number system.

3. Two computational elements (f and g) are connected only if the address of f differs from the address of g by only one base b digit. The rule is used that 0 and b-1 are separated by 1.

4. The number of connections supported by each PME is 2 ^* n.

This is as described in the basic application, and the number of communication buses connecting the non-adjacent PMEs is selected as 0.

Intra-node PME interconnect: PMEs are organized as a 2xn array within the node. 1 for each PME
Interconnection with three adjacent PMEs using a set of input / output ports provides full duplex communication between the PMEs. Each PME external input / output port is connected to a node input / output pin. The I / O ports can be connected so that the pins can be shared for half-duplex communication or separated for full-duplex functionality. The interconnection of the four-dimensional modified hypercube node is shown in FIGS. 10 and 1.
1 (note that if n is an even number, the node can be considered as a 2 × 2 × n / 2 array).

FIG. 10 shows eight processing elements 50 in a node.
0, 510, 520, 530, 540, 550, 56
0 and 570 are shown. PMEs are connected in a binary hypercube communication network. This binary hypercube shows three intra-node connections between PMEs (501, 511, 521, 531, 5).
41, 551, 561, 571, 590, 591, 59
2,593). Communication between PMEs is controlled by I / O registers under the control of processing elements. This figure shows eight directions, + -w 525, 565, + -x 515, 55.
5, + -y 505, 545, + -z 535, 575 show various paths that can be used when escaping input and output. If desired, communication can be performed without storing the data in memory.

If a network switching chip is used,
It should be noted that various cards, each having a chip of the present invention, can be connected to other chips in the system, although it is not necessary and desirable to use no network switching chip. The inter-PME network of the present invention, described as a "four-dimensional torus", is the mechanism used for inter-PME communication. A PME can access any other PME in the array on this interface (the intervening PMEs can be store-switched or circuit-switched).

Interconnect Chip Relationships: Chips have been described. FIG. 12 is a block diagram of a PME processor / memory chip. This chip is composed of the following elements. Each of these elements is described below.

1. 16-bit programmable processor and 32K-word memory (64KB) each
8 PMEs consisting of.

2. A BCI that allows the controller to run all PMEs or a subset thereof and accumulate PME requests.

3. Interconnect level a. Each PME supports four 8-bit wide inter-PME communication paths. These are connected to three adjacent PMEs on the chip and one off-chip PME. b. Bus connection between broadcast PMEs. Make data or instructions available. c. A service request line that allows any PME to send a code to the controller. BCI combines requirements,
Transfer the summary. d. The serial service loop allows the controller to read all the details about the functional block.
This level of interconnection extends from the broadcast interface to all PMEs (details are omitted as can be seen with reference to FIG. 12).

Interconnect and Routing: MPP, PM
Performed by replicating E. The degree of replication is
It does not affect the interconnections and routing methods used. FIG. 7 shows an outline of the network interconnection method. The chip has eight PMEs, which are interconnected to the next PME. This interconnection pattern results in the three-dimensional cube structure shown in FIG. Each processor in the cube has a dedicated bidirectional byte port to a pin on the chip. A set of eight PMEs is called a node.

An n × n array of nodes is a cluster.
Cluster node interconnection is achieved by simply bridging between the + x and -x ports and the + y and -y ports. In this case, the preferred chip or node of the present invention has eight PMEs, each managing a single external port. This distributes network control functions and eliminates bottlenecks on ports.
Bridging the outer edges makes the cluster a logical torus. Although we have considered n = 4 and n = 8 clusters in the present invention, n = 8 is more appropriate for commercial applications, while n = 8 for military conduction cooled applications.
I think that 4 is more appropriate. The inventive concept does not use an immutable cluster size. Conversely, applications that use transformations are also envisioned.

Arraying the clusters results in the four-dimensional torus or hypercube structure shown in FIG. Bridging between the + w and -w ports and between the + z and -z ports provides a four-dimensional torus interconnect. As a result, each node in the cluster is connected to an equivalent node in all adjacent clusters (which provides 64 ports between any two adjacent clusters, or 8 for larger clusters). Is). As with cluster size, this scheme does not require an array of specific size. In the present invention, a preferred 2x1 array for workstation and military applications,
4x4 array for mainframe applications,
A 4x8 array and an 8x8 array were considered.

The development of the four-dimensional torus is beyond the gate, pin, and connector limits of the preferred chip of the present invention. However, with the alternative on-chip optical driver / receiver of the present invention, this limitation is removed. In this example, the network of the present invention is 1 per PME.
The light path of the book can be used. In that case, P per chip
The number of MEs becomes 12 instead of 8, and a 4-dimensional torus array with multi-Tflop (teraflop) performance can be formed.
It seems possible and economically feasible to use such a machine in a workstation environment. It should be noted that such alternative machines use application programs developed for the presently preferred embodiment of the present invention.

4D Cluster Organization: To build the 4D modified hypercube 360 shown in FIGS. 7 and 11, a node supporting eight external ports 315 is required. External ports are + X, + Y, + Z, + W,-
It is represented by X, -Y, -Z, -W. Next, a ring can be constructed by connecting between the + X port and the -X port using the m ₁ node. Furthermore, using m ₂ , + Y
Interconnecting the ports and -Y ports interconnects such rings and allows the construction of rings of rings. This level of structure is called cluster 320. m ₁ = m ₂ = 8
In the case of, as shown in FIG. 7 with m = 8, 512 P
MEs are available and such clusters are the building blocks of multiple size systems (330, 340, 350).

Four-dimensional array organization: When building large dense systems, use + Z and -Z ports and interconnect sets of m ₃ clusters as columns. The m ₄ columns are further interconnected using the + W and −W ports. m ₁ =. . . If m ₄ = 8, then 3276
A system with 8 or 8 ^{4 + 1} PMEs is obtained. In this organization, not all dimensions need to be isopycnic (massive dense parallel processor 370) as in FIG. For small dense processors, only one cluster can be used and unused Z and W ports can be interconnected on the card. This technique saves card connector pins and allows this scalable processor to be applied to workstations 340, 350 and avionics applications 330, which have limited connector pins. If you connect Z and W in pairs and connect +/- ports, debug, test,
And a workstation organization that enables large-scale machine software development.

Furthermore, a much smaller scale version of the structure can be developed by generating the structure with values smaller than m = 8. In this way, a single card processor can be constructed that meets the accelerator requirements for PS / 2 or RISC system 6000 workstation 350.

Input / Output Performance: Input / output performance includes overhead for setup transfers and actual burst rate data movement. The setup overhead depends on the complexity of application function I / O and network contention. For example, an application can program circuit-switched traffic by buffering to resolve conflicts, or move all PMEs to the left for synchronization. In the first example, I / O is the main task and a detailed analysis is used to size it. According to the estimation by the present inventors, the setup overhead of the simple example is 20 to 30 clocks.
Cycle or 0.8-1.2 microseconds.

Burst rate input / output is the maximum rate at which the PME can transfer data (transfer to adjacent PME on chip or external adjacent PME). The memory access limit sets the data rate to 140 nanoseconds per byte. This corresponds to 7.14 MB / sec.
This performance includes buffer address and count processing, and data read / write. With this performance,
Seven 40 nanosecond cycles are used per 16-bit word transferred.

This burst rate performance corresponds to a cluster having a maximum transfer rate of 3.65 GB / sec. That is, a set of 8 nodes along a row or column of the cluster uses 57 sets of available ports for 57 MB /
Burst data rates of seconds will be realized. Logically "unzip" edges of wrapped clusters
However, this number is important because input / output with the outside is performed by connecting to the external system bus.

Inter-PME Routing Protocol: SIMD /
MIMD PME is an inter-processor communication mechanism with external I / O mechanism, BCI, and SIMD within the same PME.
It has a switching function that enables both operation and MIMD operation. PME includes processor communication and PME
It incorporates a fully distributed programmable I / O router for transferring data between.

The PME has fully distributed interprocessor communication hardware to the on-chip PME as well as external I / O mechanisms connected to the interconnected PMEs in the modified hypercube configuration. This hardware is
Complemented by the flexible programmability of the PME, which controls I / O activity via software. The programmable I / O router function enables the generation of data packets and packet addresses. PME
Can transmit this information in a specified way through the PME's network or by multiple paths as determined by fault tolerance requirements.

A distributed fault tolerance algorithm or program algorithm can take advantage of the programmability and the circuit switched modes supported by the PME. This performance combination mode makes it possible to implement all things such as an offline PME, an optimal path data structure, etc. via a programmable input / output router.

From our investigation of applications, it has been found that sending raw data between PMEs can be the most efficient. Applications may also need data and routing information. In addition, it may be possible to plan communications so that network collisions do not occur. In some applications, the possibility of deadlock is not eliminated unless a mechanism is provided for buffering messages at intermediate nodes. Two extreme examples are shown. In the relaxation phase of the PDE method, each grid point can be assigned to a node. The inner loop process of acquiring data from neighboring nodes can be easily synchronized across the nodes. Image transformation also uses local data parameters to determine the destination or source identifier of the communication. as a result,
Data is moved between multiple PMEs, causing each PME to participate in the routing task of each packet. Preplanning for such traffic is generally not possible.

In order to make the network efficient for all kinds of transfer requirements, we divide the responsibility of data routing between PMEs between hardware and software. The software performs most task ordering functions. In the present invention, a special function for performing internal loop transfer and minimizing software overhead on the outer loop is added to the hardware.

A dedicated interrupt level input / output program manages the network. In most applications, the PME dedicates four interrupt levels to receiving data from four adjacent PMEs. When opening a buffer of each level, the register of that level is loaded and an IN instruction (using buffer address and transfer count, but does not wait for data reception) and RETURN instruction pair is executed. The hardware also accepts words from a particular input bus and stores them in a buffer. In addition, a buffer full condition causes an interrupt and restores the program counter to the instruction after RETURN. This approach for interrupt levels allows you to write I / O programs that do not need to test what caused the interrupt. The program reads the data, returns, and then moves on to processing the read data. In most situations, little or no register saving is needed, so transfer overhead is minimal. If the application uses synchronization for I / O, as in the PDE example, then the program can provide that functionality.

The write operation can be initiated in several ways. P
In the case of the DE example, at the time of sending the result to the adjacent PME,
The application-level program makes the write call. The call provides the buffer location, word count, and destination address. The write subroutine contains the register load and OUT instructions needed to boot the hardware and return to the application. The hardware performs the actual byte-by-byte data transfer. For more complex output requirements, the highest interrupt level output service function is used. Both application-level tasks and interrupt-level tasks access their services through software interrupts.

The circuit switched path setup is based on these simple read and write operations. In the present invention, all PMEs first open a sized buffer to accept the packet header but not the data. PME that needs data transmission
Initiates the transfer by sending the address / data block to the adjacent PME whose address is closer to the destination address.
Address information is stored in the adjacent PME. A buffer full condition causes an interrupt. The interrupt software tests the destination address and either expands the buffer to accept the data or writes the destination address to the output port and also sets the CTL register for transparent data movement (this allows the PME to execute the application). Can be overlapped with circuit-switched bridging operations). The CTL register is busy and transparent until reset by a signal at the end of the data stream or an abnormal reset is performed in PME programming. The above contents can be changed in any number.

System I / O and Array Director: FIG. 13 shows the array director of the preferred embodiment. This array director can perform the functions of the controller of FIGS. 14 and 15 which describe the system bus and array connections. FIG. 14 shows a bus that communicates with the cluster, and FIG. 15 shows communication of information on the bus that communicates with the PME. Loading or unloading of the array is done by connecting the edges of the cluster to the system bus. Multiple clusters can support multiple system buses. 5 for each cluster
Supports 0-57 MB / sec bandwidth. Loading or unloading a parallel array requires moving data between all PMEs or a subset thereof and standard buses (ie Micro Channel, VME Bus, Future Bus). These buses are part of the host processor or array controller and are assumed to be strictly specified. Therefore, the PME array must fit into the bus. PME arrays can be matched to the bandwidth of any bus by interleaving bus data on n PMEs. Here, n is selected so that both PME input / output and processing time are possible. 14 and 15 show how to connect the system bus to the PME at the two edges of the cluster. Such an approach can support 114 MB / sec. It is also possible to load two edges simultaneously with data at half the peak rate. This reduces the bandwidth to 57 MB / s per cluster, but allows for orthogonal data movement within the array and allows the passing of data between the two buses (using this advantage. , Provides fast transposition and matrix multiplication operations).

As shown in FIG. 14, the bus is connected to all the paths on the edge of the cluster, and the controller generates the gate signal at the necessary interleaving timing for each path. System with speed exceeding 57MB / sec
Data needs to be interleaved across multiple clusters when needed to connect to the bus. For example, 200M
Systems that require a B / sec system bus use groups of two or four clusters. Large MPPs have the ability to connect 16 to 64 such buses to the xy network path. This number can be doubled by using the w and z paths as well as the x and y paths.

FIG. 15 shows how data is routed to individual PMEs. This figure shows 7.13MB
1 illustrates one particular w, x, y, or z path that can operate in burst mode at / sec. Only one PME is required if the data on the system bus occurs in bursts and the PME memory can contain a complete burst. In the present invention, PM is used so that neither of these conditions is necessary.
The E input / output structure was designed. Data can be gated into PMEx0 at maximum speed until a buffer full condition occurs. At the moment when the data full condition occurs, PME × 0 changes to transparent mode and PME × 1 starts accepting data. The processing of the input data buffer can be started within PME × 0. A PME that takes data and processes it has the limitation that it cannot transmit results while in transparent mode. The design solves this by switching the data stream to the opposite end at spaced intervals. As shown in FIG. 15, under the control of software, P
While ME × 12 to PME × 15 unload the results,
Dedicate PMEx0 to PMEx3 to receive data,
PM while MEx0-PMEx3 unload results
Ex12 to PMEx15 can be dedicated to receiving data. The controller counts the words and
Adding a block end signal to the stream switches the direction. The controller workload is reasonable because one count applies to all paths supported by the controller.

Alternate Computer System: FIG.
1 shows a system block diagram of a host-attached large scale system with a single application processor interface (API). This diagram can also be viewed with the understanding that the present invention can be used in a stand-alone system using multiple application processor interfaces (not shown). This configuration supports DASD / graphics on all clusters or multiple clusters. The workstation accelerator eliminates the need for the illustrated host, application processor interface (API), and cluster synchronizer (CS). A cluster synchronizer is not absolutely necessary. Whether it is necessary or not depends on the type of processing to be performed and the physical drive or power supply provided to the particular application using the invention. In this case, the pulse rise time of the control bus may be very long because the workload demands placed on the control device by the application, which mainly executes the MIMD process, are not very high. Conversely, higher speed control bus functions may be required in systems that primarily perform asynchronous A-SIMD operations due to a large number of independent groupings. In this case, a cluster synchronizer is preferred.

The system block diagram of FIG. 20 shows that the system can consist of a host, an array controller, and a PME array. A PME array is a set of clusters supported by a set of cluster controllers (CCs). Although one cluster controller is shown for each cluster, this relationship is not necessary. The actual ratio of cluster to cluster controller is flexible. The cluster controller has 64 BCI /
It provides a re-drive to the cluster and an accumulation from it. Therefore, physical parameters can be used to establish a maximum ratio. In addition, the cluster controller is PME
Multiple subsets of the array can be controlled individually, and this service can be a gating requirement. Investigations can be performed to determine these requirements for a particular application of the invention. There are two versions of the cluster controller used. Clusters that connect to the system bus require the cluster controller to provide interleaved controls (see "System I / O" and Figure 20) and tri-state drivers. A simpler version without the tri-state bus function can also be used. For large systems, a second stage redrive and accumulation is added. This level is the Cluster Synchronizer (CS). An array controller is composed of a set of cluster controllers and a cluster synchronizer and an application processor interface (API). The only programmable unit is the application processor interface.

A plurality of modifications can be made to this system integration method. These variations can form different hardware configurations for different applications, but doing so does not significantly affect the supporting software.

In the workstation accelerator, the cluster controller is directly attached to the workstation system bus. The functions of the application processor interface are performed by the workstation. For RISC / 6000, the system
The bus is a Micro Channel and allows the cluster controller to insert directly into a slot in the workstation. In this configuration, I / O devices (DASD, SCSI, and display interface) are placed on the same bus that loads / unloads the array. Therefore, parallel arrays can be used for I / O-intensive tasks such as real-time image generation and processing. Other bus systems (V
For workstations using ME Bus, Future Bus, etc., use the gateway interface. Such modules are readily available on the market. In these smallest systems, a single cluster controller can be shared between a certain number of clusters, and the cluster synchronizer also
No interface is needed.

MIL avionics applications may be similar in size to workstations, but require different interfaces. Think about what a normal military situation looks like. The existing platform must be expanded with additional processing capabilities, but funding cannot completely redesign the processing system. For this reason, the present invention connects a smart memory coprocessor to the APAP array. In this case, a special application that looks like memory to the host
A program interface (API) is provided.
The data addressed to host memory is then moved to the array via the cluster controller.
Subsequent writes to memory will be detected by the application program interface and interpreted as a command, thus exposing the accelerator to the memory-mapped coprocessor.

Large-scale systems can be developed as either host-connected or stand-alone configurations. The configuration shown in FIG. 20 is useful in the host connection system. The host performs I / O, and the application program interface acts as a dispatched task manager. However, large-scale stand-alone systems are possible in special circumstances. For example, the database search system eliminates the need for a host, connects DASD to Micro Channels of any cluster, and uses multiple application program interfaces as bus masters slaved to PMEs.

Zipper array interface with external I / O: The zipper of the present invention provides a high speed I / O connection scheme and is implemented by placing a switch between two array nodes. This switch allows parallel communication with the array. High speed I / O is performed along one edge of the array ring,
Functions as a large-scale zipper to the X ring, Y ring, W ring, and Z ring. This high-speed input / output is named "zipper connection". The ability to transfer data to and from the network while transferring data between processors by simply adding a switch delay is a unique loading technique. This switching scheme does not affect the ring topology created by the X-bus, Y-bus, W-bus, and Z-bus, and special support hardware allows the processing element to process or route data. The zipper operation can be executed.

If data can be exchanged at high speed with a large-scale parallel system, the performance of the entire system will be greatly improved. The inventors believe that the method of the present invention for performing high-speed input / output without reducing the number of processors or the dimension of the array network is unprecedented in the field of massively parallel environment.

Expanding the modified hypercube array,
Topologies with rings within rings are possible. Any or all of the rings can be logically destroyed when supporting interfaces to external I / O. The two ends of the broken ring can then be connected to the external I / O bus. Since it is a logical operation to destroy the ring, regular inter-PME communication is possible at fixed time intervals, and at the same time input / output is possible at other time intervals. This process of breaking a level of the ring in the modified hypercube effectively "unzips" the ring for input / output purposes. The high speed input / output "zippers" provide another interface to the array. The zipper can reside on 1 to n edges of the modified hypercube and can support either parallel input to multiple dimensions of the array, or broadcast to multiple dimensions of the array. Subsequent data transfers to and from the array can alternate between two nodes directly connected to the zipper. This I / O approach is unique and allows different zipper sizes to be developed to meet specific application requirements. For example, in the particular configuration shown in FIG. 7, called large dense processor 360, the Z and W bus zippers are connected to the MCA bus. This approach optimizes the matrix transpose time to meet the specific application requirements of the processor.
For more information on the "zipper" construction, see the co-filed "APA
See US patent entitled "PI / O Zipper Connection". The zipper is shown in FIG.

The size of the zipper can be varied depending on the configuration and the need for the program to exchange data and programs with individual processing elements. The actual speed of the I / O zipper is approximately equal to the number of rings connected, the PME bus width, and the PME clock speed divided by two (this division allows data to be transferred during the receive PME time). The output zipper can send data to any of the n locations, so I / O contention is completely absorbed throughout the array). The existing technology using a PME transfer rate of 5 MB / s, 64 rings on the zipper, and interleaving of two nodes, allows an array transfer rate of 320 MB / s (typical of FIG. 16). See zipper configuration). FIG. 16 illustrates high speed input / output, a so-called “zipper connection” 700, 710 that exists as another interface to the array. This zipper is arranged in multiple directions 770, 780, 790, 751, 75 at multiple nodes in arrays 751, 752, 753, 754.
5, 757 broadcast bus 720, 730, 740, 7
By connecting over one edge 700 or two edges 70 of the hypercube network.
0,710 can exist.

Today's MCA buses are 80-160M / s
Since it supports B transfer rates, it is well suited for simple mode or non-interleaved mode single zippers. However, the actual transfer rate has some channel overhead, so the efficiency is somewhat lower. For systems with much higher I / O requirements, multiple zippers and MCA buses can be used. These techniques appear to be important for processors that support large external storage combined with nodes or clusters that are characteristic of database machines.
Such I / O expansion capability is completely unique to such a machine, and has not been conventionally incorporated in a massively parallel processor, a conventional single processor, or a sparse parallel machine.

Array Director Architecture: The massively parallel system of the present invention consists of a node component of a multiprocessor node, a node cluster, and an array of PMEs already packaged in the cluster. To control these package systems, the present invention provides a system array director that performs the entire processor memory element (PME) array controller function in a massively parallel processing environment with a hardware controller. Array Director is an application interface, a cluster synchronizer,
And is usually composed of three functional areas called a cluster controller. The Array Director uses the broadcast bus and the zipper connection of the present invention to send data and commands to all PMEs for total control of the PME array. The Array Director functions as a software system that acts as a shell for the operating system by interacting with the hardware. The array director, in playing this role,
It receives commands from the application interface and issues the appropriate array instructions and hardware sequences to perform specified tasks. The main function of the Array Director is to keep the traffic high and the collision low by continuously sending commands to the PME and routing the data in an optimal sequence.

The APAP computer system shown in FIG. 7 is shown in more detail in FIG. FIG. 13 shows an array director that can function as a controller or array controller as shown in FIGS. 14 and 15 and FIGS. 19 and 20. This array director 610 shown in FIG. 13 has n identical array clusters 665, 670, 680, 690 and 512 P
A typical implementation of APAP is shown as a typical configuration of an array director 610 for a cluster of MEs and an application processor interface 630 for an application processor 600.
The cluster synchronizer 650 provides the necessary sequences to the cluster controller 640, and the cluster synchronizer 650 and the cluster controller 640 form an “array director” 610. Application processor interface 630 supports host processor 600 and test / debug workstations. For APAP devices connected to one or more hosts, the array director 610
Acts as an interface between the user and the array of PMEs. In APAP, which acts as a stand-alone parallel processing machine, the array director 610 becomes a host device and thus becomes involved in device I / O activity.

The array director 610 is composed of the following four functional areas (see the functional block diagram of FIG. 13). 1. Application processor interface (API) 600 2. Cluster Synchronizer (CS) 650 (8x8 array of clusters) 3. Cluster controller (CC) 640 (8 × 1 of nodes
Array) 4. High-speed input / output (zipper connection) 620

Application Processor Interface (API) 630: When operating in connected mode, one application processor interface 630 is used for each host. Application processor
Interface 630 monitors the incoming data stream for array clusters 665, 670, 680, 6
Which instruction to 90, and high-speed input / output (zipper) 6
Decide which data for 20. Stand-alone
In mode, the application processor interface 630 acts as the primary user program host.

In order to support these various requirements,
Application processor interface 630
Has a unique processor in array director 610 and dedicated storage for API programs and commands. The command received from the host is to execute the API subroutine, load the additional function to the API memory,
Alternatively, a request can be made to load new software into the CC memory and the PME memory. As mentioned in the software overview section, these various requests can be restricted to some users through an initial program loaded into the application processor interface 630. That is, the operating program loaded determines the type of support provided. This support is adjustable to fit the performance capabilities of the application processor interface 630. Thus, APAP can be further tailored to the needs of multiple users who require managed and well-tested services, or individual users who want to achieve peak performance for a particular application.

The application processor interface 630 also manages the path to and from the I / O zipper. Data received from a host system in connected mode or a device in standalone mode is transferred to the array. Before this type of operation is started, the PME in the array that manages I / O is started. MIM
PMEs operating in D mode can use high speed interrupt functions and standard software or special functions for this transfer,
Detailed control instructions need to be provided to the PME operating in SIMD mode. The data sent from the input and output zippers requires almost the opposite adjustment. The PME operating in MIMD mode signals the application processor interface 630 via the high speed serial interface to
It must wait for a response from interface 630, while all PMEs in SIMD mode
Since it is already synchronized with the processor interface 630, the data can be output immediately. The ability to switch modes in the system provides unparalleled ability to tailor programs to your application.

Cluster Synchronizer (CS) 65
0: The cluster synchronizer 650 provides a bridge between the application processor interface 630 and the cluster controller 640. The cluster synchronizer 650 stores the output of the application processor interface 63 in the FIFO stack and monitors and initiates the status returned by the cluster controller 640 (both parallel input acknowledge and high speed serial bus data). It provides the cluster controller 640 with the desired routines or operations that need to be done in a timely manner. The cluster synchronizer 650 provides the functionality to support different cluster controllers 640 and different PMEs within the cluster, allowing the array to be divided into subsets. When doing this, after partitioning the array, the associated cluster controller 640 is instructed to selectively transfer the desired operation. The main function of cluster synchronizer 650 is to operate and organize all clusters so that overhead time is minimized or buried in PME run time. Above, the cluster synchronizer 65 with the A-SIMD configuration
The reason why it is particularly preferable to use 0 is explained.

Cluster Controller (CC) 640: Cluster Controller 640 interconnects with BCI 605 for a set of nodes in array cluster 665 (4D modified hypercube with 8 nodes per ring). , The cluster controller 640 is connected to 64 BCIs 605 in an 8 × 8 node array and 512
It shows that the individual PMEs are controlled. Again, if there are 64 such clusters in an 8x8 array, 32768
Complete system with PMEs). When the cluster controller 640 operates in the MIMD mode, the cluster controller 640 sends the command and data supplied from the cluster synchronizer 650 to the BCI parallel port, and returns the acknowledge data to the cluster synchronizer 650. SIMD
In mode, the interface operates synchronously and does not require step-by-step acknowledgment. Cluster controller 64
The 0 also manages and monitors the high speed serial port to determine when the PME in the node requests service.
Such a request is passed to the cluster synchronizer 650 while raw data from the high speed serial interface is available to the status display interface. The cluster controller 640 is a standard speed serial
Through the interface, it provides the cluster synchronizer 650 with an interface to a particular node in the cluster.

In SIMD mode, the cluster controller 6
40 is instructed to send an instruction or address to all PMEs on the broadcast bus. When in SIMD mode, the cluster controller 640 can dispatch all PME 16-bit instructions every 40 nanoseconds. PM
By broadcasting a group of instructions specific to E, an emulated instruction set is formed.

In the MIMD mode, the cluster controller 640 waits for the endop signal, and after receiving the signal, P
Issue a new command to the ME. The concept of MIMD mode is
Constructing a string of microroutines with unique instructions resident in the PME. These strings can be put together to form emulated instructions, and these emulated instructions can be combined to create service / canned routines or library functions.

In the SIMD / MIMD (SIMIMD) mode, the cluster controller 640 issues an instruction as in the SIMD mode, and the end from a certain PME is issued.
Check if there is an op signal. The MIME mode PME does not respond to the broadcast command and continues these specified operations. Unique status indicators help the cluster controller 640 manage this operation and determine when and where to provide subsequent instructions.

Operational Software Level: An overview of the Operational Software Level is provided herein for a detailed description of the services performed by the various hardware components.

A commonly used computer system has an operating system. Most large MIMD machines must have a relatively complete operating system kernel. In such machines, a workstation class CPU chip runs a kernel such as Mach. The operating system kernel supports message passing or memory coherency. Other massively parallel systems based on the SIMD model have little intelligence in the array. "Program counter" in the array
, There is no program to be run locally. All orders are broadcast.

The P of the present invention as the basis of a cluster array
In an ME based system, no operating system is required for each chip or node. The present invention provides within each processing element (PME) a library of important functions for operations and / or communications that can be invoked at higher levels. A specific set of Ps
SIMD type instructions are broadcast to the array to configure each ME. Then these PMEs
One or more of these library functions can be executed in full MIMD mode. In addition, each PME has resident basic interrupt handlers and communication routines that allow the PME to handle communication dynamically. Unlike existing MIMD machines, PME is used in APAP structure.
It is not necessary to include the entire program in memory. Instead of all such code, which is serial in nature, there is a cluster controller 640. Therefore (usually)
Such codes, 90% space related and 10% time related, can be broadcast to an array of PMEs in SIMD mode. Only truly parallel inner loops are PM
Dynamically distributed to E. These loops are in MIMD mode at the start, as they are for other "library" routines. This allows the use of single program, multiple data type program models. This means that the same program will run on each PM with built-in synchronization code.
Used when loaded on an E-node and running on a local PME. High bandwidth links can be used on the target network because the design parameters affect the bandwidth available on the various links and the system path can be configured programmatically, and the off-chip inter-PME link dynamic The bandwidth that a partition can provide on a particular path can be increased to meet the needs of a particular application. The links leaving the chip are directly interconnected and no external logic is required. The system can have various interconnection topologies and the routing can be done dynamically programmatically, as there are enough links and there are no default constraints on which other links can connect them.

According to this system, an existing compiler and command analysis function can be used to create a parallel program executable on the host or workstation based on the configuration. Single Program Extends the program source by performing program analysis on sequential source code for multiple data systems, examining dependencies, data, and control, call graphs,
It can include dependency tables, aliases, usage tables, etc. After that, program conversion is performed to create an explicit compiler instruction by combination of sequences or pattern recognition to create a modified version of the program that expands the degree of parallelism. The next step is the data allocation and partitioning step, including message generation. Here, the data usage patterns are analyzed and allocated so that the combined elements share a common indexing, addressing pattern. These operations provide embedded program compiler directives and calls to the communication services. At this point, the program moves to the level division step. In the level partition step, the program is separated into an array, an array controller (array director 610 or cluster controller 640), and parts that execute on the host. Each array section is interleaved with the required message passing synchronization functions in the section. At this point, level processing can proceed. The host source is passed to the level compiler (FORTRAN) for assembly compilation. The controller source is passed to the microprocessor controller compiler and items that are required in a single PME but not available in the library call are passed to the command parser (FORTRAN or C language) to optimize the P
Translated into an intermediate level language representation that produces ME code and array controller code. The PME code is created at the PME machine level and comprises a library extension that passes the load to PME memory. During execution, a PME parallel program can call already coded assembly service functions from the runtime library kernel in the SPMD execution process. APAP can function as a connection processor that is tightly or loosely coupled to the host or as a standalone processor, so
There are several variations on the software model. However, these variants work by integrating various applications so that only one set of sub-functions can satisfy all three applications. We will first describe the connected version of the software and then the modifications required for stand-alone mode.

As shown in FIG. 20, the APAP host
In a processor-attached system, the user's main program resides in the host and executes the APAP device tasks and associated data necessary to achieve the desired load balancing.
The user chooses whether to interpret the program of the dispatched task in the host or in the array director 610. The host level interpretation allows the array director 610 to operate in the presence of interleaving users who do not use the tight control of the array, whereas the APAP interpretation minimizes latency in the control branch. A for executing user management tasks
It tends to limit PAP time. As a result, APA
The concept that P and the host can be tightly or loosely established holds.

Two extreme examples will be given.

1. 30 with floating point vector mechanism
When connecting APAP to a 90-class machine, APAP
Compressed user data can be stored inside. A host program that called a vector operation on two vectors with different sparsity characteristics realigns the data into element-wise matched pairs, outputs the results to the vector mechanism, and reads the response from the vector mechanism. Finally, APA to reconstruct the data into the final sparse data format
Send a command to P. Each segment of APAP interprets and builds a sparse matrix bitmap, the other sections
Calculate how data can be moved between PMEs for proper alignment with the zipper.

[0254] 2. When connected to a small Air Force computer, APAP can perform the full workload associated with sensor fusion processing. In this case, the host initiates the process and sends the received sensor data to A
Wait for results after sending to PAP. The Array Director then needs to schedule and order the PME array through perhaps dozens of processing steps required to perform the process.

APAP supports three levels of user control:

1. Casual user. This user is
Work with the supplied routines and library functions. These routines are either host level or AP
Maintained at the I level and can be called by the user using subroutine calls in his program.

2. Customizer user. This user uses the application processor interface 6
Special functions can be written that operate within 30 and directly call routines provided by the application processor interface or services provided by the cluster controller or PME.

3. Development user. This user creates a program that runs on the cluster controller or PME, depending on the API service for program loading and status feedback.

To satisfy these three levels of users in a tightly or loosely coupled system, it is necessary to partition the hardware / software control tasks.

API software task: The application processor interface 630 should test the first word of the received data and interpret that data at the application processor interface or store it in the array director or PME. It has a software service that can decide if it should be loaded into the area or handed off to an I / O zipper.

If the data is to be interpreted, the application processor interface 630 determines the required action and calls the function. The most common type of operation is to request the array to perform a function that is performed as a result of an API write to cluster controller 640 (and indirectly to cluster controller 640).
Cluster synchronizer 650 / cluster controller 6
The actual data written to 40 is typically from the host to the application processor interface 63.
Constructed by the API operational routine based on the parameters passed to zero. The data sent to the cluster synchronizer 650 / cluster controller 640 is transferred to the PME via the node BCI.

Data is stored in API storage area, CC storage area, P
It can be loaded into any of the ME memories. Furthermore, PME
The data loaded into memory can be loaded via the I / O zipper or the node BSI. If the data is to be stored in API memory, the incoming bus is read and then written to storage. Cluster controller 64
Similarly, data addressed to 0 is read and then CC
Written to memory. Finally, the data for the PME memory (in this case, usually a new or additional MIMD program) is sent to all PMEs via the cluster synchronizer 650 / cluster controller 640 / node BSI.
Or it can be sent to a specific PME, or to a subset of PMEs via I / O zippers for selective redistribution.

When sending data to the input / output zipper, the PME
Inline commands can be placed in front of the data that allow the MIMD program to determine the final destination. It is also possible to place a call to the API service function in front of the data to perform MIMD initiation or SIMD transmission.

The API program not only responds to service requests received via the host interface, but also to requests from the PME. Such a request is generated on the high speed serial port and the cluster controller 6
40 / cluster synchronizer 650 combination. This type of request allows the API program to determine whether to directly execute the PME's request or access the PME via a standard speed serial port to further qualify the data for service requests.

PME Software: The software plan includes: Generation of PME resident service routines (ie, "extended ISA") for complex operations and I / O management.・ Create control data and parameter data, and BC
Definition and development of subroutines executed by the controller that are passed to the PME via the I-bus. These subroutines do the following: 1. Allow a set of PMEs to perform operations on distributed objects. 2. It provides input / output data management and synchronization services for PME array and system bus interactions. 3. Provides initial program load of processor memory, program overlay, and program task management. -Development of data allocation support services for host level programs. Assembler, simulator, and hardware
Development of programming support system including monitor and debug workstation.

The inventors have used military sensor fusion, optimization, image conversion, US Postal optical character recognition,
Based on a survey of and FBI fingerprint matching applications, it was concluded that a parallel processor programmed with vector and array commands (eg, BRAS calls) would work. The basic programming model is a PME that can be realized with today's technology.
Must match the characteristics of the array. In particular,
It is as follows. The PME can be an independent program storage processor. The array can contain thousands of PMEs, suitable for tight parallelism. -The inter-PME network has a very large total bandwidth and a small "logical diameter".・ However, network-connected microprocessor MI
Due to the MD standard, each PME is memory limited.

In conventional programming for MIMD parallel processors, a task dispatch approach is used. Such an approach requires each PME to access part of a larger program. Due to this property and the non-shared memory property of the hardware, PME memory is heavily used for critical issues. Therefore, in the present invention, "asynchronous SIMD" (A-SIM
D) Target a new programming model called type processing. In this regard, see U.S. Patent Application No. 798788 filed November 27, 1991.
This application is incorporated herein.

A-SIMD programming of the APAP design of the present invention means that a group of PMEs is instructed by commands broadcast to it, as in the SIMD model. The broadcast command is
Start executing the MIMD function in each PME. This execution may require data dependent branching and addressing within the PME, and I / O based synchronization with other PMEs or BCIs. Usually, PME
The process is completed and synchronized by reading the next command from the BCI.

The A-SIMD technique includes both MIMD and SIMD operating modes. With this method, there is no actual time limit for command execution time, so PMEs that are synchronized and infinitely executed during data transfer
The operation can start. Such features are extremely useful in data filtering, DSP, and systolic operation (they can be terminated by BCI interrupts or commands on the serial control bus). SI
MD operation is A- that does not include MIMD mode command.
It originates from the SIMD control stream. Such a control stream can include any PME specific instructions. These instructions are sent directly to the PME's instruction decode logic. Eliminating PME instruction fetches increases the performance mode of tasks that do not involve data-dependent branches.

This programming model (supported by the hardware function) can be extended to allow the array of PMEs to be divided into independent sections.
A separate A-SIMD command stream controls each section. Studies by the inventors have shown that the program may be split into separate phases suitable for pipelined data processing (ie, input, input buffering, multiple processing steps, and output formatting). I know it. Tight parallelism is created by applying n processor memories in a section to the program phase. Applying sparse partitioning to your application, small repetitive tasks suitable for MIMD,
Alternatively, a memory bandwidth limited task suitable for SIMD processing is often found. In the present invention, the MIMD portion is programmed using conventional techniques and the remaining phases are programmed as A-SIMD sections coded with vectorized commands and ordered by the array controller. This makes the large controller memory a program storage area. The number of PMEs per section can be varied to balance the workload. Changing the dispatched task size allows the BCI bus bandwidth to be tailored to control requirements.

The programming model also considers allocating data elements to PMEs. In this approach, data elements are evenly distributed among PMEs.
In early versions of software, this would be done by the programmer or software. The inventors believe that the IBM parallelizing compiler technology can be applied to this problem, and plan to investigate the usage of the technology. However, due to the inter-PME bandwidth provided,
This approach tends to be less important. This links data allocation with I / O mechanism performance.

The hardware requires the PME to initiate a data transfer from its memory and supports controlled writes to the PME memory without involvement of the PME program. Input control is provided at the receiving PME by providing the input buffer address and maximum length. When an I / O to a PME results in a buffer overflow, the hardware interrupts the receiving PME. The lower level I / O functions developed for the PME are based on this service. The present invention supports both the movement of raw data between adjacent PMEs and the movement of addressed data between PMEs. The latter function depends on the circuit switching and store-and-forward mechanism. Interrupt addresses and transfer operations are important to performance. In the present invention, hardware and software are optimized to support this operation. With a one-word buffer, an interrupt occurs when the address header is received. By comparing the target ID and the local ID, it becomes possible to select the output path. Subsequent transfers of data words occur in circuit switched mode. A slight modification of this process with larger buffers creates a store-and-forward mechanism.

Due to the high performance inter-PME bandwidth, careful storage of data elements in the PME array is not always necessary or desirable. Consider the case of moving vector data elements distributed between PMEs. The architecture of the present invention allows data to be sent without an address header, which allows very fast I / O. However, in many applications it has been found that optimizing the data structure to be suitable for movement in one direction slows the movement of data in the orthogonal direction. The time loss in such cases is typically close to the average time to randomly route data within the network. Therefore, storing data sequentially (rather than aligning the data) or randomly allows for applications with shorter average processing times.

Many applications can take advantage of average access times through synchronization (eg, PDE mitigation processes can obtain data from neighboring processes and thus average access to at least four I / O operations). Considering factors applicable to vector processes and array processes such as scatter / set and row / column operations, many users may find that a crude forced data allocation is suitable for their application. However, there are also examples in which application characteristics indicate that a particular data allocation pattern tends to be enforced (such as mandatory synchronization of shift directions or use of bias). For this property, tools and techniques developed need to support manual adjustment of data storage, or simple non-optimal data allocation (in the present invention, MPP is used to support nearly transparent ports of vectorized host programs). Support for non-optimal data allocation by host-level macros to allow users to see the resulting performance with a hardware monitor workstation).

[0275] Note that the Gaussian elimination by the standard pivot operation requires shifting of rows instead of columns. The performance difference obtained by arranging the data such that the columns are in the fast-shift direction exceeds 2: 1. Even so,
There is no benefit to aligning the rows to have a specific relationship with the bus.

FIG. 21 shows a general software development and use environment. The host application processor is optional because program execution can be controlled by either the host or the monitor. further,
In some cases, it is more effective to use a monitor instead of the array controller. This environment supports program execution on real or simulated MPP hardware. Because the monitor is scenario driven, developers performing test and debug operations can create procedures that enable effective behavior at any level of abstraction. FIG. 22 shows the levels of hardware supported within MPP and the user interface for these levels.

We believe that there are two possible application programming techniques for MPP.
The least programmer-intensive approach is to write your application in a vectorized high-level language. in this case,
If the user does not think that the data storage is coordinated, then use compile-time services to allocate the data to the PME array. Application is PME
Uses vector calls, such as BLAS, passed to the controller for interpretation and execution on the array. Unique calls are used when moving data between the host and the PME array. In short, the user does not need to know how the MPP organized or processed the data. Two optimization techniques are supported for this type of application:

1. Modifying the data allocation by creating a data allocation table allows the program to force data storage.

2. The generation of additional vector commands executed by the array controller allows adjustment of sub-functions (i.e. calling Gaussian elimination as a single operation).

Further, the present inventors believe that this processor can be applied to the special application referred to at the beginning of this section. In that case, use the code tailored to the application. However, even in such applications, the degree of coordination depends on how important a particular task is to the application. In this situation, tasks that are individually adapted to SIMD mode, MIMD mode, or A-SIMD mode would be required. These programs use the following combinations:

1. A sequence of PME-specific instructions passed to the emulator function in the array controller. The emulator broadcasts the instructions and their parameters to the set of PMEs. The SIMD mode PME passes the instruction to the decode function to simulate a memory fetch operation.

2. A tight inner loop that can be synchronized with I / O uses PME-specific ISA programs. Those loops start with a change of SIMD mode and then
It runs continuously in IMD mode (there is an option to return to SIMD mode via the ^ RETURN ^ instruction).

3. A more complex program written in the vectorized command set executes a subroutine in the array controller that called the PME specific function. For example, BL for vectors that are sequentially loaded on the PME
A simple array controller program that executes the AS ^ SAXPY ^ command initiates a sequence within the PME that performs the following operations. a. Processor ID and broadcast ^ incx ^ value and ^ X
PM with required x elements by comparison with _addr ^
Enable E. b. Compress the x value by writing to a continuous PME. c. The address of the PME having the y element is calculated from the broadcast data. d. Send the compressed x data to yPME. e. Perform a single precision floating point operation in the PME that received the x value and complete the operation.

Finally, the SAXPY example illustrates another aspect of vectorized application program execution. The major steps are performed in the application processor interface 630 and may be programmed by either the optimizer or the product developer.
Usually in vectorization applications, this level o
Call and use without embedding code. These steps are written as C or FORTRAN code and use memory mapped reads or writes to control the PME array over the BCI bus. Such a program may use the PME array as an API.
A series of Ms synchronized by a return to the program
Operate as IMD step. Minor steps such as single precision floating point routines are developed by the customizer or product developer. These actions are unique PME
Coded using ISA and tailored to machine characteristics. In general, this is the realm of product developers. This is because coding, testing, and optimization at this level requires the use of a complete product development toolset.

APAP executes the application sequential FO
Can be written in RTRAN. The route for this is completely different. Fig. 23 shows available FORTA
An outline of the RN compiler is shown. In the first step, FOR
The TRAN compiler uses some of the existing parallelizing compilers to develop program dependencies. The source and these tables are the inputs to the process that enhances parallelism through APAP MMP and source characterization.

Since this MMP is a non-shared memory machine, it is a PME for local memory and a PM for global memory.
Allocate data between E. The ultra-high-speed data transfer time is very fast and the network bandwidth is wide, which reduces the time impact of data allocation.
Is addressed. Our approach treats a portion of memory as global memory and uses software service functions. It is also possible to execute data allocation using the dependency information in the second method. The final step of converting the source into multiple sequential programs is performed by the level partitioning step. This segmentation step
It is similar to the Fortran ³ work being funded by the Department of Defense Advanced Research Projects Agency (DAPRA). The final process in compilation is the generation of executable code at all individual functional levels. For PMEs this is done by programming the code generator on an existing compiler system.
The host code compiler and API code compiler generate code targeted for the corresponding machine.

The PME is MIMD from its own memory.
Can run software. Generally, a plurality of PMEs do not execute completely different programs but execute the same small program asynchronously. The following three basic types of software can be considered. However, in this design method, APAP is not limited to only those methods.

1. With the dedicated emulation function, P
The ME array emulates a set of services provided by standard user libraries such as LINPACK and VPSS. Such an emulation package allows the PME array to use that set of devices to perform one operation required by a normal vector call. When connected to a vector processing device, this type of emulation package can use the vector device for one operation while performing another operation internally.

2. PME array parallelism provides a new set of compute and service functions in the PME1
Available by operating a suite of software.
This set of primitives is the code that the customizing user uses to create his application. The previous example of performing sensor fusion on an APAP connected to a military platform uses such an approach. Customizer Supplied 1
Kalman Filter, Track Optimum A using a set of function names
Write a routine to execute ssignment and Threat Assessment. This application is a series of API calls, each time the call is made, the PME set is started
A basic operation such as matrix multiplication is performed on the data stored in the ME array.

3. If there is no effective way to consider performance goals or application needs, PME
You can develop and run custom software in-house. "Sort" is a concrete example of this. While there are many ways to classify data, their purpose is always to tailor processes and programs to the machine architecture. The modified hypercube is well suited for batcher sorting. However, this sort requires extensive computation to determine the specific elements to compare with very short comparison cycles. The computer program of FIG. 19 is 1P
Batcher sort 10 with one element per ME
Simple example of a PME program for executing 00 11
00 is shown. Each line of the program description is extended to 3-6 PME machine level instructions, and all PM
E will execute the program in MIMD mode. Program synchronization is managed by I / O statements. This program is a simple 1PME
It extends to multiple data elements per hit, and extends to very large parallel sorts.

CC Storage Contents: CC storage data is used by the PME array in one of two ways. When the PME is operating in SIMD mode, the cluster controller 640 can fetch a series of instructions and pass them to the node BCI, which offloads the application processor interface 630 and the cluster synchronizer 650.
Alternatively, PME failure reconfiguration software, PME
Functions that are not often needed, such as diagnostics and possibly translation routines, can be stored in CC memory. In that case, such a function may be implemented by a working PMEM.
IMD program can request or AP
It can be transferred to the PME upon request of an I program instruction.

8-Directional Modified Hypercube Packaging: The packaging technique of the present invention uses eight PMEs packaged in a single chip and aligned in an N-dimensional modified hypercube configuration. This chip-level package or array node is the smallest building block in an APAP design. These nodes are further packaged in an 8x8 array. In this array, + -X and + -Y form a ring within the array or cluster, and + -W and + -Z extend to adjacent clusters. Clusters together form an array. This step significantly reduces the number of wires for data and control of the array. The W and Z buses are connected with adjacent clusters to form W and Z rings to collectively connect finished arrays of various sizes. A massively parallel system is made up of these cluster building units to form a massive array of PMEs. A
The PAP consists of an 8x8 array of clusters, each cluster having its own controller. All control units
It is synchronized by the array director 610.

Many trade-offs between the wiring possibility and the topology have been taken into consideration. Even if these are taken into consideration, the present inventors give priority to the configuration exemplified in this respect.
The concept disclosed herein has the advantage of keeping the X and Y dimensions within the cluster packaging level and distributing W and Z bus connections to all adjacent clusters. After implementation of the techniques described herein, the routability and manufacturability of a product is obtained while maintaining the unique characteristics of the defined topology.

The concept used here is to mix, match, and modify topologies at various packaging levels to achieve the desired result in terms of wire count.

For a method for defining the actual degree of modification of the hypercube, see above-referenced US patent application Ser. No. 07 / 689,866. In this preferred embodiment, two packaging levels are described for simplicity. It is extensible.

The first step is the chip design or package shown in FIGS. Eight processing elements and their associated memory and communication logic are contained in a single chip defined as a node. The internal structure is classified as a binary hypercube or a secondary hypercube, where every PME is connected to two adjacent PMEs. Communication diagram between PMEs of FIG. 10, especially 500, 510, 520, 530, 54
0,550,560,570.

In the second step, the nodes are configured as an 8x8 array to form clusters. The fully equipped machine consists of an 8x8 array of clusters, offering a maximum capacity of 32,768 PMEs. These 4096 nodes are connected to form an eighth-order hypercube network with programmable inter-node communication. In this way, the various designated routes are programmable, which increases the flexibility for transmitting messages of various lengths. With these programmability features,
Not only the difference in message length but also algorithm optimization can be dealt with.

This packaging concept aims to significantly reduce the number of off-page wires in each cluster. This concept uses a cluster defined as an 8x8 array of nodes 820, with each node 825 having 8 processing elements for a total of 512 PMEs, and then limiting the X and Y rings within the cluster. Finally, extend the W and Z buses to all clusters. The physical appearance is a sphere structure 80 consisting of 64 small spheres 830.
You should draw 0, 810 on your head. See Figure 17 showing full-up packaging technology for future packaging. Here, the X ring and the Y ring 800 are restricted in the cluster, and the W bus and the Z
Buses extend to all clusters 810. For the physical appearance, a sphere structure composed of 64 small spheres 830 may be drawn on the head.

The actual connections of a single node to adjacent XPMEs and YPMEs are in the same cluster. Wiring can be saved by extending the Z and W buses to adjacent clusters as shown in FIG. In FIG.
Loosely connected 4D hypercube or torus 900,
Also shown is a set of chips or nodes that can be configured as 905, 910, 915. Each of the eight external ports is + X, + Y, + Z, + W, -X, -Y, -Z,-
It will be represented by W950 and 975. Then, by connecting the + X port and the -X port, a ring can be constructed.
Furthermore, if the corresponding + Y port and -Y are interconnected,
M of such rings can be interconnected to form a ring of rings. This level of structure is called a cluster. This structure comprises 512 PMEs and is a building block for multiple size systems. FIG. 18 shows two such connections (950, 975).

Deskside MPP application:
Deskside MPP in workstations can be effectively applied in multiple application areas including:

1. Small production tasks that rely on numerically intensive processes. The United States Postal Service requires a processor that can find and read the postal code after receiving a fax image of a machine-printed envelope. This process is required by all regional classification schemes and is an example of a highly repetitive, computationally intensive process. In the present invention, the APL of the sample of the necessary program
Language version implemented. These models are MPP
Vectors and arrays used to perform work on
Emulate the process. Based on this testing, it turns out that this task fits perfectly into this processing architecture.

2. A task in which the analyst requests a sequence of data transformations as a result of previous output or expected needs. In the US Defense Agency example, satellite images are transformed and smoothed pixel by pixel and transformed into other coordinate systems. In such a situation, the transformation parameters of the image will vary from region to region as a result of land elevation and slope. Therefore, the analyst needs to add fixed control points and reprocess the transformation. A similar need arises when users require near real-time rotation or recognizable changes when utilizing scientific simulation results.

3. Program development for production versions of MPPs uses workstation-sized MPPs. Consider a tuning process that requires an analysis of processor and network performance. Such a task is a machine-analyst interaction. This task requires time when the machine is idle and the analyst is working. Running this on a supercomputer is very expensive. However, it is possible to add a supercomputer M to an affordable workstation MPP.
Given the same characteristics as PP (but with a smaller scale),
The programmer's inefficiencies related to accessing the remote processor are resolved, which offsets costs and facilitates the testing and debugging process.

FIG. 24 is a diagram of a workstation accelerator. The workstation accelerator uses a storage device that is the same size as the RISC / 6000 model 530. Each with a complete cluster,
Two swing out gates are shown. Combining the two clusters, 5 GOPS fixed point performance and 530 MFLOPS processing power and about 10
0 MB of I / O bandwidth is provided to the array. This device is suitable for any conventional application. This device, when mass-produced and including a host RISC / 6000, would be worth the value of a high-performance workstation without wasting a comparable machine using older technology.

Description of AWACS sensor fusion:
The military environment provides a series of examples showing the need for an extended math processor.

Communication in the targeted noisy environment requires digitally coded communication as used in the ICNIA system. The process of encoding data for transmission and recovering information after reception is a computationally intensive process. This task can be performed by the special signal processing module, but in situations where the communication coding represents burst-by-burst activity, the special module will usually be idle. With MPP, multiple such tasks can be allocated to a single module to reduce weight, power,
Volume and cost can be saved.

Fusion of sensor data is MPP
Provides a particularly clear example of augmenting an existing platform with the computing power gained by adding In the Air Force E3 AWACS, there are more than four sensors on the platform, but there is currently no way to consolidate all available data into a truck. Moreover, existing production tracks are of very poor quality due to their sampling characteristics. Therefore, there is a need to use fusion to increase the effective sample rate.

The inventors have investigated this sensor fusion problem in detail and can propose verifiable and effective analytical solutions, but the computational power available in the AWACS data processor does not lead to this method. Is insufficient. Figure 2
FIG. 5 shows a conventional track fusion process. This traditional track fusion process
The flaw is that individual processes are prone to errors and tend to be collected in the final merge rather than being eliminated. This process is also characterized by very high latency. This is because the merge will not complete until the slowest sensor has finished executing. FIG. 26 shows the improvements made by this method and the numerical calculation problems obtained thereby. The present inventors
Although we cannot solve the NP-hard problem, we have developed a good method to approximate the solution. Details of the application are described by the inventors in other patent applications. The application is 512 i860 (80860)
Intel Touchstone with processor (Intel
Touchstone) and IBM's scientific research visualization system (Sc
suitable for MMPs that use the APAP designs described herein and significantly outperform these other systems, eg, 128,000 PMEs, as they can be used on a variety of machines, such as the Intelligent Visualization System). Can be used as an application. Application experiments have shown that the quality of the approximation is below the level of sensor noise, so its response is applicable to applications such as AWACS. FIG. 27 shows a processing loop for the proposed Lagrangian reduced n-dimensional assignment algorithm. This problem uses a very controlled iteration of the well-known two-dimensional assignment problem. This algorithm is the same as that used in conventional sensor fusion processing.

For example, it is assumed that the n-dimensional algorithm is applied to the seven sets of observation values shown in FIG. 26 and subsequent figures, and the two-dimensional allocation process needs to be repeated four times for each pass of the reduction process. Then, in the new 8-dimensional assignment problem, the 2-dimensional assignment problem needs to be repeated 4000 times. AWACS workload is currently around 90 machine capacity
%. Fusion probably requires 10% of total capacity, but even with such a small capacity 4000
When scaled up twice, the total utilization is 370 times the capacity of AWACS. Not only does this workload exceed the performance of existing processors, but it is also marginal with existing sparse parallel processing systems that are compatible with the new MIL environment, or systems that are expected over the next few years. If the algorithm requires on average 5 iterations instead of 4 per step, then even the performance of the hypothetical system is exceeded. Conversely, the MPP solution can provide this computational power, even at the level of 5 iterations.

Mechanical Packaging: As shown in FIG. 4 and other figures, the preferred chip of the present invention is constructed in a quad flat pack format. Thus, the chips can be bricked to form various two-dimensional and three-dimensional configurations within the package. 8 or more P
One chip consisting of ME is the first level package module. Similarly, a single DRAM memory
Chips are the first foundry to package chips
It is a level package module. However, the quad flat pack format allows four-way interconnection. Each connection is a point-to-point connection (one chip in the first level package is a foundry module). In the present invention, this feature allows the construction of PE arrays of sufficient scale to achieve the performance goals of the present invention. In practice, these chips can be connected between two points that are 3 feet, 4 feet, and sometimes 5 feet apart, i.e. between multiple processor nodes, and still have proper control without optical fiber.

This benefits the drive / receive circuitry required on the module. Since the present invention does not have a bus system for connecting modules in a daisy chain, high performance can be realized and power dissipation can be suppressed.
In the present invention, broadcast communication between nodes is performed, but this does not need to be a high performance path. Since most data operations can be performed at the nodes, the required data path requirements are reduced. The broadcast path of the present invention is primarily used primarily as a controller routing tool. The data stream is connected to and flows through the ZWXY communication path system.

The power dissipation in the present invention is 2.2 W per node module in the commercial workstation of the present invention. Therefore, air-cooled packaging can be used. Therefore, the power requirements of the system of the present invention are also reasonable. In the illustrated power supply system of the present invention,
Multiply the number of supported modules by about 2.5W per module. Such a 5V power supply is very cost effective. In terms of power consumption, surprisingly, 32 microcomputers can operate with less power than is consumed by the light needed to read a book.

The thermal design of the present invention is improved by this packaging. In the present invention, the hot spot due to the mixture of the high dissipation portion and the low dissipation portion is avoided. this is,
It is directly reflected in the assembly cost.

The cost of the system of the present invention is very attractive compared to the approach of placing a superscalar processor on the card. The performance levels per assembly per dollar per component per dollar per connector per watt of the present invention are outstanding.

Moreover, the present invention does not require the same number of packaging levels as other technologies. No modules / cards / backplanes and cables are required. You can omit the card level if you wish. As shown in the workstation module of the present invention, the brick level technique omitted the card level.

Further, as shown, each bricked node housing in the workstation module can even have multiple duplicate dies in the same chip housing, as shown in FIG. Normally, one die is placed in the air-cooled package, but eight die can be placed on the substrate using the multi-chip module approach. Thus, a dream wristwatch with 32 or more processors can be realized and many other applications are possible. The number of possible applications is unlimited thanks to packaging and power and flexibility. At home, monitor all controllable instruments,
Can be coordinated with very small parts. All chips used extensively in automobiles for engine monitoring, brake adjustment, etc. can have a monitor in the housing. In addition, 386 microprocessor chips with fully programmable functions and memory are mounted (all in one chip) on the same substrate using the hybrid technology, and are mounted on an array controller of the substrate package. Can be used as

A number of system configurations have been shown, from the control system of FIG. 4 to a much larger system. Equipped with 8 or more PMEs on the chip in the dip,
Like SIM modules, chips can be packaged with a pin array that fits standard DRAM memory modules, so the control mechanism allows for 30 frames of frame instead of just 15 frames with existing technology today. Infinite applications are possible, down to wall-sized video displays, which have a repetition rate and whose processor can be assigned to monitor nodes consisting of one or a few pixels. With the brick wall (brick wall) quad flat pack of the present invention,
It becomes easy to duplicate the same part repeatedly. Furthermore, the duplicated processor is actually a memory with processor replacement capability. Dedicate a portion of memory to a specific monitoring task and another portion (size is programmatically defined) to a large global memory that is point-to-point addressed and has broadcast capability for all functions. be able to.

The basic workstation, supercomputer, controller and AWACS of the present invention are all examples of packages in which the new technology of the present invention may be used. An array of memory with an embedded CPU chip and I / O serves as a PME for massively parallel applications and even more limited applications. The packaging and programming flexibility add imagination and, using the techniques of the present application, one part can be assigned to multiple concepts and images.

Military Avionics Application: M
The cost advantage of building an IL MPP is AWA
Especially clear on CS. The AWACS is a storage device developed 20 years ago, and a new technology memory module is used in place of the conventional core memory, so that the free space is increasing. FIG. 28 shows two MIL-compliant cluster systems that are housed directly in the rack empty space and use existing memory buses for interconnection.

The example of AWACS is extremely advantageous because there is free space, but it is possible to create space in other systems. Usually, existing memory is
It is easy to replace with a PP or separate MPP gateway. In such cases, a quarter cluster and an adapter module would require 8MB memory and 640M
The IP is obtained and probably two slots are used.

Supercomputer Application: 64 Cluster MPP is a 13.6 GFLOP supercomputer. This can be configured as the system shown in FIG. In this system, as shown in FIG. 29, a node chip can be brick-laid on a cluster card to construct a system having great cost and size advantages. It is not necessary to incorporate extra chips into the system, such as network switching, as it is costly.

The interconnection system of the present invention with a "brick wall" chip allows large scale DRA
It can be constructed similarly to M memory. The interconnect system has predefined bus adapters that meet strict bus specifications, such as Micro Channel bus adapters. Each system has a smaller power system and cooling design than other systems based on many current microprocessors.

Unlike most supercomputers,
This preferred APAP of the invention with floating point emulation capability is much faster when performing integer operations (164 GIPS) than floating point operations. Therefore, this processor is most effective when used for very character or integer processing intensive applications. In the present invention, in addition to the above-mentioned other applications, the problems of three programs that need to be solved have been examined. Applications that may be more important than some "far-reaching challenges" to daily life include:

The 1.390 vector processor is equipped with a very high performance floating point unit. This device, like most vectored floating point devices, requires pipeline operations on dense vectors. Applications that make heavy use of non-normal sparse matrices (matrixes that are described in bit maps rather than Cartesian coordinates) waste the performance capabilities of floating-point arithmetic units. MPP solves this problem by providing storage for that data and using its computing power and network bandwidth to build dense vectors without performing calculations and decompress dense results. are doing. The vector processor is kept busy by the continuous flow of operations on the dense vector supplied by the MPP. Setting the size of the MPP so that the vector mechanism processes can be effectively compressed and decompressed at the same rate will allow both devices to be fully busy.

2. Another host connection system that we have considered is a solution to the FBI fingerprint matching problem. In this case, a machine with more than 65 clusters was considered. The question was how to match about 6000 fingerprints per hour against the entire database of fingerprint history. With large DASDs and the full bandwidth of the MPP to host attachment, the entire database can be matched to incoming fingerprints in about 20 minutes. Approximately 7 MPP in SIMD mode loose butting operation
Using 5% balances the processing with the required throughput rate. In that case, when 15% of the machines in A-SIMD processing mode pass the sparse filtering operation, the unknown fingerprint is matched with the fingerprint of the file to perform a detailed inspection to complete the matching. Estimated. During this time, the rest of the machine is MIMD, which has spare capacity, work queue management,
And allocated for output formatting.

3. The application of MPP to database operation was considered. This work is very preliminary,
Suitability seems good. Two aspects of MPP support this premise. a. The connection between the cluster controller 640 and the application processor interface 630 is Micro Channel. Therefore, the DASD dedicated to the cluster and directly accessed from the cluster can be arranged.
A 64-cluster system with six 640MB hard drives per cluster provides 246GB of storage. Furthermore, the entire database is 10-2
You can search sequentially in 0 seconds. b. Databases are generally not searched sequentially. Instead, it uses multiple levels of pointers. Database indexing can be performed within a cluster. DAS
Each bank of D has a processing capacity of 2.5 GIPS and 32
Supported by MB storage. This is sufficient for index search and storage. Indexes are now often stored in DASD, so
The performance is greatly improved. Using such an approach, distributing DASD over a SCSI interface connected to Cluster Micro Channel can create a database of virtually unlimited size.

FIG. 29 shows a supercomputer-scale MMP constructed using APAP. This approach uses unit replication again, but in this case it is the enclosure that contains the 16 clusters that is replicated. Identification of this replication method The base (operand) size is 16 bit words. In PME storage, the operands are located on unified word boundaries. Not only the word operand size,
Other operand sizes in multiples of 16 bits can also be used to support additional functionality.

Within the range of the operand length, the bit positions of the operand are sequentially numbered from left to right starting from 0. Whenever the upper bit or the most significant bit is referenced, the leftmost bit position is referenced. Whenever the lower bit or the least significant bit is referenced, the rightmost bit position is referenced.

Instruction Format: The length of the instruction format can be 16 bits or 32 bits. In PME storage, instructions must lie on 16-bit boundaries.

The general instruction format shown in Table 1 is used. Usually, the first 4 bits of an instruction define the instruction code and are called OP bits. Extend the definition of the instruction,
Or to define the unique conditions that apply to an order,
Additional bits may be needed. These bits are called OPX bits.

[Table 1]

There is one field common to all formats. This field and its interpretation are as follows:

Bits 0-3: Instruction Code--This instruction code, together with the instruction code extension field, defines the operation to be performed.

A detailed view of the individual formats and their field interpretations is given in the following sections. Some instructions combine the two formats to form a variant of the instruction. These are primarily concerned with the addressing mode of the instruction. As an example, a cross-storage instruction may have a form related to direct addressing or register addressing.

RR Format: Register Register (RR) format provides two general purpose register addresses as shown in FIG. 30 and is 16 bits in length.

The RR format includes the following fields in addition to the instruction code field.

Bits 4-7: Register Address 1-R
The A field is used to specify which of the 16 general purpose registers will be used as an operand, a destination, or both.

Bits 8-11: 0-If bit 8 is 0, the format is defined as RR format or DA format; if bits 9-11 are 0, the operation is defined as register-to-register operation (direct address・ In case of special format).

Bits 12-15: Register Address 2
The RB field is used to specify which of the 16 general purpose registers to use as an operand.

DA format: Direct address (DA)
The format provides one general register address and one direct storage address, as shown in FIG.

The DA format includes the following fields in addition to the instruction code field.

Bits 4-7: Register Address 1-R
The A field is used to specify which of the 16 general purpose registers will be used as an operand, a destination, or both.

Bit 8: 0-If Bit 8 is 0, the operation is defined as a direct address operation or a register-to-register operation.

Bits 9-15: Direct Storage Address--The Direct Storage Address field is used as an address to a level-specific storage block or a common storage block. Direct address field bits 9-11
Must be non-zero since it directly defines the address format.

RS format: Register storage area (R
The R) format provides one general register address and indirect storage address, as shown in FIG.

The RS format includes the following fields in addition to the instruction code field.

Bits 4-7: Register Address 1-R
The A field is used to specify which of the 16 general purpose registers will be used as an operand, a destination, or both.

Bit 8: 1-If this bit is 1, the operation is defined as a register storage operation.

Bits 9-11: Register Data-These bits are considered to be signed values used to modify the contents of the register specified by the RB field.

Bits 12-15: Register Address 2
The RB field is used to specify which of the 16 general purpose registers to use as an operand.

RI format: register immediate value (RI)
The format provides one general purpose register address and 16 bits of immediate data. The RI format has a length of 32 bits as shown in FIG.

The RI format includes the following fields in addition to the instruction code field.

Bits 4-7: Register Address 1-R
The A field is used to specify which of the 16 general purpose registers will be used as an operand, a destination, or both.

Bit 8: 1-If this bit is 1, the operation is defined as a register storage operation.

Bits 9-11: Register Data-These bits are considered to be signed values used to modify the contents of the program counter. Normally, this field takes a value of 1 in the register immediate format.

Bits 12-15: 0-If this field is 0, it is specified that the updated program counter, which points to the immediate data field, is used as the storage address of the operand.

Bits 16-31: Immediate Data-This field contains the 16-bit immediate data of the register immediate instruction.
Functions as an operand.

SS Format: The inter-storage (SS) format provides two storage addresses. One of these is explicit and the other is implicit. The implicit storage address is placed in general register 1. Register 1
Are modified during the execution of the instruction. Fig. 34 shows the SS command.
There are two formats, the direct address format and the storage address format, as shown in FIG.

The SS format includes the following fields in addition to the instruction code field.

Bits 4-7: Instruction Extension Code--The OPX field, together with the instruction code, defines the operation to be performed. Bits 4-5 are for ADD and SUBSTRACT
Define the operation type such as. Bits 6-7 control how carry, overflow, and condition code are set. When bit 6 = 0, overflow is ignored, and when bit 6 = 1, overflow is enabled. When bit 7 = 0, carrystat in the operation is ignored, and when bit 7 = 1, carrystat is included in the operation.

Bits 8: 0-Define the direct address format. 1-Define the format as a storage address format.

Bits 9-15: Direct Address (Direct Address Format)-The direct storage address field is used as an address to a level-specific storage block or a common storage block. Bits 9-11 of the direct address field must be non-zero to define a direct address format.

Bits 9-11: Register Delta (Storage Address Format)-These bits are considered a signed value used to modify the contents of the register specified by the RB field.

Bits 12-15: Register Address 2
Storage Address Format--The RB field is used to specify which of the 16 general purpose registers to use as the storage address of the operand.

SPC format 1: special (SPC1)
The format provides one general register storage operand address, as shown in FIG.

The SPC1 format includes the following fields in addition to the instruction code field.

Bits 4-7: OP Extend-The OPX field is used to extend the opcode.

Bit 8: 0 or 1-If this bit is 0, the operation is defined as a register operation. If this bit is 1, the operation is defined as a register storage operation.

Bits 9-11: Instruction Length-These bits are considered to be signed values used to specify the length of the operand in 16-bit words. A value of 0 corresponds to a length of 0, and a value of B ^ 111 ^ corresponds to a length of 8.

Bits 12-15: Register Address 2
The RB field is used to specify which of the 16 general purpose registers to use as the storage address of the operand.

SPC format 2: special (SPC2)
The format provides one general register storage operand address, as shown in FIG.

SPC2 includes the following fields in addition to the instruction code field.

Bits 4-7: Register Address 1-R
The A field is used to specify which of the 16 general purpose registers will be used as an operand, a destination, or both.

[Table 2]

[Table 3]

[Table 4]

[Table 5]

[Table 6]

[Table 7]

[Table 8]

Functional Summary: APAP Machine Positioning: A detailed aspect of the invention has been described which is considered to be technically positioned between the CM-1 and N-cubes. CM-1 uses a point design for processing elements, similar to the APAP of the present invention,
The processing elements and the memory are combined on the basic chip.
However, while the CM-1 uses a 1-bit wide serial processor, the APAP series uses a 16-bit wide processor. CM series machines are
Starting with 4 kilobits of memory per processor,
It has grown to 8 or 16 kilobits.
On the other hand, the first APAP chip of the present invention provides 32 kilobits × 16 bits of memory. CM-1 and its successors are strictly SIMD machines,
M-5 is a hybrid type. Instead, the APAP of the present invention effectively uses the MIMD mode of operation with the SIMD mode. The parallel 16-bit wide PME of the present invention may seem like one step closer to the N-cube, but it is not appropriate to see so. A
Unlike N-Cube type machines, PAP does not separate memory and routing from processing elements. Also,
APAP implements a 32 kilobit by 16 bit PME, whereas an N-cube implements only a 4 kilobit by 32 bit processor.

Although there are superficial similarities as described above, A
The PAP concept is quite different from the CM and N-Cube series in the following points.

1. The modified hypercube incorporated into the APAP of the present invention is a novel invention with significant packaging and addressing advantages over the hypercube topology. For example, it should be noted that the 32KPME APAP in the first preferred embodiment has a network diameter of 19 logical steps, and transparency can reduce this to 16 effective logical steps. Furthermore, by comparison, if we use a pure hypercube and all PMEs send data over an 8-step path, we have 8 PMEs.
On average, 2 of them are active, while the rest are blocked and delayed.

Also consider the 64K hypercube required when CM-1 is a pure hypercube. In that case, each PME would require ports to the other 16 PMEs, allowing data to be routed between the two most distant PMEs of the 15 logical steps. If all PMEs try to transfer 7 steps in average distance, 2 of the 7 PMEs will be active. However, CM-1 does not use a 16-dimensional hypercube. The CM-1 interconnects 16 nodes on the chip with the NEWS network and then provides one router function within the chip. 40
96 routers are connected to form a 12-dimensional hypercube. In the absence of collisions, the hybrid has a logical diameter of 15, but its effective diameter is much larger because 16 PMEs contend for the link. That is, if there are 8 step moves, only 2 of the 16 processors are active. This means that 8 complete cycles are needed instead of 4 to complete all data movements.

The N-Cube actually uses a pure hypercube, but currently only supports 4096 PMEs, and therefore a 12-dimensional hypercube (P
In the case of ME8192 pieces, 13 dimensions are used. N
-To expand the cube to 16K processors and have the same processing data width as APAP, quadruple the hardware and 25% of the connection ports to each PME router.
Need to increase. There is no solid data to support this conclusion, but with the N-Cube architecture, 16K
It seems that the connector pins are running short before the PME machine is reached.

2. The ability to fully integrate and distribute key tasks within an APAP machine is a clear advantage. CM
As mentioned in the description of the Series and N-Cube series machines, they must have separate units for message routing and floating point coprocessor, respectively. The APAP system combines integer processing, floating point processing, message routing, and I / O control as a single point design PME. This design is then replicated eight times on the chip, and then the chip is replicated 4K times to form the array. This has the following advantages. a. The use of one chip maximizes production runs and minimizes system element costs. b. A regular architecture provides the most effective programming system. c. Since almost all chip pins can be dedicated to the common problem of interprocessor communication, the interchip I / O bandwidth, which tends to be an important limiting factor in MPP designs, is maximized.

3. APAP has unique design capabilities that can take advantage of chip technology and capital investment in custom chip designs.

Let us consider the issue of floating point performance. APAP PME performance on DAXPY is 1FLO
There are about 125 cycles per P. In contrast,
The ^ 387 auxillary processor is approximately 14 cycles, while the CM-1 Weitec Coproce
ssor) is about 6 cycles. However, in the case of CM, P
There is only one floating point unit for every 16 MEs, while the N-Cube probably has one 387 type chip associated with each 386 processor. AP of the present invention
The AP has 16 times the PME and can therefore almost completely compensate for the single unit performance delta value.

More importantly, the eight APAP PMEs in the chip are built from the 50K gates currently technologically feasible. As memory macros shrink, the number of gates available for logic circuits increases. If the increased gate is used for extended floating-point normalization, A
PAP floating point performance should be able to far exceed other devices. Alternatively, the PME design or PME subsection design can be performed using custom design techniques to improve overall performance without affecting the software developed for the machine.

The inventors believe that the APAP design of the present invention can take advantage of future processing technology developments. In contrast, the most similar machines, such as CM-x and N-Cube, that use a system such as that shown in FIG. 1, are suitable for utilizing conventional techniques that appear to be stuck. I think that the.

The advantage of the APAP concept is that DASD combined with a group of PMEs can be used. This A
The PAP functionality and the ability to connect displays and auxiliary storage is a byproduct of choosing the MC bus as an interface to the external input / output ports of the PME array.
Therefore, the APAP system is configurable and P
It may have a card mounted hard drive selected from either the S / 2 unit or a set of units compatible with RISC / 6000 units. In addition, this feature could be used without designing additional component modules. However, in this case, the number of back panel and basic enclosure replicas that must be used will be greater than that required for APAP.

Picket and APAP Multiplex PME MIM
D / SIMD FUNCTIONS The picket processor of the present invention is extremely compact and its usefulness of being able to place 1000 processors on 2-8 cards seems particularly advantageous for military applications. Be done. However, the concepts within this system could be applied to processors using less advanced technologies and would replace processor memory chips. For example, some of our original concepts could be implemented in a machine with a workstation RISC microprocessor, even one having only one processor per card. Each processing unit is one element of the array. Each processing unit comprises memory and its own instruction stream, and according to the MIMD implementation it can and does run completely autonomously on its own code stream. APAP if multiple elements perform the same copy of the instruction stream and are synchronized to run more or less synchronously
Or other such machine with the picket SIMD of the present invention
It can emulate an architecture. The APAP machine of the present invention is structured so as to function as a MIMD element capable of emulating SIMD.
In comparison, the picket processor of the present invention is preferably arranged to be based on the SIMD architecture, in which some data elements are controlled by one instruction interpretation element. The picket machine emulates MIMD by having one instruction-interpreting element command read all the data elements from its memory and interpreting it as an element instruction. Each element can also keep track of its next instruction address. In this way, the picket processor provides MIMD operation.

The APAP machine is a flexible machine that implements the SIMD architecture and performs various functions in commercial and military environments as the inventors have developed for the picket machine of the present invention. be able to.
This machine is an element with a separate instruction stream, S
A control structure that emulates the IMD can be implemented. The APAP of the present invention has multiple small processors on a single chip, each with performance in the range of a picket machine. In some respects, APAP's VSLI design (and picket version) can achieve relatively coarse processor functionality, but it can and does provide a much denser array design.

The picket machine of the present invention provides the control network and the associated processing power required when a program requires the control network to perform "shrink" and related operations.

Some of the features of the present invention that were advances in the art at the time the picket machine was developed are SI
Includes MIMD functionality. The SIMIMD feature of the present invention can load picket memory with a small amount of program code executed by each picket. Control is held by the controller, after which a smaller amount of code can be loaded and executed. The MIMD mode processing unit provides the ability to do independent things within each picket processing unit. It is no longer necessary to transfer the entire program to the array processor. Normally you would not transfer the entire program to the picket.

• For example, the partition manager can load the same code on every processing node in a partition. Data is distributed to the nodes. Given an array of m values and a partition of n nodes, each node will process m / n values. No synchronization or communication is required as long as the computation stays local, as each node can independently execute the part of the program that is in MIMD mode and branch based on its own data value. It carries communication network data and performs the necessary synchronization when data needs to be transferred between processors, for example when each processor must contribute a value to the global sum. For global combination operations such as union, the controller serves to organize the controlled network so that reduction can be performed.

Another feature of the picket machine of the present invention is the slide bath. Use a slide bus to broadcast data from an array controller to an array of pickets or from a single picket to an array controller for use by the controller or to rebroadcast to the array. be able to. By implementing the slide bus of the present invention, the functionality of many machines can be improved. Augmenting data transfer with picket processing activity allows more powerful functions to be performed on the slide bus. The functions that can be performed in a system with a slide bus include:

HORIZONTAL SUM (Water Peace)
This process produces the sum of the numbers presented to the process from each activity picket. This is just one example of the type of command that a slide bus is particularly useful for producing a single value from a sequence of numbers.

ACCUMULATE LEFT In this process, each picket will eventually contain the sum of all right "picket" numbers. There are two implementations that illustrate this capability. A. Simply shift the number to the right (with zero fill) and each picket is simply added to the sum. B. In the second embodiment, this process is performed in a more parallel fashion and only four steps are required to handle 16 numbers.

FIND VALUE The picket containing the given value of the parameter is identified.

FIND MAX The picket containing the maximum value of a parameter is identified.

To give a further example, the system can divide parallel processing nodes into groups, which can be considered partitions. The control device can manage each section. User processes can run on a single partition.

Interprocessor communication can be done by duplication by copying data values. For example, a single value can be broadcast to all processors for use during calculations. You can copy the vector into each column or row of the matrix. This is called a spread. Less regular patterns are divisors of certain aggregates into arbitrary subsets of various sizes. Different values within each subset can be broadcast. If the subsets are ordered and not interleaved, then they can be viewed as a collection of vectors of various sizes. This common case can be implemented like the general case.

Reduction can also be used with interprocessor communication. Shrinking takes data values and reduces the number of data values by combining them. For example, a single value can be generated by calculating the sum of a set of values. In this case, addition is a combination operation. The reducing operation includes taking a maximum value or a minimum value, taking a logical product, and taking a logical sum. These all start with a large collection of values and reduce it to a single result.

• Permutation can be used with interprocessor communication. Permutation reorders its inputs to produce the same number of results. Everything goes from one place to one place. Examples are matrix transposition, vector inversion, multidimensional lattice shifting, and FFT butterfly patterns.

Global functions include broadcasting data or instructions from host to node, reducing data to all nodes, performing traversal across nodes, performing segmented parallel prefix operation, all Contains the concatenation of elements to the buffer on the node or from the node to the buffer on the host. The reduce and parallel prefix add operations can perform additions, find maximum or minimum values, or perform bitwise AND, OR or XOR. In the picket and APAP machines of the present invention, these operations can be performed at the node level or at the individual processor memory element level. The individual processor memory elements function similarly to the nodes of the machine, providing the card with only one memory processor.

The controller contains integer and logic hardware. The controller can compute the parallel prefix add operation and segment the parallel operation. All processors can go into synchronous mode and when ready, the controller can tell that it is ready for SIMD operation. However, the processor can operate in MIMD mode. The processor may overlap irrelevant processing within the latency. Programs for SIMD can be used, such as operations in which thousands of processors all perform SIMD operations without having to be strictly synchronized.

Global operations including global reduction, integer addition, integer maximum value discovery, logical sum, exclusive OR and floating point operations can be performed, all done by one node. Matrix operation can be executed.

The functions of the control device and controlled network of the present invention include synchronization of processing nodes (and processing elements within the nodes), combining values from all processing elements to produce a single result. , And the parallel prefix addition operation can be calculated. The picket processor of the present invention provides a separate control network and data network. See U.S. Patent Application No. 6111594 and other related applications.

In the present invention, a hardware cluster is provided. However, the hardware cluster does not have to refrain from using the UNIX software cluster partitioning for partitioning. The duplication can be done in an array defined in software, which can be in one hardware cluster or can contain multiple hardware clusters. Broadcast to the array will include all defined processing elements defined for an array process. Spread and inverse spread operations (in the Fortran sense) are supported and processors can be partitioned into clusters in the software sense for broadcast operations.

The picket process of the present invention has a broadcast from a node or from a processing element within a node. That is, different levels of broadcast operation having the same functions are provided and the supervisory functions are reserved for the array controller. The processor can receive broadcasts while executing MIMD internal instructions. The present invention broadcasts instructions. The mask bit allows the processor to refrain from broadcasting.

Group Picket or array controllers can assign pickets to one or several of multiple groups. Pickets can be in multiple groups at the same time. Groups can be selected in SIMD or MIMD mode for some part of the process and can move freely between them.

Partitions can exchange data with processes in other partitions. Multiple users can access the partition without interfering with the usage of another user.

Each processing element need not (and in fact do not) have an entire operating system.
The code may be downloaded or broadcast along with the data to the processing elements of a node, for example for memory mapping. Each processing element and node has a memory, which can be allocated for that operation and for global memory operations. By broadcasting the code to one or more processing elements, each processing element may execute the provided code, each acting on its own data, and performing calculations and branches accordingly. it can.

Partitions can be freely configured for single user tasks. Each partition can complete certain tasks as part of a system that can be used for time-shared processing, batch processing, or both.

Virtual network address allocation is
You can have protection checks that prevent your process from sending messages to destinations outside the partition. The array controller can send messages from one partition to another. I / O can be coordinated at various levels from host to node.

Autonomy Each picket contains a status latch that controls the process. If this latch is set, the picket refrains from participating in the process. In addition, the picket can reactivate itself based on testing conditions inside its own memory.

Vector elements can be handled within each processing element and node. Vector instructions can be in short 32-bit format or longer format.

The local autonomy of the processor and memory elements can suppress broadcast communication. The latch can be set or reset and the picket will or will not participate depending on its setting. The processing elements to join are
It is a processing element whose participation is not suppressed.

Controllers Instructions are provided for manipulating numeric vectors. These vectors can be horizontal (one for each picket) or vertical (the entire numeric vector is contained in one picket). However, in addition, each picket can contain one numeric vector. The instruction is the addition of two vectors,
It provides all of the desired vector commands, such as the subtraction of a constant from each member of the vector, or various vector products or reductions.

The controller can provide all of the desired vector commands.

Status Funnel The Picket Array Controller uses the "Status Funnel" to collect status from active pickets and send cumulative results to the Array Controller.

The status funnel allows all processors to indicate that they have completed their processing steps and that the next operation can proceed according to commands from the controller.

There are other features common to the picket processor that can be applied to other machines such as the APAP of the present invention. These features can be used on machines with parallel array processors.

The inventors have provided independent processing elements with scalability at the processor level as well as the system level. Therefore, processing communication and input / output can be scaled. Floating point integer processing is provided. Has high bandwidth. The processor array executes high-level language programs, can execute multiple jobs in a time-sharing manner and in a piecewise manner, has multi-user access, secures between users, and communicates in parallel between processing elements. With high bandwidth I / O, the system is able to perform balanced scalar and parallel execution of I / O, processing and memory with high reliability and fail-safe capability.

Single task and multiple tasks can be executed,
Processing partitions, nodes and elements can be time shared. Nodes can fetch the same instruction from the same address in processor memory to execute in SIMD mode,
Alternatively, it can be fetched from an individually selected address to execute an independent MIMD mode instruction.

The control device controls the execution of multiple jobs. Usually only one controller is provided, but multiple arrays can be implemented.

The system uses and extends VLSI to provide a RISC processor system for nodes of the system with local and global memory for each processing element.

This system can execute programs by data parallel coding, and can execute applications in Fortran, C language or other high level languages for parallel processing.

Picket loads a small amount of application code into its own PE memory. Execution can utilize run-time code provided by the library function.

The machine supports data parallelism and provides branching and synchronization.

Memory is provided at each point within each PE or PME, allowing the processor to retrieve SIMD styles from the same address and MIMD styles from individually selected addresses.

The controller can and does broadcast a block of instructions to the processing elements of the array.
Duplicate, spread, shrink and transpose functions can be performed.

A separate PME broadcasts to controllers as needed for the process.

Only one node broadcast communication and PME broadcast communication can be executed at a time and can be controlled by the control device.

-Processors can be divided into groups.

This array processor system
It provides a way to route messages between processors.

Provide memory address space for thousands of processors. Each address space can be considered local to each element or a global address for the entire array of processing elements.

The Picket Processor provides conditional enablement for the autonomy of processing elements.

Conditional processing can be performed using mask bits.

Mask bits allow individual processing elements to refrain from participating in a task.

Each processing element can assign itself to a partition, and a partition can contain from zero to all processing elements of the system. A processing element, processor memory element or picket can enter one or more compartments at the same time.

The picket processor may include thousands of computer processing nodes, one or more control processors, and input / output units that support mass storage, graphic displays and infinite peripherals. it can. Each processing node can be thought of as a unit that acts as a regular node or as a mesh of individual processing memory elements, each of which is capable of fetching and interpreting its own instruction stream and processing an array of vectors. I will provide a.

The status funnel is used to indicate the end of the array process.

Global bit operations produce a logical or of controller status for all participating processors.

The global operation may be synchronous or asynchronous and can be used independently.

Grouping results in a group of elements for a partition.

Grouping of processing elements is done using the address of the processing element or picket, assigned based on hardware availability and possible failures.

Each node is a multi-layer processor, similar to the preferred embodiment of the invention in which many other features may be implemented.
A memory element node, which can be formed as a RAM with embedded memory, which is locally
It can also control memory and function as part of a distributed system.

This brief comparison is not meant to be limiting, and one of ordinary skill in the art will consider the above description and use the numerous inventions described above to teach the techniques of massively parallel systems and programming. It helps us to think about how we can make progress by the time the cost of such a system becomes much lower. A system of the invention type can be manufactured for a large number of people, not just a few, because it can be manufactured at a commercial sector level of procurement at an affordable cost.

[Brief description of drawings]

FIG. 1 is a diagram showing a parallel processor processing element using a conventional technique.

FIG. 2 is a diagram illustrating parallel processor processing elements utilizing conventional technology.

FIG. 3 is a diagram showing a structural unit of a large-scale parallel processor, which represents a novel chip design of the present invention.

FIG. 4 shows a preferred chip physical cluster layout for the preferred embodiment of the chip single node dense parallel processor of the present invention on the right side and an alternative technique on the left side. Here, each chip is a scalable parallel processor chip with CMOS DRAM memory and logic, providing 5 MIPS performance, enabling air-cooled implementation of massively parallel systems.

FIG. 5 is a functional block diagram of a computer processor according to the present invention.

FIG. 6 is a diagram showing a typical APAP computer system configuration.

FIG. 7 shows a system overview of the dense parallel processor technology of the present invention, exemplifying a system construction using replication of PME elements that enables the development of systems with 40 to 193840 MIPS performance.

FIG. 8 shows hardware for processing element (PME) data flow and local memory according to the present invention.

FIG. 9 is a diagram illustrating a PME configured as a general-purpose computer connected to a bird wire and performing fixed-point processing of about 5 MIPS or 0.4 M by program-controlled floating-point arithmetic.
FIG. 3 is a diagram showing a PME data flow that realizes FLOPS.

FIG. 10 is a diagram showing inter-PME connections (binary hypercubes) and data paths that can be used in accordance with the present invention.

FIG. 11: Eight PMs that manage a single external port each, enable distribution of network control functions, and eliminate bottlenecks in functional hardware ports
FIG. 7 shows a node interconnect for a chip or node with E.

FIG. 12 is a 16-bit wide processor where each PME is a 32K word local memory with I / O port operation for the broadcast port providing an interface between the controller and all features; The external port is
FIG. 3 illustrates a scalable parallel processor chip, which is a bidirectional point-to-point interface that allows ring torus connections within and outside the chip.

FIG. 13 illustrates the preferred embodiment array director.

FIG. 14 shows an example in which an edge of a cluster is connected to a system bus (FIG. 15).
FIG. 3 shows a system bus to and from a cluster array bond, which allows the loading and unloading of arrays by connecting to (reference).

FIG. 15 shows a bus to and from the processing element part.
14 and 15 show how multiple system buses are supported by multiple clusters. Each cluster can support a bandwidth of 50-57 MB.

FIG. 16 is a diagram showing “zipper connection” for high-speed input / output connection.

FIG. 17 is a diagram showing an 8th order hypercube connection showing the packaging technique according to the present invention applied to the 8th order hypercube.

FIG. 18 is a diagram showing two independent node connections in a hypercube.

FIG. 19 illustrates the Bitonic Sort algorithm as an example and illustrates a predefined SIMD / MIMD processor
It is the figure which showed the advantage of the system.

FIG. 20 is a system block diagram of a host-attached large scale system with one application processor interface. This figure can also be seen with the understanding that the present invention can be used in a stand-alone system using multiple application processor interfaces. Such an interface in FIG. 20 supports DASD / graphics on all clusters or multiple clusters. Workstation Accelerator is a host, application processor
The interface (API) and cluster synchronizer (CS) can be eliminated. cluster·
Synchronizers are not required for all examples.

FIG. 21 is a diagram showing a software development environment of the system of the present invention. The program can be created by and executed by the host application processor. Both the program and the machine debug are
Supported on base console. Both of these services are real MMPs or simulated MMs.
It supports applications running on P and allows applications to be developed both at the workstation level and on the supercomputer style APAP MMPs. This common software environment enhances programmability and distributed use.

FIG. 22 shows the programming levels enabled by this new system. As different users require more or less detailed knowledge, software systems are developed to support these needs. At the highest level, the architecture is actually MM
The user does not need to know that it is P. This system can be used with existing language systems for program partitioning, such as parallel FORTRAN.

FIG. 23: MM provided by the above APAP configuration
FIG. 2 shows a parallel FORTRAN compiler system for P. The Sequential-Parallel compiler system uses a combination of existing compiler features and new data allocation features to enable partitioned programs such as FORTRAN D.

FIG. 24 is a diagram of a APAP workstation application in which APAP becomes a workstation accelerator. The device is RISC / 6000
Note that although it is the same physical size as model 530, this model now has an MMP connected to the workstation via the illustrated bus expansion module.

FIG. 25 shows an application for an APAP MMP module for AWACS military or commercial applications. This is one way to efficiently handle the problem of conventional distributed sensor fusion shown in this figure. Here, the nearest neighbor, two-dimensional line assignment (Mun
kes), stochastic data association, multiple hypothesis testing, etc.
These can now be implemented with the improved method shown in FIGS.

FIG. 26 illustrates how the system enables an n-dimensional assignment problem to be processed in real time.

FIG. 27 is a diagram showing a processing flow of an n-dimensional assignment problem using APAP.

FIG. 28 shows how the unit uses only 8-10 enhanced SEM-E modules for 424 MFLOPS
Or achieve 5120 MIPS and only about 0.017
FIG. 6 shows an expansion unit provided by the system enclosure described above, showing whether m ³ can provide performance comparable to that of a special signal processor module. The system is SIMD with 1024 parallel processors executing 2 billion instructions per second (GOPS).
It can be a large machine and can be expanded by adding 1024 additional processors and 32 MB of additional storage.

FIG. 29 is a diagram showing APAP packaging of a super computer. This is a large system that is comparable in performance to other systems, but has a much smaller footprint than other systems. This can be constructed by replicating the APAP cluster in a storage device such as those used in smaller machines.

FIG. 30 is a diagram showing a register / register (RR) format.

FIG. 31 is a diagram showing a direct address (DA) format.

FIG. 32 is a diagram showing a register storage area (RS) format.

FIG. 33 is a diagram showing a register immediate value (RI) format.

FIG. 34 is a diagram showing an inter-storage area (SS) format.

FIG. 35 is a diagram showing a special (SPC1) format.

FIG. 36 is a diagram showing a special (SPC2) format.

[Explanation of symbols]

200 Application Processor 240 Test / Debug Device 250 Array Director 300 Single Processor Unit 301 32K Halfword Memory 302 16-bit Processor 310 Network Node 313 Network Router 314 Signal I / O 405 AR Register 406 Multiplexer 420 Memory 460 Arithmetic logic 630 Application processor interface 640 Cluster controller 650 Cluster synchronizer

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor James Warren Defendelfer USA 13827, Front Street, Owego, NY 396 (72) Inventor Peter Michael Koche USA 13760, Endicott, NY Dorchester Drive 7 (72) Inventor Nicholas Jerome Schonover, USA 13845, P.O. Box 18 at Tioga Center, NY

Claims (20)

(57) [Claims] 1. A plurality of processing units, each processing unit having its own memory and instruction stream, said instruction stream being autonomously executable on its own code stream by a MIMD implementation. When and when it is actually executed and more than one element is assembled under the control of the processor to manipulate SIMD instructions, the element can execute a copy of the same instruction stream and synchronize the elements. Even if multiple processing units execute instructions in SIMD mode at the same time, other processing units can
Arrays that can execute instructions in modes other than MD mode
Computer system for the processor. 2. Computer system according to claim 1, characterized in that the processing elements of the machine are able to emulate the SIMD architecture with MIMD elements capable of implementing SIMD. 3. The computer of claim 1, wherein the system provides SIMIMD functionality that causes a small amount of program code executed by each picket to be loaded into picket memory. system. 4. An array controller to an array of pickets,
Or from a single picket to the array controller provided with a slide bus used to broadcast data for use by the controller or for rebroadcasting to the array, The computer system according to claim 1. 5. The computer system of claim 1, wherein reduction is provided for use with inter-processor communication, taking data values and combining them to reduce the number of data values. . 6. Computer system according to claim 1, characterized in that there is a separate control network and data network. 7. A computer system according to claim 1, wherein each processing element comprises means for broadcasting from a node or a processing element within a node. 8. The computer system of claim 1, wherein the processing element or array controller can assign the processing element to one or several of the plurality of groups. 9. The computer system of claim 1, wherein each processing element includes a status latch that controls processing. 10. The computer of claim 1, wherein vector elements can be handled within each processing element and the nodes and vector instructions can be in a short 32-bit format or a longer format. system. 11. A status funnel is provided, wherein the array controller uses the status funnel to collect status from active processing elements, which sends cumulative results to the array controller. The computer system described in. 12. The status funnel of claim 1, wherein all processors can indicate that their processing steps have been completed and that the next operation can proceed in response to a command from the controller. Computer system. 13. The computer system of claim 1, wherein single and multiple tasks can be performed and processing partitions, nodes and processing elements can be time shared. 14. A node is capable of fetching from the same address in processor memory to execute the same instruction in SIMD mode, or from individually selected addresses to execute instructions in independent MIMD mode. The computer system of claim 1, wherein the computer system is capable of being. 15. A V in which a processing element resides in a node of the system.
The computer system according to claim 1, wherein the computer system is an LSI chip RISC processor system, each processing element including a local memory and a global memory. 16. The computer system according to claim 1, wherein each element is a memory, and the processor can retrieve the SIMD style from the same address or the MIMD style from an individually selected address. . 17. An array processing system comprising a plurality of processing elements, each having a processor and a memory, each of the processing elements being selectively and autonomously independent of a plurality of data streams. For providing a MIMD mode and thereby for instructing said processing element to execute on a plurality of independent data streams, one for each processing element. A controller for dispatching a single instruction stream to multiple processing elements, the processing elements executing the single instruction stream regardless of a fixed time relationship between or among them. An array processing system having: 18. An array processing system comprising a plurality of processing elements interconnected as an array processor, each processing element having a processor and a memory connected to the processor, each of the processing elements being selected. And autonomously executing independent instruction streams on independent data streams, thereby
Providing an IMD mode, and providing a single instruction stream to the plurality of processing elements to instruct the processing elements to execute on multiple independent data streams, one for each processing element. An array processing system comprising: a dispatching controller, wherein the processing elements execute the single instruction stream regardless of a fixed time relationship between or within them. 19. A parallel array processing system, comprising a plurality of processing elements, each having a processor, a memory, and a data path interconnecting the processor and the memory, and each of the processing elements. A single instruction stream to the plurality of processing elements for execution on the plurality of independent data streams, one for each processing element, in memory of , A controller thereby providing a SIMD mode, an interconnection network interconnecting the plurality of processing elements for communication between or within the processing elements and between the controller and the processing elements. And each of the processing elements is selectively and autonomously independent of its own memory in its respective memory. Execute an independent instruction stream over a number of data streams,
A parallel array, thereby providing a MIMD mode, each processing element having means for training local autonomy to selectively discontinue participation in a broadcast or task. Processing system. 20. An array processing system comprising: an array controller, data processing means having a processing element each having a memory connected to a processor; and cycle-by-cycle control of the array controller in SIMIMD mode. A plurality of independent data streams, one for each processing element, each executing means within the plurality of processing elements for performing the original instructions, wherein each individual data stream is executed by a respective processor. Execution of the plurality of independent instruction streams above is controlled by the SIMD instruction stream and that every processing element has completed execution of each of the plurality of independent instruction streams is controlled on a cycle-by-cycle basis. An array processing system comprising: a unit shown in the array controller as a reference.