DE102022116882A1 - Computer-implemented method for parallelizing data access and circuit for parallelized data access - Google Patents

Abstract

The invention relates to a computer-implemented method for parallelizing data access using a circuit, in particular a user-programmable integrated circuit, which has a network with a plurality of connection units, storage units and processing units that can be selectively connected to one another. The method includes providing a quantity of processing data to be processed; providing a set of reference data, the reference data indicating data keys of the processing data for a data value result to be retrieved, in particular as a lookup table; dividing the reference data into N subsets of reference data, where N is a number of processing subunits (3) included in the circuit, and storing a reference data subset in a memory of each respective processing subunit (3); storing a subset index, which assigns the reference data subsets to the respective processing subunits (3), in a first upper processing unit (1); entering the processing data into the first processing upper unit (1); entering subsets of the processing data as data keys via the first upper processing unit (1) into the corresponding processing subunits (3) using the subset index; outputting data value results corresponding to the data keys according to the reference data subset as retrieved data by means of the processing subunits (3) to a second processing upper unit (5); and outputting an aggregated amount of retrieved data, comprising an aggregation of all data value results of all processing subunits (3), via the second processing upper unit (5). The invention also relates to a circuit, in particular an FPGA.

Description

The invention relates to a computer-implemented method for parallelizing data access. The invention also relates to a circuit for parallelized data access.

Traditionally, computer processing has been characterized by a CPU and memory, resulting in a phenomenon known as the “von Neumann bottleneck.” Over the last decade, more CPUs and/or more cores have been added to computers. In addition to more cores, more memory channels have been added, albeit at a slower speed (e.g. 128 CPUs with 16 memory channels). Further, a technology known as NUMA (non-uniform memory architecture) makes these memory channels explicit. However, these solutions provide a single logical memory with a single global address mapping.

Thus, in conventional processing, a problem arises that, regardless of the number of CPUs, the processing speed and/or the access speed is severely limited by a memory bottleneck, since all CPUs or cores must access this single logical memory.

Although the number of (DDR) memories has been increased, albeit by a smaller factor than the number of CPUs/cores, their organization is still that of a single logical memory, commonly referred to as "main memory". High-bandwidth memory (HBM) simply offers more channels than DDR in this regard.

Furthermore, FPGAs (field programmable gate arrays) are conventionally known. These have thousands of storage units and therefore provide an enormous bandwidth, sometimes two or more orders of magnitude higher than HBM.

However, FPGAs are traditionally programmed as hardware by allocating space in a “main memory,” i.e., a single logical storage unit.

Essentially, none of the traditional methods, either in a software implementation on conventional computers or in a hardware implementation in FPGAs, provide true parallel data access and/or processing, since they only rely on single logical storage units. Therefore, these conventional methods are limited in access and processing speeds by the von Neumann bottleneck.

It is an object of the present invention to provide parallelized, in particular massively parallelized, data access and processing.

This problem is solved by the features of the independent claims. The subclaims relate to advantageous embodiments of the present invention.

In particular, the solution to this problem is provided by a computer-implemented method of parallelizing data access using a circuit. The circuit is in particular a user-programmable integrated circuit, for example an FPGA or ASIC. The circuit has a network with a plurality of connection units that can be selectively connected to one another, which can be connected to one another by the user (for the exemplary case of an FPGA) or can be connected to one another during production (for the exemplary case of an ASIC). The present invention is not limited to user-programmable circuits. For example, and as will be described in further detail below, the method described below may also be implemented on a conventional computer having a plurality of CPUs and/or GPUs and a plurality of memory units.

• Eine Menge von zu verarbeitenden Verarbeitungsdaten wird bereitgestellt. Diese Verarbeitungsdaten sind durch die Schaltung zu verarbeiten.
• Eine Menge von Referenzdaten wird bereitgestellt. Dabei indizieren die Referenzdaten (d.h., stellen einen Index bereit von/für) Datenschlüssel der Verarbeitungsdaten zu einem Datenwertergebnis. Vorzugsweise sind die Referenzdaten eine Nachschlagtabelle, herkömmlich als „LUT“ bezeichnet. Die Nachschlagtabelle korreliert vorzugsweise einen Datenschlüssel der Verarbeitungsdaten mit einem Datenwertergebnis.

The procedure includes the following steps:

• A lot of processing data to be processed is provided. This processing data is to be processed by the circuit.
• A lot of reference data is provided. The reference data indexes (ie, provides an index of/for) data keys of the processing data to a data value result. Preferably, the reference data is a lookup table, commonly referred to as a “LUT”. The Lookup table preferably correlates a data key of the processing data with a data value result.

The reference data (preferably the lookup table) is divided into N subsets of reference data. Where N is a number of processing subunits included in the circuit. Further, a reference data subset (i.e., one of the N reference data subsets) is stored in a memory of each respective processing subunit. This division is also known as “de-aggregation”.

A subset index is stored in a first processing master unit. In this respect, “upper” refers to the opposite of “under”, i.e. a sub-unit is subordinate to a superior unit and vice versa. The subset index assigns the reference data subsets to respective processing subunits. In other words, the subset index provides an address indicating at which of the N subunits the respective subset of reference data is stored.

The processing data is input into the first upper processing unit.

The first processing master unit inputs subsets of processing data as data keys into the corresponding processing subunits using the subset index. In other words, the first processing master unit divides processing data (and/or receives a stream thereof) into processing data subsets. The processing data subsets correspond to or correlate with the data keys. Using the subset index to find the address (i.e., "which subunit") of the processing subunit that has the corresponding data value result, the first processing subunit inputs the subset into the corresponding processing subunit.

The processing subunits output the data value results corresponding to the data keys according to the reference data subset. The output data value results are called “retrieved data” because they are retrieved from the memory of the processing subunits. The processing subunits output the data value results to a second processing subunit.

The second processing unit outputs an aggregated set of retrieved data (i.e., data values). The aggregated set of retrieved data includes an aggregation of all data value results from all processing subunits.

Thereby, the present invention provides the technical advantage of both parallelizing the reference data and accessing and processing it. Since the reference data is divided into N subsets corresponding to the number of processing subunits, no single storage unit includes the entirety of the reference data. Thus, the “von Neumann bottleneck” described above is avoided and the processing of the processing data and/or the retrieval of the reference data are highly parallelized.

Preferably, the aggregated set of retrieved data is output as one set. More preferably, the aggregated amount of retrieved data is output as a stream.

Preferably, the first upper processing unit and/or the second upper processing unit are contained in the circuit, in particular in the user-programmable integrated circuit. On the other hand, the first upper processing unit and/or the second upper processing unit are preferably included in separate circuits.

Preferably, the first upper processing unit is connected to the second upper processing unit. In other words, the circuit preferably has a connection (preferably as an activated connection), in particular a direct connection, between the first upper processing unit and the second upper processing unit. Preferably, the method further comprises outputting, from the first upper processing unit to the second upper processing unit, an order in which the subsets of processing data are input by the first upper processing unit to the corresponding processing subunits. The method further preferably comprises sorting the data value results corresponding to the order output by the first processing upper unit by means of the second processing upper unit. In other words, the first communicates Processing master unit the order in which the processing data subsets (data keys) have been issued to the processing subunits to the second processing master unit, and the second processing master unit preferably sorts the received data value results in this order.

Preferably, the first processing unit also outputs to the second processing unit the index (or address) of the corresponding processing subunit to which the respective processing data subset (data key) is output. As a result, the second upper processing unit knows in simple terms which processing subunit is to expect a data value and in what order.

The circuit preferably also has N first connection units. The first connection units each connect a processing subunit to the first processing unit, in particular directly. The first connection units each have a storage unit. The method is further characterized by inputting and buffering a plurality of subsets of processing data as data keys into the storage unit of the corresponding first connection unit by means of the first upper processing unit. In other words, as an example, if and when the first processing master unit processes multiple processing data subsets to be output to the same subunit according to the subset index, they are output to the storage unit of the first connecting unit connecting the two, thereby acting as a buffer for the multiple processing data subsets. This has the further advantage of further increasing parallelization as it will ensure or promote that processing subunits always have a processing data subset or data key "waiting" in the buffer at their input, and thus the subunits process without interruption.

Preferably, the first connection units each have a buffer processing unit. The method further preferably includes inputting processing data or data keys from the storage unit of the first connection unit to the respective processing subunit according to a first-in-first-out (“FIFO”) principle by means of the buffer processing unit. This has the advantage that the buffer storage unit of the first connection units can have several data keys without losing any sorting, in particular a temporal sorting.

In an advantageous embodiment, the circuit further has N second connection units, each of which connects a processing subunit to the second processing unit. Preferably, the second connection units each have a storage unit. Preferably, the method includes inputting and buffering a plurality of data value results in the storage unit of the corresponding second connection unit by means of the processing subunit.

As with the first connection units, the second connection units preferably each have a buffer processing unit. The method preferably includes inputting the data value results from the storage unit of the second connection unit to the second upper processing unit according to a first-in, first-out principle.

Preferably, the method includes calculating a hash value using a hash function for each subset of processing data, in particular by means of the first upper processing unit.

Furthermore, the reference data preferably has hash values as the data keys. In other words, the reference data preferably correlates hash values with data value results, in particular via a LUT.

Furthermore, the hash value preferably includes the subset index, which assigns the reference data subsets to the respective processing subsets and the data key for the reference data.

In other words, the hash value preferably comprises multiple bits of information, with one or more bits used to identify the corresponding subunit.

As an example: The hash function transforms a subset of processing data into a hash value. The subset of processing data can, for example, be a word, for example Example: “Einstein”. The hash value will have multiple entries or bits, for example XXYYYYYY. The In this example, the reference data correlating the data key YYYYYY includes a definition of “Einstein”, namely, for example, “eminent physicist”. The division of reference data can be achieved in this example by an alphabetical division, so one subunit will have all the reference data corresponding to all words starting with "A", and another subunit will have all the reference data starting with "B", and so on further. The value XX = 05 can therefore also be thought of as addressing a fifth subunit in which all words beginning with “E” are contained in the reference data. Thus, the first superunit converts "Einstein" to "05YYYYYY" and uses the subset index (in this case a LUT with entry "05" = "fifth subunit") to find the subunit that has the reference data for "Einstein" and to issue to them. This data key "05YYYYYY" is then entered into the sub-unit and the sub-unit begins to search for the reference data that correlates to "05YYYYYY", that is, it searches for an entry for "Einstein". Once he is found, she will say “outstanding physicist”.

This example further demonstrates the massive parallelization achieved by the present invention, since no single processing unit needs to search or retrieve the entirety of reference data, in this example a dictionary or thesaurus (or a history book, Wikipedia, etc.). Instead, subsets of reference data (e.g., all words beginning with "E") are stored in the memory of different subunits, and thus each subunit only needs to search within its respective subset. Furthermore, the subunits can search in parallel since no individual memory needs to be accessed (each subunit accesses its own memory). The hash function provides a more even distribution of subsets of reference data, which in the exemplary case of the dictionary means that there is no need to strictly adhere to an alphabetical order.

Furthermore, the reference data is preferably only stored in the processing subunits. Preferably no reference data is stored in the main units. Preferably, the first superunit only stores the subset index.

This further increases parallelization since the size of the reference data retrieved per search or query (entering a data key) is only of size 1/N multiplied by the total size of the reference data when a uniform division and distribution among the subunits as one exemplary embodiment is adopted. Thus, particularly in the case of orders of several hundred to thousands of processing subunits and their respective storage units, the relative size of reference data retrieved is small. In this exemplary case of each subunit performing a binary search, as opposed to a processing unit performing a binary search on the entirety of reference data (in a single logical memory), this achieves massive processing speed increases.

Inline initialization:

• Eine Menge von Referenzdaten wird bereitgestellt. Darin indizieren die Referenzdaten (d.h., stellen einen Index bereit von/für) Datenschlüssel von zu verarbeitenden Verarbeitungsdaten zu einem Datenwertergebnis. Vorzugsweise sind die Referenzdaten eine Nachschlagtabelle, allgemein als „LUT“ bezeichnet. Die Nachschlagtabelle korreliert vorzugsweise einen Datenschlüssel der Verarbeitungsdaten mit einem Datenwertergebnis.

The present invention also relates to a computer-implemented method of inline initialization of the circuit. This procedure includes the following steps:

• A lot of reference data is provided. Therein, the reference data indexes (ie, provides an index of/for) data keys of processing data to be processed to a data value result. Preferably, the reference data is a lookup table, commonly referred to as a “LUT”. The lookup table preferably correlates a data key of the processing data with a data value result.

The reference data (preferably the lookup table) is divided into N subsets of reference data. Where N is a number of processing subunits included in the circuit. Further, a reference data subset (i.e., one of the N reference data subsets) is stored in a memory of a respective processing subunit.

A subset index is stored in a first processing master unit. In this respect, "upper" refers to the antonym of "under", that is, a sub-unit is subordinate to a superior unit and vice versa. The subset index assigns the reference data subsets to the respective processing subunits. In other words, the subset index provides an address indicating at which of the N subunits the respective subset of reference data is stored.

This initializes the circuit inline. In other words, no additional circuit is necessary to initialize this circuit. In a traditional non-inline initialization, many components of the FPGA would have to be activated and thus “used” only for the purpose of initialization. The present invention circumvents this problem by performing the initialization using the same components (enabled components of the FPGA) for the initialization as for the actual data processing/access.

Preferably, in the initialization process, the processing subunits do not output any data value results. Preferably, the processing subunits output a signal indicating successful initialization.

After initialization, the method described above for parallelizing data access can be carried out, in particular by inputting processing data to be processed into the initialized circuit.

The above methods and their embodiments are most preferably implemented in an FPGA. Thereby, the present invention achieves highly parallelized processing in an FPGA.

The present invention further relates to a circuit for parallelized data access. The circuit is preferably a user-programmable integrated circuit, in particular an FPGA. The circuit has: a first upper processing unit configured to receive a quantity of processing data to be processed; a plurality of processing subunits configured to store and retrieve subsets of reference data to be retrieved, the reference data indexing data keys of the processing data to a data value result, in particular as a lookup table, each processing subunit configured to store and retrieve a reference data subset; and a second processing master unit configured to receive data value results corresponding to the data keys according to the reference data subset as retrieved data from the processing subunits, and configured to aggregate the data value results and output the aggregated set of retrieved data. The first upper processing unit is set up to process the amount of processing data, in particular via a hash function, and is set up to use a subset index, which assigns the reference data subsets to the respective processing subunits, in order to input the corresponding subsets of processing data to the corresponding processing subunit.

Preferably, the above-mentioned circuit is set up to carry out the above-described computer-implemented method for parallelizing data access.

Preferably, the above-described computer-implemented method of parallelizing data access using a circuit and the above-described computer-implemented method of initialization and their embodiments are achieved through a hardware implementation, for example through a hardware description language (HDL). In other words, the methods described above are preferably not only carried out as software on a conventional computer, but correspond to a hardware implementation. Therefore, the above functional descriptions, in particular the method steps, preferably characterize the circuit on a hardware level and vice versa.

The above functional descriptions, for example of the “processing upper units”, “subunits” and “connection units” are preferably understood as logical blocks, in particular of an FPGA architecture. For example, the first processing head unit is preferably a de-multiplexer and the second processing head unit is preferably a multiplexer.

Preferably, the memory units described above are not shared memories, in particular not distributed shared memories. Preferably, the storage units have non-shared local address spaces. Of course, each local address space works independently. The memory is preferably mapped directly to URAM, DRAM, AlE, HBM, DDR channels, NUMA.

• 1 ein schematisches logisches Schaltungsdiagramm einer Schaltung gemäß einer ersten Ausführungsform der vorliegenden Erfindung; und
• 2 ein schematisches logisches Schaltungsdiagramm einer Schaltung gemäß einer zweiten Ausführungsform der vorliegenden Erfindung.

Further details, advantages and features of the present invention will become clearer from the following description of embodiments in view of the figures. It shows:

• 1 a schematic logic circuit diagram of a circuit according to a first embodiment of the present invention; and
• 2 a schematic logic circuit diagram of a circuit according to a second embodiment of the present invention.

As an introduction, reference is made to Table (1) below: Table (1) Storage type Number of units Custom Size (GB) Total Bandwidth (GB/s) GDR 2 32 38 HBM 32 8th 460 URAM 960 0.27 4320 BRAM 2160 0.038 19440

As an example, a general computer employs two units of 32 GB DDR memory, resulting in a total bandwidth of 38 GB/s. Furthermore, an HBM leads to a total bandwidth of 460 GB/s.

On the other hand, an FPGA example has 960 units of URAM (“Ultra RAM”) and 2160 units of BRAM (“Block RAM”). These on-chip memory resources are conventionally known.

Essentially, with respect to the conventional art, the present invention aims at allocating space to all (DDR, HBM, URAM, BRAM) memory units (on-chip memory resources) of the above table in an FPGA for parallel access and processing achieve.

The following two embodiments of the present invention are aimed at implementing the present invention on an FPGA.

1 shows a schematic logic circuit diagram of a circuit according to a first embodiment of the present invention. In particular shows 1 a logic circuit diagram of an FPGA as a circuit in which the computer-implemented method for parallelizing data access is implemented. 1 also demonstrates a sequence of the process steps described below.

How out 1 can be seen, the circuit has a first upper processing unit 1, a plurality of processing subunits 3 and a second upper processing unit 5. The first upper processing unit 1 and the respective processing subunits 3 are directly connected via first connection units 2. Furthermore, the respective processing subunits 3 are connected directly to the second processing upper unit 5 via second connection units 4.

The first upper processing unit 1 receives an input 9. The second upper processing unit 5 outputs an output 10.

In the present embodiment, each of the processing subunits 3 includes a storage unit and a processing unit. The first upper processing unit 1 and the second upper processing unit 5 each have a processing unit and preferably also have a storage unit. The first upper processing unit 1 is a de-multiplexer (also referred to as “de-aggregate”). The second upper processing unit 5 is a multiplexer (also referred to as “aggregate”).

Furthermore, each of the first connection units 2 has a memory unit, also called a buffer memory unit. Furthermore, each of the second connection units 4 has a memory unit, also called a buffer memory unit.

The circuit according to the present invention may have a large plurality of subunits 3, as shown by the dotted line in 1 marked. For example, the circuit may have several thousand subunits 3, as introduced above.

In the present embodiment, the invention is described as an implementation example with respect to dictionary learning. For example, the circuit may assign meaning to all words in a book based on dictionary entries for each word or phrase in the book. In this case, the book will henceforth be referred to as “processing data” and the dictionary will henceforth be referred to as “reference data”. The following implementation example of books and dictionaries is only for the purpose of explaining the embodiment and should not be construed as an implementation limitation of the present embodiment. In fact, the present embodiment can be implemented for any process in which data, particularly large amounts of data, needs to be processed/retrieved quickly and efficiently.

In the conventional art, a processing unit, for example a CPU, would search for the definition of a word in the entire dictionary stored in a single logical memory (“main memory”). During this search, it would generally not be possible to search for another word (processing data subset) at the same time.

• Die Verarbeitungsdaten werden bereitgestellt.

In contrast, in the present embodiment, the following method is implemented as hardware, in particular as logical cells of an FPGA:

• The processing data is provided.

Reference data is also provided. The reference data index data keys of the processing data to a data result to be retrieved. The reference data is a lookup table (LUT) of the FPGA. In other words, the reference data is a lookup table that correlates a data key with a data result. For example, in a dictionary implementation, the word “Einstein” would be correlated with “outstanding physicist.”

The reference data is divided into N subsets of reference data. N is the number of processing subunits 3. In the case of 1 Seven subunits 3 are shown regardless of the dashed line. Therefore, the reference data would be divided into seven subsets. However, in real applications the number N would be, for example, several thousand (see table (1) above).

Each of the N reference data subsets is stored in a subunit 3. In other words, each subunit 3 in memory has a subset of reference data.

Furthermore, a subset index is stored in the first upper processing unit 1. The subset index assigns the reference data subsets to the respective processing subunits 3. In other words, the subset index is an address of a subunit 3 indicating which subset of reference data is stored therein. For example, this subset index will indicate that the "sixth" reference data subset is stored in the "sixth" subunit 3.

In real applications, reference data separation and subset index mapping are achieved using hash functions, so that the distribution of reference data among subunits 3 is balanced and overall entropy is increased. For example, in the exemplary case of the dictionary, instead of storing all reference data for words starting with “A” in one subunit 3 and storing all words starting with “Z” in another subunit 3, the amount of in each subunit 3 would greatly imbalance the necessary memory, the reference data is divided so that all words are evenly distributed among all subunits 3. As a result, in the example, a subunit 3 can have all words starting with "Z" but also some words starting with "Y", whereas another subunit 3 only has some of the words starting with "A".

Further, the processing data is input into the first upper processing unit 1, i.e. the de-multiplexer 1.

The first processing unit 1 calculates a hash value via the hash function(s) for subsets of processing data. In the exemplary case, this would correspond to calculating a hash value for each word.

This hash value is also called a “data key”. As indicated above, the hash value for the reference data is also calculated so that the lookup table has hash value entries and their correlated or corresponding data values. For example, the hash value of “Einstein” is correlated with “eminent physicist” in the lookup table.

Furthermore, the data key hash value has entries that indicate a subset of reference data. As a result, the data key has information about which subunit 3 comprises the subset of reference data, which correlates the corresponding data key with the data values. This would essentially correspond to the hash value entry for the letter “E” in the alphabetical division of the reference data described above.

The data key, which is calculated by the first upper processing unit 1 via the hash function, is then given to the first connection unit 2. The subset index indicates to which subunit 3 (to which corresponding first connection unit 2) the data key is to be issued by the first processing unit 1.

The first connection unit 2 each has a storage unit and a processing unit, also called a buffer storage unit and a buffer processing unit. This allows multiple data keys to be entered into the first connection unit 2 (“buffering”). Furthermore, the buffer processing unit will output the data key from the first connection unit 2 to the corresponding subunit 3 according to the first-in-first-out (“FIFO”) principle.

Then the subunit 3, which has received the data key, searches for the data key in the subset of reference data. Once found, subunit 3 outputs the data value that corresponds to or correlates with this data key (e.g., “outstanding physicist”). This data value is also referred to as “retrieved data” because it has been retrieved by subunit 3.

The subunit 3 passes this data value to the second connection unit 4, which also has a storage unit (buffer memory) and a processing unit (buffer processing unit). This allows the data values output by the subunit 3 to also be buffered and sorted according to FIFO.

The second connection unit 4 outputs the data values to the second upper processing unit 5 in the FIFO order. The second upper processing unit 5 aggregates all data values of the subunits 3 into a set of aggregated data and/or a stream of aggregated data and further outputs this aggregation of the retrieved data to the output 10.

In particular, it should be noted that all of the reference data is stored in the subunits 3. In other words, no reference data is stored in the main units 1, 5. As a result, the upper units 1, 5 do not search for the data key in the reference data. Instead, each subunit 3 searches for the data key in its own subset of the reference data. Since the circuit is provided with the buffer memories, in particular the first connection units 2, the first processing unit 1 can continue to quickly output a large number of data keys. These data keys are preferably (almost or in the best case) evenly distributed among the subunits 3. This means that no downtime of one of the subunits 3 is necessary.

Furthermore, as indicated by the dashed line with reference numeral 6, in a preferred modification of the present embodiment, the circuit has a further direct connection between the first processing upper unit 1 and the second processing upper unit 5 in the form of a third connection unit 6.

The third connection unit 6 preferably also has a storage unit as a buffer memory and a processing unit as a buffer processing unit.

Via the third connection unit 6, the first upper unit 1 issues to the second upper unit 5 an order in which the data keys were issued by the first upper unit 1 to the subunits 3. For example, this can also be achieved by outputting parts or all of the hash values to the second upper unit 5. This allows the second upper unit 5 to store the data values, which are output by the subunits 3, sort in the order in which they were entered as data keys therein.

Preferably, the first upper unit 1 additionally outputs the address of the subunit 3 to which the data key was issued to the second upper unit 5 via the third connection unit 6. This allows the second upper unit 5 to identify, for example, whether an error has occurred or whether the corresponding subunit 3 has not found the data key in the reference data. In this case, if the subunit 3 simply does not output anything, the second superunit 5 can identify that the subunit 3 should have outputted a data value, and therefore can set the subsequent data values returned by the other (or the same subunit 3) to the zero value. (following output) are output, arrange them correctly.

The third connection unit 6 can preferably also output according to FIFO.

This achieves massive parallelization, since the reference data is not stored in its entirety in one place and can be searched efficiently and quickly by the subunits 3.

In a modern FPGA that uses embedded fabric memories (e.g. BRAM and URAM), this approach makes it possible, in the exemplary case of a dictionary covering virtually the entire English language, on the order of one word per clock cycle or even faster to identify/look up. This can result in a single FPGA capable of looking up 500 million words per second.

For a million words, a 32-bit hash function can already achieve few or no collisions.

2 shows a schematic logic circuit diagram of a circuit according to a second embodiment of the present invention.

The present embodiment is similar to the first embodiment. In addition, the second embodiment has an additional layer or hierarchy of subunits 3 and superunits 7, 8. The upper unit 7 is a de-multiplexer. The upper unit 8 is a multiplexer.

In this embodiment, a hierarchy explained in the first embodiment is essentially introduced instead of a subunit 3. This can be extended to any number of subunits 3 corresponding to the number of memory units in the FPGA.

In addition to the above written description of the invention, explicit reference is made to the 1 and 2 referenced, which show the features of the present invention in detail.

Additional Supplementary Charts:

Traditional concurrency

• • Pipeline
  • - only useful for streaming or other enumerable data (1, 2, 3, ..., N)
  • - an operation has a latency > 1 (time unit e.g. clock cycle)
  • - Intermediate results are saved and organized so that “later” stages work with “earlier” data (N, N+1, N+2, ...)
• • Map Reduce
  • - also referred to as “embarrassingly parallel”.
  • - parallel processes are copies of each other, only input data differs (N, N+1, N+2, ...)
• • In both cases there are no side effects, data dependencies or “state” are allowed

Special parallelism

• • Data dependencies, side effects and “state”
• • Systolic “pipeline”
  • - extended form of pipeline parallelism
  • - can be multidimensional
  • - characterized by multiple data movements (e.g. matrix cross product)
• • De-aggregate / Aggregate
  • - extended form of Map-Reduce
  • - parallel processes work on different independent data sets (e.g. dictionary)
• • In both cases, side effects, data dependencies or “state” are taken into account by the design

Information Entropy and Order&Hash

• • “Democracy”: 9 symbols of 8/16 bits (Unicode) = 80/160 bits (dynamic character string)
• • estimated # of words worldwide ≈ 1 billion → 32 bits sufficient to fully characterize in a linear manner
• • Huffmann: short (shorter) codes for common symbols
• • Hash: allow collisions or not?
  • → more bits needed if not
  • → etra logic if so
• • Spelling/language tolerance: Hidden Markov, Viterby
• • Cortical.io: “semantic folding” - hash function based on a meaning with a semantic order - i.e., similar meanings have a similar hash, “fingerprint”
• • Rarity - specific (e.g. Matrix) and systemic (all of the above)

De-aggregate/aggregate

• • Scatter-Gather and Map-Reduce have similarities but are not the same for (i=0; i<N; ++i) // or while(i) x[i] = y[idx[i]]; (please refer 3 )
• • this scheme has different operations in each c_0, c_1, ... etc., or at least different data
• while(i) push c_<i> // push into FIFO and get output from c_<i> after work completed
• • c_0, c_1, ... can contain/“form” distributed memory and work in parallel
• • simple example:
  • If there was a dictionary to store and look up words, then c_0 could contain all words with the first letter, c_1 with the letter B, and so on. If a word in an input stream started with A, it would go to c_0, words with B would go to c_1, and so on.
• • In reality, use hash functions to balance a distribution and increase entropy

De-aggregate/aggregate

• • In a modern FPGA that uses embedded fabric memory (e.g. Bram and Uram in Xilinx), this approach makes it possible to write on the order of one word per clock cycle, or even faster in a dictionary containing virtually the entire English language identify/ look it up (see 3 )
• • Example calculation:
  1. 1 million words - 32 bit hash with no/few collisions Xilinx Uram 4072 words, 72 bits → 8000 words per Uram Alveo u250 has 1280 Urams → space for >8 million words Binary search per Uram: log2(4072) = 12 clock cycles parallelism for a search: Up to >1000
• • → Lookup over 500 million words per second on a single FPGA

Inline initialization of distributed memory

• • Explicit initialization or upload of data causes the need to build additional hardware, which can be preventatively expensive and even exceed functional routing. (please refer 3 )
• • Initialization is carried out once or at least very irregularly, so timing is not critical.
• • Use of inline initialization/data upload - in other words, reuse of infrastructure, buses, etc. required for functionality by sending inline signals.
• • Each “node” (e.g. C_0, C_1, ..., etc.) interprets the signals to switch an operating mode to “initialization” and instead of executing the function (e.g. search) save the input data stream in a predetermined manner.
• • This eliminates the need to build additional hardware, particularly expensive buses, and allows content to be updated on-the-fly while normal operations can continue.

Hierarchical organization

• • Fan-out and fan-in: Number of end points of a signal but also number of connections of a module - e.g. aggregate has 16 inputs (see 4 )
• • Fan out/in over about 8-16 becomes increasingly difficult, inefficient and slow in placement and routes, but also in timing closure
• • For higher levels of fan out/in, use multi-level hierarchical approaches instead
• • For example, using 3 stages → 8 ³ = 512 parallel modules with a maximum fan out/in of 8
• • This particularly shows the usefulness of inline initialization (see slide above)
• • Each stage introduces 1-2 cycles of latency, but it is buffer stage latency so throughput is not affected. Reducing one clock cycle due to routing overhead would reduce throughput. (please refer 5 , parts 1 and 2)

Reference symbols

1

first upper processing unit

2

first connection unit

3

Processing subunit

4

second connection unit

5

second upper processing unit

6

third connection unit

7

Upper processing unit

8th

Upper processing unit

9

input

10

output

Claims (10)

Computer-implemented method for parallelizing data access using a circuit, in particular a user-programmable integrated circuit, which has a network with a plurality of connection units, storage units and processing units that can be selectively interconnected, the method comprising: • Providing a quantity of processing data to be processed; • Providing a set of reference data, the reference data indexing data keys of the processing data to a data value result to be retrieved, in particular as a lookup table; • dividing the reference data into N subsets of reference data, where N is a number of processing subunits (3) included in the circuit, and storing a reference data subset in a memory of each respective processing subunit (3); • Storing a subset index, which assigns the reference data subsets to the respective processing subunit (3), in a first upper processing unit (1); • Entering the processing data into the first upper processing unit (1); • Entering subsets of the processing data as data keys into the corresponding processing subunits (3) via the first upper processing unit (1) using the subset index; • Outputting data value results corresponding to the data keys according to the reference data subset as retrieved data via the processing subunits (3) to a second processing upper unit (5); and • Outputting an aggregated set of retrieved data, which includes an aggregation of all data value results of all processing sub-units (3), via the second processing sub-unit (5). Computer-implemented method for parallelizing data access according to Claim 1 , wherein the first upper processing unit (1) is connected to the second upper processing unit (5), the method comprising outputting an order in which the subsets of processing data are input from the first upper processing unit (1) to the corresponding processing subunits (3). the first upper processing unit (1) to the second upper processing unit (5) and sorting the data value results corresponding to the order by means of the second upper processing unit (5). Computer-implemented method for parallelizing data access according to one of the preceding claims, wherein the circuit further comprises N first connection units (2), each of which connects a processing sub-unit (3) to the first processing upper unit (1), the first connection units (2) each have a storage unit, the method comprising entering and buffering a plurality of subsets of processing data as data keys in the storage unit of the corresponding first connection unit (2) by means of the first upper processing unit (1). Computer-implemented method for parallelizing data access according to Claim 3 , wherein the first connection units (2) each have a buffer processing unit, the method comprising entering data keys from the storage unit to the respective processing subunit (3) according to a first-in-first-out principle by means of the buffer processing unit. Computer-implemented method for parallelizing data access according to one of the preceding claims, wherein the circuit further comprises N second connection units (4), each of which connects a processing sub-unit (3) to the second processing upper unit (5), the second connection units (4) each have a storage unit, the method comprising entering and buffering a plurality of data value results in the storage unit of the corresponding second connection unit (4) by means of the processing subunit (3). Computer-implemented method for parallelizing data access according to one of the preceding claims, further comprising calculating a hash value using a hash function for each subset of processing data, in particular by means of the first upper processing unit (1). Computer-implemented method for parallelizing data access according to Claim 6 , wherein the hash value comprises the subset index, which assigns the reference data subsets to the respective processing subunits (3), and the data key for the reference data. Computer-implemented method for parallelizing data access according to one of the preceding claims, wherein the reference data is stored only in the processing subunits (3). Circuit, in particular a user-programmable integrated circuit for parallelized data access, comprising: • a first upper processing unit (1) which is set up to receive a quantity of processing data to be processed; • a plurality of processing subunits (3) which are set up to store and retrieve subsets of reference data to be retrieved, the reference data indexing data keys of the processing data to a data value result, in particular as a lookup table, each processing subunit (3) being set up to assign a reference data subset store and retrieve; and • a second upper processing unit (5) which is set up to receive data value results corresponding to the data keys as retrieved data from the processing subunits (3) according to the reference data subset, and which is set up to aggregate the data value results and the aggregated set of retrieved to output data, where • the first upper processing unit (1) is set up to process the amount of processing data, in particular via a hash function, and is set up to convert the corresponding subset of processing data into the corresponding subset of processing data using a subset index, which assigns the reference data subsets to the respective processing subunits (3). to enter the corresponding processing subunit (3). Circuit according to Claim 9 , wherein the circuit is set up to implement the computer-implemented method according to Claims 1 until 8th in particular, to execute on a predetermined amount of processing data or a stream of processing data input into the circuit.

Images