KR100279740B1 - Code distribution method and apparatus for parallel processing - Google Patents

Abstract

In the present invention, a code distribution technique that can be used in a parallel computer is developed. The purpose of a parallel computer is to reduce flood time by dividing a large number of jobs into several microprocessors. To do this, we need a way to distribute the data efficiently to the processor and a method to process these data in parallel. It is an object of the present invention to develop a method and apparatus for distributing data stored in a code form to a processor in the parallel processing. Once code-type data is entered, it is stored in a distributed manner on a parallel processor in an efficient form for retrieval. When searching, the codes to be searched are distributed and searched. The retrieval characteristic that is the subject of the present invention is not only the retrieval object given as a query when performing retrieval but all the codes including the query code are retrieved. These codes are called equivalent codes. In the stage of storing the code, it is possible to increase the efficiency of the search by evenly dividing these peer codes into different processors. To do this, we distribute the code to the processor in a folding fashion so that the same code is searched in the processor as evenly as possible. By doing this, the execution time can be reduced when storing and retrieving data of the same code type in the parallel computer.

Description

Code distribution method and apparatus for parallel processing

The present invention relates to a method and apparatus for distributing data to a plurality of processors in order to process data in ASCII code or binary code form.

In order to process a certain amount of data, it takes a lot of time to run on a computer with a single processor, so that a parallel computer is developed. A parallel computer has a number of processors and processes a certain amount of data at the same time. However, it is very difficult to equally distribute the amount of data and distribute the same time. In addition, when there is a correlation between data, more problems are added. Thus, many techniques for parallelizing data have been developed and many research efforts have been made.

Conventionally, the code distribution method for parallel processing can not be found in the domestic case. In the case of overseas, the 19th "Very Large Database Conference" held in Ireland in 1993 We can hear the article "Hamming Filter: A Dynamic Signature File Organization for Parallel Stores" by Zezula et al. On pages 314-327 of the published paper. In this paper, we use Hamming Code, a linear error correcting code, to distribute the code.

The Hamming code is a method of transferring data in a method proposed in 1973 for transferring data, adding code to the data, receiving the data, and then using the added code to find the location of the error during transmission. In the Hamming filter, the Hamming code is made into a matrix, called a check matrix, and the number of the processor is determined by multiplying the code by the contrast matrix and obtaining the result. The creation of this contrast matrix follows the Hamming code generation rules, the number of rows is determined by the log value below 2 of the number of processors, and the number of columns is determined by the processor number minus one. That is, when a processor has four processors (3x2), a matching matrix is obtained, and 3 bits (bits) of the code are used to determine the processor number.

However, such a Hamming filter has the disadvantage that it achieves the best code dispersion only when a matching matrix is formed in the form determined above. In other cases, the efficiency is reduced, and there is a problem that the verification matrix must be obtained again every time the number of processors is changed.

However, the main purpose of performing parallel processing in data processing is to reduce the execution time by dividing a certain amount of work into a plurality of processors. For this parallel processing, there is a need to allocate the data to the processor equally, and to divide the execution equally among the processors. The criteria for measuring this are the data skew and the execution skew.

The amount of data to be executed indicates the amount of data to be executed on the processor, and the smaller the value, the more efficient parallel execution is achieved. Execution bias indicates the degree to which the amount of work to be executed is biased toward the processor. The smaller the value is, the more efficient parallel execution is performed, thereby reducing the execution time.

Accordingly, an object of the present invention is to reduce the execution time of a task by minimizing data bias and execution bias for parallel processing of a certain amount of data. In order to achieve the above object, the present invention provides a technique of allocating data to parallel processors so that the data bias and execution biases are minimized.

The characteristics of the data for parallel processing in the present invention are that the parallel codes for the query must be simultaneously processed by the parallel processor when performing the search. The nature of such a search is due to the requirement that the data to be searched for the query data not only matches the query but also finds everything that contains the query. These requirements are mainly found in key-based searches using hashing. Here, the key refers to the part of the code that is used for the search. Equivalent keys are defined as all keys containing the key of the query data in bitwise logical product form as follows.

EK (QK) = {Ki | (Ki & QK) = QK, 0? I <n}

Here, QK means the key of the query, and n is the number of all keys that can be generated in the length of the query key. When the length of the query key is L, For example, if the query key is 1010, the peer keys are 1? 1? That is, 1010, 1011, 1110 and 1111 are obtained. By evenly distributing this homogeneous key to the processing node, it is possible to minimize the biasing phenomenon.

In the parallel computer architecture used in the present invention, a processor for parallel processing is logically divided into a potential processor and a posterior processor.

When storing the data, the potential processor receives data from the user, determines the processor to which the data should be input, and then passes the data to the processor. The backend processor has both a disk that stores the actual data, its processor, and memory, and stores and manages the data.

When retrieving the data, the potential processor receives the query from the user, obtains the homogeneous key, passes the query data to the processor having the homogeneous key, and then collects the query result and transmits it to the user. The posterior processor performs the actual parallel processing search on the query document received from the potential processor and passes the result to the potential processor. These processors are connected to a high-speed network.

Accordingly, the present invention can distribute data in a code form based on the code, thereby minimizing quantitative and execution bias of data and maximizing the efficiency of parallel processing.

Brief Description of the Drawings Fig. 1 is a flow chart in which codes are distributed to a parallel processor by an overlapping method when the number of processors of the present invention is 8; Fig.

2 is a flow chart for determining a processor number to be stored by an input code according to the present invention;

3 is a flow chart for determining an assignment direction of an overlapping method to determine a processor to which a code according to the present invention is to be stored.

FIG. 4 is a block diagram of a stacked chip when a method for allocating a code according to the present invention to a processor is designed as a chip. FIG.

FIG. 5 is a block diagram of the internal structure of a bonded chip when a method for assigning a code according to the present invention to a processor is designed as a chip. FIG.

Description of the Related Art

100: stacking chip 110: code overlapping step (FS)

120: initial _K R calculation unit 130: two feet Rom

140: _RK calculation unit by repetition 150: Assignment direction determination unit

160: processor number determination unit 200: demultiplexer

300: Processor

The code distribution technique according to the present invention is a method of efficiently distributing data when storing data in a parallel processor in order to minimize execution bias in retrieving data.

The process of performing this distribution takes place in the dislocation processor, and in order to distribute the code, the order of the codes is basically set in the direction of ↓ ↑↑ ↓. The direction of this arrow is called the allocation direction of the code, ↓ is the forward direction, and ↑ is the reverse direction.

When the above step, that is, the fourth step, is exceeded, as shown in FIG. 1, an arrow is once again generated inside. This process is defined as folding. That is, when it becomes an array of codes corresponding to a multiple of 4, overlapping occurs once again inside, and this process is repeated until no further overlap occurs.

The smallest arrow in the column containing 69 in FIG. 1 represents the allocation direction of the two codes, the longer arrow is the position where the small arrow is placed in the order of magnitude, and likewise the longest arrow is the overall allocation direction. In this case, a repetitive overlap causes an arrow in three steps, and no more overlaps can occur. The number of occurrences (NF), ie the maximum number of arrows in a row, depends on the number of processors (NP) and is determined by NF = log2 (NP). For example, on a parallel computer with eight processors, the maximum number of overlaps is log2 (8) = 3. ↓ ↓ ↓ ↓ of the first arrow When the allocation direction is exceeded, the determination of the overall code allocation direction of the next column also follows the allocation direction of ↓ ↑↑ ↓. That is, the allocation direction is determined in the forward direction of 32, 33, 34, 35 in the middle, and in the forward direction of 36, 37, 38, 39 in the upper row. A large arrow in FIG. 1 means this process. For example, since the allocation direction of the column including 0 is the forward direction, the overall allocation direction of the column including 36 becomes the reverse direction.

A flow chart of the process in which the code is inputted and distributed to the processor is shown in FIG. 2 and FIG. 5 is an internal block diagram of an overlapping chip according to the present invention. As shown in FIG. 5, the code overlapping step FS is calculated based on the input code K, the number of processors N _p and the overlapping unit FU and code the overlapping step calculating unit 110, the code (K) and the number of processors beginning R _K calculation unit 120, a stacking unit FU, overlapping minimum code of calculating an initial R _K on the basis of the (N _P) FB, and two feet ROM 130 put in the overlap bit pattern BP stored in advance, and overlapping ultra low code FB and the code stacking stage FS and the portion R _K by repeatedly calculating the R _K by an iterative calculation based on the R _K value 140, the code (K) and a number of processors (N _P) and the overlap bit and the pattern BP determining assignment direction for determining the assigned direction on the basis of the portion (150), R _K calculated by the repetition unit (140 ) of the R value _K and outputs the determination to the processor numbers (PN) based on the assigned direction AD of the assigned direction-determining unit 150 It consists of a processor number determining unit 140. The

2, when the code type data is input, the overlapping unit FU and the minimum code FB and the overlap length FL are set as initial values The first step is to initialize the overlapping unit FU, which is the basic unit of overlapping, to a predetermined constant value. Since a new overlapping shape appears four times, FU = 4 (S1). Next, the lowest code FB is set to the initial value '0' (S2). This is the lowest code of the overlap and is the smallest code value in a column. And, the overlapping length reset the initial value of the FL to the number of processors, FL = _P N (S3).

The second step of obtaining the overlapping step FS, which is the number of times of overlapping of the codes, is performed after the first step. The duplexing step FS adds the code value K to the value obtained by multiplying the overlapping unit FU by the number of processors N _P And obtains the residual value by the log function of the number of processes N _P as the code overlapping step FS value (S4). That is, it is obtained by the following equation (1).

FS = (K / (N _P * FU))% log ₂ (N _P )

Then, the third step (S5) of obtaining the remainder of the code (K) with respect to the process number (N _P ) by the allocation number R _K is performed. And R _K denotes a processor number indicating how many of the processors are to be allocated.

Then the overlapping length decreasing the FL counter updates the minimum code FB, and that takes the lowest code FB assignment number R _K new assignment number R _K is subtracted from said new assigned number for the overlapping length FL reduced to the half R _K Performing a fourth step of finally obtaining a processor allocation number R _K by repeating the number of times of the code overlapping step FS by obtaining the allocation number R _K by adding the minimum code FB after taking a complement value.

In the fourth step, the overlap length FL is divided by 2 (FL = FL / 2) to obtain the updated overlap length FL (S6). When overlapping is performed once, .

Then, the allocation number R _K is compared with the minimum code FB plus the updated overlap length FL (R _K ≥ FB + FL) (S7).

If so, the minimum code FB is added to the updated overlap length FL (FB = FB + FL) to obtain a new minimum code FB (S8).

The new allocation number R _K is obtained by subtracting the new FB from the allocation number R _K (R _K = R _K -FB) (S 9).

When the new allocation number R _K is obtained, a complement value R _K = (FL-1) - R _K of the new allocation number R _K for the overlapping length FL reduced to half is obtained.

And, in addition to the updated allocated variable FB is assigned in this case the number of R _K is again seek the assigned number R _K (S11).

In this way to obtain again the allocation number _K R using the case of the FL and FB including repeatedly as many times as the code FS stacking step is to seek the finally assigned number R _K.

Thereafter, a fifth step of obtaining an allocation direction AD from a pattern to be allocated a code is performed (S12). The flow of obtaining the code allocation direction AD is as shown in Fig.

Code assignment direction AD is obtained when the code assignment direction AD the complement value of the allocation number R _K for the forward if the judgment (S13) the number of processors to decide (S14), if the reverse of the processor number to the assigned number R _K is the forward A sixth step of determining PN = (N _P -1) -R _K as a processor number (PN) (S 15) and outputting the processor number (PN) (S 16).

Meanwhile, FIG. 3 is a flowchart for obtaining the allocation direction AD. When the input code K is input as shown in FIG. 3, the assignment direction is determined by bit determination of the hexadecimal value whether the allocation direction is forward or backward S31). BP = 0X6996 This is a bit pattern for determining overlap, where bit is 0 for forward and 1 for reverse.

And obtains the quotient D _K = K / N _P to the code by dividing the code K to the number N _P processors. (S32) obtains the hexadecimal value of the quotient D _K. (S33), which is that the BP hex length Because. Subsequently, the initial value of the allocation direction AD is set to forward AD = 0 (S34).

여기서, D_i는 D_K의 16진수 자리값이고,

는 배타적 논리합(익스클루시브 오아)을 수행한다(S35).And,

Where D _i is the hexadecimal digit value of D _K ,

Performs an exclusive OR (exclusive OR) (S35).

The thus obtained allocation direction AD is output (S36), and in the sixth step of FIG. 2, it is determined whether the allocation direction is forward AD = 0 and the processor number PN is determined as the allocation number R _k or the complement value of R _K for the processor number And outputs it.

The steps (S4) to obtain the stacking step (FS) of the code by the number (N _p) of the processor in the second step, the value refers to happen a few overlapping for the entered code and, if 0 Basic overlap, 1 means overlap once inside, 2 means overlap twice inside.

The minimum code FB of the overlap and the length FL of the overlap are determined according to the number of overlaps. That is, the start position and length of the arrow determined by the code.

In the allocation direction determination unit of FIG. 3, the allocation pattern is set to a value of 0 when the forward direction is 0 and a value of 1 when the backward direction is set, and the allocation pattern is obtained as a hexadecimal number '0x6996'. After computing the quotient (D _K ) for the code, calculating the 16-digit value, determining the allocation direction from the allocation pattern for each digit, and then performing an exclusive OR operation to determine the overall direction.

이므로 순방향이다. 결국 할당 방향은 순방향이고 프로세서의 번호는 R_K그대로 5이다. 이것은 도 1에서 코드 79가 프로세서 5에 할당된 것과 같다.Depending on this direction of overlap, the code is numbered by the processor. For example, when the code is 79, the overlapping step (FS) is obtained as (79 / (8 * 4))% log2 (8) = 2 and the remainder (R _K ) Two repeated loops are executed. In the first step, the length FL of the overlap is reduced by half to 4 and R _K (= 7) is greater than FB (= 0) + FL (= 4) The code (FB) becomes 4. R _K is again 4 by calculation. In the second step, the length of the overlap (FL) is reduced again to 2 and R _K (= 4) is not greater than or equal to FB (= 4) + FL (= 2) , R _K is again 5 by calculation. Next, the process goes to the allocation direction determination unit of FIG. The share (D _K ) for code K (= 79) is K / N _P = 79/8 = 9. The hexadecimal value is 0x9. Since the hexadecimal digit length of the quotient is 1, it performs loop repetition once.

Therefore, it is forward direction. As a result, the allocation direction is forward and the number of processors is 5 as R _K. This is the same as code 79 in FIG. 1 is assigned to processor 5.

The algorithm of this code distribution scheme is used as a means of determining the number of the processor when the code is inserted. In addition, it is also used as a means for determining the processor number in which a newly generated block is to be stored when data is stored in block units in order to efficiently search. If the data is stored in block units as in this case, the block is divided into an excess of data for one block. At this time, the representative key for the block also has a length of one long and is divided. You need to decide which processor to store the newly generated keys and blocks, and you can use code distribution techniques. The code distribution method developed in the present invention is suitable for a parallel computer system in which the number of processors is a square of 2. Generally, it is reasonable to use this code distribution technique because the processors of a parallel computer system are made of squared form of 2.

When the code distribution scheme of the present invention is designed in hardware, the code can be assigned to the processor by the structure shown in FIG. (100) for determining and controlling the overlap length, the processor number and the allocation direction by analyzing each code (K), and a processor processor of the demultiplexer (inverse multiple splitters, demultiplexer) 200, and a plurality (N _P) for processing the code (K) provided is distributed from the demultiplexer 200 to output a K for that processor (0,1, 2, ..., P).

The de-multiplexer 200 is provided with a demultiplexer as much as the code K-length, respectively, of the demultiplexer is provided with the output of as much as the number of processors (N _P) to the first input input under the control of the overlap chip 100 And outputs the generated code to a predetermined processor.

Thus, a 1xN _P demultiplexer is required as much as the key length of the code, and the code is distributed according to the result of the processor number of the overlapping chip.

A block diagram of the internal structure of the stacked chip 100 is the same as that of FIG. 5, and the flow of the process is the same as that of FIG. A code overlapping step calculating unit for receiving a code unit K and a processor number N _P and receiving a stack unit FU from an erasable programmable read-only memory (EPROM) 110) and the initial R _K calculation unit 120, can be performed in EPROM (130) assigned to the direction determination unit 150 independently from each other for receiving the gyeopchip bit pattern (BP) from a, R _K calculated according to the repeats ( 140 are performed only after receiving the code overlapping step FS and R _K and the overlapping lowest code FB from the EPROM 130. The processor number determining unit 160 receives R _K from the iterative R _K calculator 140, And receives the allocation direction AD from the allocation direction determination unit 150. [ As a result, the output value of the processor number is output.

In the code distribution technique of the present invention, the algorithm is not changed according to the number of processors, so the chip design is not changed. However, since the check matrix is different according to the number of processors in the conventional Hamming filter, the chip must be redesigned when the number of processors is changed.

The code distribution technique developed in the present invention can be stored uniformly in terms of quantities of data when storing data in code form. In searching for the same code of a query to be searched when a search is performed, So that parallel processing can be performed effectively. In addition, since the code distribution technique does not change the algorithm or chip configuration even if the number of processors is changed, it is possible to distribute code by inputting only the number of processors.

Claims (6)

When the code type data is inputted, it is arbitrarily determined whether a new overlapping form appears after overlapping the overlapping unit (FU), initialized to a constant value, the overlap length FL is initialized to the number of processors N _P , To an initial value &quot; 0 &quot; A number of processors and an overlapping step FS indicating how many times overlapping occurs with respect to the code inputted by the overlapping unit FS = (K / (N _P * FU))% log ₂ (N _P ) And a second step of obtaining A third step of dividing the code value K by the number of processes (N _P ) and obtaining the remaining R _K ; Updating the lowest code FB while reducing the overlap length FL by half, subtracting the lowest code FB from the allocation number R _K to obtain a new allocation number R _K , and allocating the new allocation number R _K A fourth step of finally obtaining the processor allocation number R _K by repeating the number of times of the code overlapping step FS by obtaining the allocation number R _K by adding the lowest code FB after taking the complement value of the code number FB, (D _K ) is obtained for the code by calculating the allocation pattern as a hexadecimal value '0x6996' by setting the value to 0 in the forward direction and to 1 in the reverse direction to calculate the 16-digit value. A fifth step of determining an overall allocation direction AD by performing an exclusive OR operation after determining the direction, Code assignment direction AD is obtained when the code assignment direction AD is decided the processor number is a forward is determined whether the forward to the assignment number R _K, the reverse is complemented value of the allocation number R _K of the number of processors PN = (N _P - 1) -R _K as a processor number (PN) and outputting a processor number (PN) to which the currently input code K is assigned. The method as claimed in claim 1, A step (S6) of dividing the overlapping length (FL) by 2 (FL = FL / 2) to obtain an updated overlapping length (FL) (S7) comparing whether the allocation number R _K is equal to or larger than a value obtained by adding the minimum code FB and the updated overlap length FL (R _K ≥ FB + FL) (S8) of adding the minimum code FB to the updated overlap length FL (FB = FB + FL) to obtain a new minimum code FB, (S9) of subtracting a new FB from the allocation number R _K (R _K = R _K -FB) to obtain a new allocation number R _K , (S10) of obtaining a complementary value R _K = (FL-1) - R _K of the new allocation number R _K for the overlapping length FL reduced to half in the case that the new allocation number R _K is obtained, The step S11 of adding the R _K in the step S10 to the updated allocation variable FB to obtain the allocation number R _K again, Parallelism, characterized in that, using the allocation number R _K and a case such as the FL, and FB in the step S11 the number of times of the code overlap step FS to obtain a final allocation number R _K repeatedly S11 different from step S6 A code distribution method for. The method according to claim 1, wherein the fifth step of obtaining an allocation direction AD from a pattern to which the code is to be allocated, When the code K is inputted, a step S31 of determining whether the allocation direction is forward or backward by a hexadecimal value bit pattern BP = 0X6996, And the share of the code by dividing the code number N by K _P = _K D processor stage (S32) to obtain the K / N _P, A step (S33) of obtaining a hexadecimal value for the quotient D _K , A step (S34) of setting an initial value of the allocation direction AD to forward AD = 0,

Where D _i is the hexadecimal digit value of D _K ,

Is an exclusive OR (exclusive OR), a step (S35) of obtaining an AD value, And outputting the allocation direction AD obtained in the step (S36). 4. The method according to any one of claims 1 to 3, Wherein the code distribution method according to the overlapping method includes not only the query code but also the same code, and is distributed to the various processors in the overlapping manner according to the dispersion method. (100) for analyzing each input code (K) to determine and control the overlap length, the processor number to be assigned, and the allocation direction, and a processor and the demultiplexer 200 to output the input code K to the processor, such that the demultiplexer plurality of processing code (K) provided is distributed from the 200 processors in the (N _P), (0, 1, 2, ... P) 300, The de-multiplexer 200, the a de-processors as much as the number of processors (300) are provided, each of the demultiplexer is provided with the output of as much as the number of processors (N _P) to the first input the stacking chip 100 And outputting the code inputted by the control of the processor to the specified processor. The semiconductor device according to claim 5, wherein the stacked chip (100) And the code from the input (K) and a number of processors (N _P) for receiving the code overlap processing take gyeopchip unit (FU) from the EPROM (130) for storing the constant calculating unit 110, the initial R _K calculation section (120 An allocation direction determination unit 150 that receives an overlapped bit pattern BP from the EPROM 130 storing a constant and determines an allocation direction; Receiving a code stacking stage (FS) from the code stacking step calculating unit 110 receives the R _K value from the initial R _K calculation unit 120, the stacking lowest code (FB) from the EPROM (130) for storing the constant An R _K calculation unit 140 for calculating an R _K value by repetition; And a processor number determination unit 160 configured to receive R _K from the R _K calculator 140 and to receive the allocation direction AD from the allocation direction determination unit 150 and determine the processor number PN. (100). &Lt; / RTI &gt;

Images