TW201913404A - Method of executing tuple graphics program across the network - Google Patents

Abstract

A programming model provides a method for executing a program in a distributed architecture. One or more first shards of the distributed architecture execute one or more operations, and sending tuples to at least one second shard, the tuples being part of a stream and being based on the one or more operations. The one or more first shards send a token value to the at least one second shard when the sending of the tuples in the stream is complete. The at least one second shard determines whether a total of the token values matches a number of the one or more first shards, and takes a first action in response to determining that the total of the token values matches the number of the one or more first shards. The first action may include marking the stream as being complete and/or generating a message indicating that the stream is complete.

Description

Method for executing tuple graphic program across network

Cloud computing allows users to have various computing capabilities to use shared configurable resource pools to store and process data to achieve cost and computing efficiency. The current stylized models for cloud computing include MapReduce, Dryad, and Bulk Synchronous Parallel cloud processing. One of the problems facing distributed computing is performance. The performance in a distributed computing is related to the proximity of data to one of the computing units and the cost of data transfer between the computing units.

The present invention describes a new stylized model for cloud computing. The new programming model can be used to write decentralized low-latency non-batch programs. An application builds a program based on the model, and then submits the program for execution. The program consists of directed acyclic graphs, one of the operators. The value stream flows from one operator to another along the edges in the graph. Each value sent through a stream is a tuple. Different operators in the same program can run on different machines. The stylized model coordinates the execution of these operators on different machines and propagates data from one operator to another.

An aspect of the stylized model provides a method for executing a program in a decentralized architecture, the method comprising: performing one or more operations from one or more first partitions of the decentralized architecture; One or more first partitions send tuples to at least one second partition, the tuples are part of a stream and are based on the one or more operations; when the sending of the tuples in the stream Upon completion, a token value is sent from each of the one or more first partitions to the at least one second partition. The method further includes: determining from the second partition whether a total of the token values matches one of the one or more first partitions; and in response to determining that the total of the token values matches the one or more A first action is taken for this number of first partitions. The first action may include marking the stream as completed and / or generating a message indicating that the stream is completed.

The at least one second partition is a receiving partition of one of the one or more first partitions. The method may further include: generating, from the one of the one or more first partitions, a list of the receiving partitions with which the one or more first partitions communicate; and by the one or more first partitions The one of the partitions transfers the list to a controller. In addition, the controller can track all the receiving partitions that have started processing; determine whether one or more of the started processing in these receiving partitions does not exist in the list; and for the started processing and does not exist in Each receiving partition in the list sends a token value representing the one of the one or more first partitions to the receiving partition. In some examples, the method may further include: determining by a controller whether any partitions have not started processing; determining by the controller whether the partitions that have not started processing are intentionally skipped by the design of the program; and by The controller sends a token value to the second partition on behalf of any partition that has been deliberately skipped.

Another aspect of the present invention provides a system including: one or more first partitions in a distributed computing environment; and at least one second partition in the distributed computing environment, the at least one second partition Away from the one or more first partitions. The one or more first partitions are configured to: perform one or more operations; send tuples to at least one second partition, the tuples are part of a stream and are based on the one or more operations; And when the sending of the tuples in the stream is completed, sending a token value to the at least one second partition. The at least one second partition is configured to: determine whether a total of the symbol values matches one of the one or more first partitions; and in response to determining that the total of the symbol values matches the one or A first action is taken for the number of multiple first partitions.

The system may further include a client device in communication with at least one of the one or more first partitions, the at least one second partition, or the controller. The client device can be configured to: construct a graph, where each node of the graph represents a partition; and based on the graph, verify whether the program will be accurately executed across the distributed architecture. The client device can be further configured to dynamically construct the start of the graphic when the program is executed.

In some instances, a dynamic transmission operation is performed on a computing device in a decentralized architecture. The dynamic sending operation sends a data input stream to all activations of a destination graphic; and receives new tuples from the controller, which are received when an additional activation of the destination graphic is detected .

I. Overview

A new programming model can be used to write decentralized low-latency non-batch programs. An application builds a program based on the new model, and then submits the program for execution. The program consists of directed acyclic graphs, one of the operators. The value stream flows along the edges in the graph. Each value sent through a stream is a tuple. Different operators in the same program can run on different machines. The stylized model coordinates the execution of these operators on different machines and propagates data from one operator to another.

The construction program includes operations that define the nodes that form the graph. The operation receives the value stream as input and sends the value stream as output. Each stream has a tuple type, and all tuples flowing through the stream must match that type. The tuple type is defined by a field containing a name identifier and a field type identifier. When defining operations, type inference is used to provide a standardized way for operations to interact with each other. For example, as part of its definition, an operation can refer to its inputs and outputs and set various constraints on those inputs and outputs. An example of this constraint is that an output type can be constrained to contain each field of input.

Operations in graphics can be performed at various locations in a decentralized architecture. Although certain operator positions may be defined in a stylized stage, other operator positions may not be defined, and operators may be automatically assigned to other positions during graph creation and segmentation. In this regard, locations are automatically assigned in a way that reduces overall network traffic.

Create graphics based on the operations defined in the programming stage. The division of graphics can be performed in two stages (including a main stage and a local stage). Perform each stage according to a set of constraints. The first set of constraints for the main segmentation may be different from the second set of constraints for the local segmentation.

In the main stage, a first step merges subgraphs according to the first set of constraints, thereby minimizing the total number of one of the subgraphs in the program. Then, by incorporating adjacent unassigned nodes into some subgraphs, the subgraphs are enlarged. Candidate operations are first checked to determine whether they have been marked as splittable, meaning that they can be split into separate operations without changing the functionality of the operation. If not, the candidate operations are not merged into adjacent subgraphs. If these candidate operations are splittable, placing their candidates into adjacent subgraphs is subject to constraints. The location is assigned to all unassigned operations by copying the location from the assigned node to its neighbors. The possible unpartitioned subgraph pairs running at the same location are merged to minimize the total number of subgraphs. At some point, further mergers will be impossible.

In the local segmentation stage, identify subgraphs that need to be split (for example, to prevent inefficiencies in execution). These sub-graphs may only be sub-graphs that contain blocking operations. These blocking operations may follow a thread while performing I / O, thereby preventing other operations from being able to run. Prepare the graphic for splitting. This may include modifying the subgraph to impose local segmentation constraints. Construct a merged graph, where each operation ends in one of its own subgraphs. Then repeatedly merge these subgraphs together. Specifically, all operations with external incoming edges are merged together into the same subgraph. In addition, all possible sub-picture pairs with non-blocking operations are merged.

If an operator is implemented for a partitioned service, the new stylized model automatically partitions the calculation by instantiating the subgraph multiple times. Partitioning provides a latency benefit (this is due to the parallel execution of partitions) and a data efficiency benefit. As an example of data efficiency benefits, an operator placed after a partitioned operator can usually run on the same partitioned instance to filter the final output and reduce the final output, thus minimizing network traffic.

Once divided, the graphics can be executed. Each of the subgraphs is executed at a separate position in the subgraph, and each subgraph is executed in a separate single thread. Data transfer along the edges within a subgraph is optimized based on its execution in a single thread environment.

Various aspects of the stylized model allow efficient execution of the program. Such aspects include (by way of example and not limitation) pipelining and the partitioning described above. Pipelining provides extremely low latency. For example, for a calculation consisting of a series of 5 operations that each spend 10 ms but involve hundreds of thousands of independent values, processing the operations one by one will take 50 ms. However, a proper pipelined solution can be completed in as little as 10 ms. To achieve this, tuples are streamed between operators during execution, which results in a better pipeline across the entire program. This tuple streaming format provides efficient serialization / deserialization across network periods. To enable the pipeline to start early but achieve higher throughput, the new stylized model uses dynamic buffer expansion. For example, small messages are sent early in the calculation but expanded later, because larger messages are more efficient.

The new stylized model also (for example) provides low buffering by implementing flow control between network nodes. For example, the sending node determines whether a recipient is busy, and if so, blocks transmission. Within a sub-graph, the new stylized model can efficiently deliver data between operations via local procedure calls. The new stylized model efficiently determines when a calculation is complete, and provides lower latency by determining completion faster. II. Example system

FIG. 1 illustrates an example system including a decentralized computing environment. The plurality of data centers 160, 170, 180 may be (for example) communicatively coupled via a network 150. The data center 160, 170, 180 may further communicate with one or more client devices (such as the client 110) via the network 150. Therefore, for example, the user terminal 110 can perform operations in the "cloud". In some examples, the data center 160, 170, 180 may further communicate with a controller 190.

The client 110 can execute one or more applications for creating programs using the new programming model. Each client 110 may be a personal computer intended for use by a person, which has all the internal components normally present in a personal computer, such as a central processing unit (CPU), CD-ROM, hard drive, and a display device ( (For example, a monitor with a screen, a projector, a touch screen, a small LCD screen, a TV, or another device such as an electric device operable to display information processed by the processor 120) , Speakers, a modem and / or network interface device, user input (such as a mouse, keyboard, touch screen, or microphone), and all these components are used to connect these components to each other. In addition, computers according to the systems and methods described herein may include devices capable of processing instructions and transferring data to and from humans and other computers, including general-purpose computers, PDAs, tablet computers, mobile phones, and smart Watches, network computers that lack local storage capacity, set-top boxes for TVs, and other network devices.

The client 110 may include a processor 120, memory 130, and other components commonly found in general-purpose computers. The memory 130 may store information accessible by the processor 120, including instructions 132 executable by the processor 120. The memory may also include data 134 that can be retrieved, manipulated, or stored by the processor 120. The memory 130 may be a type of non-transitory computer-readable medium capable of storing information accessible by the processor 120, such as a hard drive, solid state drive, tape drive, optical storage device, memory card, ROM , RAM, DVD, CD-ROM, memory capable of writing and read-only memory. The processor 120 may be a well-known processor or other less known types of processors. Alternatively, the processor 120 may be a dedicated controller, such as an ASIC.

The instructions 132 may be a set of instructions directly executed by the processor 120 (such as machine code) or indirectly executed (such as a descriptive language). In this regard, the terms "command", "step" and "program" may be used interchangeably in this article. The instructions 132 may be stored in a target code format (for direct processing by the processor 120) or other types of computer languages (including a collection of instruction codes or independent source code modules that are interpreted or compiled in advance as required).

The data 134 can be retrieved, stored or modified by the processor 120 according to the instruction 132. For example, although the system and method are not limited by a specific data structure, the data 134 can be stored in a computer register or a related database as a table or XML document with multiple different fields and records. The data 134 may also be formatted in a computer-readable format (such as but not limited to binary values, ASCII, or Unicode). In addition, the data 134 may contain sufficient information to identify relevant information (such as numbers, descriptive text, proprietary code, indicators, references to data stored in other memory (including other network locations), or used by a function to calculate relevant Information).

The application program 136 can be used to construct a program according to the new programming mode. For example, the application 136 can be downloaded, executed from the command 132, or accessed remotely. In some instances, applications can be executed remotely. For example, the client 110 may compile a program and send the program to the cloud for execution. The application program 136 can perform different functions, such as type inference, graph creation, graph segmentation, and so on. For example, one application can perform many different functions, or each application can perform one or more different functions.

For the type inference function, the application can be configured to receive information that defines the properties of an operation by the field name and type distinguisher. The application can further receive information that defines an action of the operation regarding these attributes. Determine constraints on operations based on attributes and behavior. Information defining an input of an operation can also be received and used in conjunction with these constraints to determine a type of output of an operation. The determined output type can be associated with the output of the operation.

For graph creation, multiple nodes can be generated, where each node corresponds to an operation of the program. These nodes are connected by edges or vertices, and these edges or vertices represent the stream sent between these nodes. For example, the location can be assigned to a specific node either automatically or manually by the programmer based on the program requirements and capabilities of the computing device.

For graphics segmentation, graphics are optimized to reduce overall network traffic. For example, where possible, locations for performing one or more operations are automatically assigned together. In this way, nodes are merged and split according to a certain number of predefined constraints. The segmentation can be further performed at a local level, for example, so that the operation is performed at a partitioned location. This local segmentation can be performed according to a set of second individual constraints. When compiling the program, both the main split and the local split can be performed. As a result of the segmentation, the program is ready to be executed by computing devices in one or more data centers 160, 170, 180, and can be sent for execution.

Although FIG. 1 functionally illustrates the processor 120 and the memory 130 as being in the same block, the processor 120 and the memory 130 may actually include multiple processors and may or may not be stored in the same physical housing. Memory. For example, some of the commands 132 and data 134 can be stored on a removable CD-ROM and other commands and data can be stored in a read-only computer chip. Some or all of the instructions and data may be stored in a location physically remote from the processor 120 but still accessible by the processor 120. Similarly, the processor 120 may actually include a set of processors that may or may not operate in parallel.

The data centers 160 to 180 can be located at a considerable distance from each other. For example, data centers can be located in various countries around the world. Each data center 160, 170, 180 may include one or more computing devices, such as processors, servers, partitions, or the like. For example, as shown in FIG. 1, data center 160 includes computing devices 162, 164, data center 170 includes computing device 172, and data center 180 includes computing devices 181-186. For example, programs can be executed across these computing devices so that certain operations are performed by one or more computing devices in a first data center and other operations are performed by one or more computing devices in a second data center. In some examples, computing devices in various data centers may have different capabilities. For example, different computing devices may have different processing speeds, workloads, and so on. Although only a few of these computing devices are shown, it should be understood that each data center 160, 170, 180 may include any number of computing devices, and the number of computing devices in a first data center may be different One of the number of computing devices in the second data center. In addition, it should be understood that the number of computing devices in each data center 160 to 180 may change over time (for example, when hardware is removed, replaced, upgraded, or expanded).

In some examples, each data center 160 to 180 may also contain a certain number of storage devices (not shown), such as hard drives, random access memory, disks, disk arrays, tape drives, or any other type Of storage devices. The data centers 160, 170, and 180 can implement any of a certain number of architectures and technologies including but not limited to the following: direct attached storage (DAS), network attached storage (NAS), storage area network (SAN) ), Fibre Channel (FC), Ethernet Fibre Channel (FCoE), hybrid architecture network, or the like. In addition to storage devices, the data center may also contain a certain number of other devices, such as cabling and routers. In addition, in some examples, the data centers 160 to 180 may be virtualized environments. Furthermore, although only a few data centers 160 to 180 are shown, many data centers can be coupled via the network 150 and / or additional networks.

In some examples, the controller 190 can communicate with the computing devices in the data centers 160 to 180 and can facilitate the execution of programs. For example, the controller 190 can track the capabilities, status, workload, or other information of each computing device and use this information to assign tasks. The controller 190 can also help determine whether the streaming through the network has been completed. For example, in some cases, the controller 190 may send tokens representing partitioned operators, which tokens are used by downstream nodes to determine that the streaming is complete. The controller 190 may include a processor 198 and memory 192 (which includes data 194 and instructions 196), similar to the client 110 described above.

The user terminal 110, the data centers 160 to 180 and the controller 190 may be capable of direct and indirect communication (such as via the network 150). For example, using an Internet socket, a client 110 can be connected to a service operating on a remote server through an Internet protocol suite. The server can create listening sockets, which can accept an initial connection for sending and receiving information. The network 150 and intervening nodes may include various configurations and protocols, including the Internet, World Wide Web, Intranet, virtual private network, wide area network, regional network, and one or more company-specific communication protocols Private networks, Ethernet, WiFi (for example, 702.71, 702.71b, g, n, or other such standards), HTTP, and various combinations of the foregoing. This communication can be facilitated by a device capable of transmitting data to and from other computers, such as a modem (eg, dial-up, cable, or optical fiber) and a wireless interface. III. Construct a program

2A to 2B illustrate an example of creating a program using a stylized model. In this program, a target extracts all the images in an album and generates a thumbnail for each image. In the code, this goal can be expressed as:

However, if the album data is stored on a different server, the image (album_name) call requires a remote access. A search call should be sent in parallel with a partitioned service that returns image data giving an image name. Thumbnail calls should be sent in parallel with a separate group of thumbnail servers. According to one of the stylized model construction and execution procedures described in this article, this distributed execution is achieved as follows:

This program is constructed to produce the graph of Figure 2A. As shown, the input operation 210 generates a stream 215 that feeds an album name to the ListImages operation 220. ListImages 220 has associated metadata, which tells the stylized model that it must run on a different server (perhaps based on a specific partition of the album name). The stylized model internally creates a remote procedure call (RPC) for the appropriate partition for this service and sends the album name to the appropriate partition. ListImages 220 generates a stream 225 of one of the image names. The stylized model again finds the appropriate partition for the search service for each image name and sends the name to these partitions. The search operator 230 generates a stream 235 of images. These images are then passed to another service-thumbnail operation 240 that generates thumbnails. The stylized model finds the appropriate partition for the thumbnail service for each image, and sends each image to the appropriate partition. The generated thumbnail 245 is saved as the output 250 of the program. Although the calculation touches three different servers in the partitioned service, the application code does not have to initiate or manage any remote communications.

FIG. 2B shows an updated graphic in which the program is fine-tuned to add a cache memory of one of the thumbnails with the image name and index key. Therefore, the lookup operation 226 in the cache memory is shown. The image thumbnails hit in the cache memory are directly transmitted to the output via the stream 228. As before, but through the stream 227, the missed image name is sent to the search 230. The application is not aware of the location or implementation of one of the cache memories. As long as the cache memory implements a lookup operator based on the stylized model (which knows the location of the cache memory), the program of FIG. 2B will meet the requirements.

According to some examples, the stylized model may provide a set of built-in operators. Examples of these built-in operators are provided in the chart of Figure 3. Each operator is described in conjunction with a corresponding function. Although only a few operators are shown, it should be understood that any number of operators can be built into the stylized model. In addition, other operators can be established. The program can be constructed by (for example) writing the operator along with the stream according to the following code:

This program combines two constants into a single stream. The operator is added to the graph by calling the operator specific constructor (constant and interlace). Each operator has this constructor that returns one or more streams. These streams can be used as arguments for later operator constructors.

When creating a program, the operator can be defined by a field containing a name and a type distinguisher. The name can be chosen by the programmer. Choosing a unique name provides the most helpful output, but this is not required.

Once the operators are connected together, the program can be constructed by constructing a program object and compiling the program object. In addition, a compiler object is constructed, and the built-in operator library is added to the compiler object. A Binding object can be constructed to supply parameters and results to the program at a later stage.

It is possible to provide input to the program and receive output from the program by adding results and parameters. An input stream and a name of the result can be used to add a result, and then associate the name of the result with an actual instance. This separation allows a program to be reused with different outputs. You can add arguments by a similar procedure, use a name to add arguments, and associate the name with an instance, allowing the program to be used with different input data. A. Type inference

Each value sent through a stream is a tuple. Each stream has a tuple type and all tuples flowing in that stream must match that type. The one-tuple type is defined by a set of fixed fields with the form <name: field type>. One example of one-tuple type is the structure <city string, population int64>. This tuple has two fields: a city field holding a string, and a population field holding a number (64-bit integer). Some values that match this type are {city: 'Tokyo', population: 13350000} and {city: 'Ithaca', population: 30515}. Examples of field types that can be supported in a tuple include but are not limited to: Boolean value (bool), byte, string (must be a valid UTF-8 byte), double precision floating point (double) , Single precision floating point (float), int32, uint32, int64 and uint64.

To allow operations to be implemented in a way that is applicable across various platforms and programming languages, an inference algorithm allows input to the operation to change or parameterize field names, field types, or structural shapes. To allow decentralized graphics execution, the rules of type inference rules and constraints are separated from the actual implementation of operators. In this way, the type flow is expressed through operation without having to decide on a specific implementation.

As part of its definition, an operation can refer to its inputs and outputs and set various constraints on those inputs and outputs. An output can be constrained based on the input. For example, an output type can be constrained to include each field of input. In addition to each input field, the output type can also be constrained to include one or more additional fields. As another example, the output type may only be constrained to contain one or more specific fields.

An example operation system is as follows:

In this example, the operation is parameterized by attributes defined by the field names key_field, fp_field, and val_field. When creating an operation, a programmer can specify the names of these fields, and configure the behavior of the operation with reference to these field names. He acts to determine type constraints. For example, the type constraint can specify that the input to one of the operations should contain fields <key: bytes> (name: index key, type: byte) and <fp: uint64>, and the output value should contain fields val: bytes ＞.

Example operations may also specify other properties, such as the number of input streams, the number of output streams, how partitioning should be implemented, and so on. For example, the operation in the above example also specifies that fp_field is used for partitioning purposes. By way of example only, operations can be extended across 100 replicas, and if distributed evenly, each replica will receive 1% of the input. The fp_field is consulted to determine which partition should receive input data through modular arithmetic.

This operation defines that it receives a single input stream called In and establishes two output streams called Hits and Misses. Misses is defined as having the same type as the input, and Hits is constrained to be a new type consisting of the input type concatenated with one of <val_field bytes>. Operations can have other properties that are not used for type inference purposes but are important for graphical execution purposes. Examples of these other properties in the above example operations include end_when_outputs_done and skip_on_empty_inputs.

Determine the type of all operations at compile time and check their correctness. For example, at compile time it is determined whether the output of one operation matches the input of another operation. The system performs type inference to convert type constraints to specific types. This can be implemented as a forward pass.

The operator constructor mentioned above returns a stream of objects. For example: Stream s = ZipConst (input, {"label", 100});

The operator constructor is used to add type-related information to a statement associated with the stream returned by the operator constructor. After the above example, ZipConst can add type related information, such as + <label: int64>. In this example note, "+" indicates that all fields in the input type should be added to the output type. "<Label: int64>" indicates that a 64 integer field called "label" should also be added to the output type. "<Label: int64>" can be called a type specifier, which can specify a series of field names and field type pairs more generally. The type inference code of the stylized model interprets these annotations and produces an output type. In some instances, such as if a programmer attempts to define an output that is inconsistent with the constraints, the inference may produce an error. For example, if the input type already contains a field called "tag", the type inference will fail because each field name can appear once in a valid type. When this error occurs, the attempted output definition can be rejected, and the programmer can be prompted to enter a different definition consistent with the constraints. In other instances, the attempted output definition that caused the error can be automatically flagged by the stylized model for further re-examination by the programmer.

Figure 4 provides a chart listing examples of output type annotations for existing operators. It should be understood that this diagram is not exhaustive, and other example output type annotations can also be used in the stylized model.

Mark some operators (such as Receive and Interleave) as "special". Type inference provides special treatment for these operators. For example, for receiving, the output type is the same as the input type for one of the sending nodes associated with the comment. For interleaving, all input types are the same, and the output type is the same as the input type. Although this can suppress writing operators that perform extremely complex types of processing, these operators are beneficial because they provide greater consistency among the operators. In addition, if the type inference code does not need to run any operator-specific code, the type inference code can be run in a location where the operator implementation is not available. For example, in a decentralized setting, type inference can be performed at the controller without having to link all operators to the controller. Type inference can be performed as a forward pass.

The stylized model can further provide type checking. The operator constructor can add a comment to a statement, where the comment is used to type check input to the operator. For example, the operator sum (Sum) requires an input to contain a numeric field, and this puts the input type annotation on its statement: <n: int64>. The stylized model will verify that any input fed to this operator contains a superset of the specified fields.

FIG. 5 provides a flowchart illustrating an example method 500 of type inference. For example, the method may be performed by a client computing device, controller, or other network computing device. Although the methods are explained below in a specific order, it should be understood that the sub-parts can be performed in a different order or simultaneously. In addition, subsections can be added or removed.

In block 510, information defining attributes by field names and field type identifiers is received. For example, referring to the example operations generated in the above code, define the attributes {key_field, bytes}, {val_field, bytes} and {fp_field, bytes}. This information is used to define the types of input and output streams for operations.

In block 520, information is received regarding these attributes to define the behavior of an operation. For example, referring to the above example, the input, output, and partition fields determine how the operation will behave.

In block 530, constraints on the operation are determined based on attributes and behavior. In some instances, these constraints can be automatically determined by stylized models. In other examples, a user may define these constraints.

In block 540, information input defining one of the operations is received. The input may include, for example, a field that includes a name and a type. This information can also be referred to as type information and is provided for one or more input streams of the operation. The type inference method determines the type information of one or more output streams of the operator based on the type information associated with the one or more input streams and an output comment associated with the operator. Type information may include constraints that restrict tuples contained in the stream to which the type information is associated. The type should correspond to one of the types defined in the attribute.

In block 550, an output type is determined based on the constraints and the defined input. For example, the output type can be limited to one of the types specified in the constraints, and can correspond to the received information that defines the input. This determination can be performed in a forward pass through one of the graphs without backtracking.

In block 560, the output type is associated with one of the output operations. For example, when the user is defining one of the output operations, the output type field can be automatically filled. In other instances, the user may be prevented from attempting to input a different output type.

Although the foregoing example illustrates the determination of an output type based on an input type and defined operators, type inference can also be used in the reverse direction. For example, the output type can be received as input, and the input type can be determined based on the defined output type and other information.

Type inference and constraint verification as explained above ensure accurate and fast query execution. The operation can also be implemented in a very general way. Inference allows input (such as attributes) to the operation to change or parameterize the field name, field type, or even the shape of the structure. To allow decentralized graphics execution, the rules of type inference rules and constraints are separated from the actual implementation of operators. The result is a format that is completely abstracted from any particular implementation, while expressing the type flow through operations. Type inference and constraint verification are part of a critical path for query execution, resulting in a requirement for rapid execution. A single pass inference and verification algorithm without backtracking further provides fast execution. B. Location assignment

Location assignment occurs during program construction. The operation in a graphic may have a position limitation indicating one or more positions where the operation can be performed. The location limit may be defined by a user, or may be determined based on the capabilities of computing devices in a distributed system. For example, if the data to be retrieved by a search operation is stored in a specific data center, the search operation is restricted to be executed at the specific data center.

For operations that do not have a position restriction, the stylized model assigns a position to the operation. These positions can be selected to optimize the calculation in a certain way. For example, a node can generate a large amount of data, but then the node is followed by a filter node, which filters out 99% of the data. In this case, it is particularly advantageous to locate the filter node at the same position of the data generation node. The location assignment can occur as part of the graph creation and segmentation discussed further below. i. Graphic construction

6A to 6C illustrate an example of position assignment during program construction. 6A provides an example graph with nodes representing operations and edges connecting a source operation and a destination operation between the nodes, where the output of a source operation is the input of the destination operation . In this program, an argument 610 (such as an index key) is sent to a remote location where a search 620 is performed. The search result is sent through a filter 630, which removes certain results and outputs the results to a result operation 640.

FIG. 6B illustrates an example of user-assigned locations for each of the operations. Argument 610 is sent from position C to position L for lookup 620. The result of the search 620 is then sent back to position C for filtering and output. This causes a large amount of data to be sent from position L to position C, so that only a lot of the other data is filtered by the filtering operation 630. FIG. 6C illustrates one of the more efficient examples of possible user-assigned positions, where both search and filtering are performed at the same position L. This assignment optimizes execution time and reduces network traffic. However, relying on users to anticipate potential inefficiencies in location assignments and adapting to these inefficiencies places a significant burden on users.

7A to 7B illustrate an example of automatic position assignment during graph creation. When constructing the program, the operation parameter 710, the search 720, and the result 740 occur with the pre-assigned position. These positions can be automatically pre-assigned by the stylized model (for example) based on the capabilities of the computing device used to perform the operation, the limitations associated with the operation definition, or any other information. The filter operation 730 does not occur with any position assignment. Therefore, when the graphic is created, it may appear as shown in FIG. 7A. When the program is submitted for execution, the stylized model will recognize that the filter 730 is a data reduction operation and assign the filter 730 to position L. Therefore, the program will appear as shown in FIG. 7B.

Since the location assignment is automated, the division of the program into a graph that retains location constraints should also be automated. This segmentation is done in one way to maximize performance.

8A to 8C illustrate an example of dividing a graph to minimize the number of one of the sub-graphs while following the constraints. For example, graphics must remain acyclic. As shown in FIG. 8A, each operation runs in a single graph G0, G1, G2, G3. As such, data needs to be serialized and deserialized on each edge 815, 825, 835. For edges 815 and 835 across two locations C and L, network transmission requires this data serialization. However, the edge 825 does not require serialization of data. However, minimizing the number of graph divisions will produce the graph of FIG. 8B, which introduces a cycle between graphs G0 to G1 and G1 to G0. This can lead to deadlocks and therefore should be avoided. Minimizing the number of graphics while preventing loops will produce the graphics of Figure 8C, which is the best segmentation for the program.

Additional considerations for graphics segmentation are presented by the partition location. 9A-9D provide an example of graphical segmentation for partitioned locations. The partitioned location may include any location with multiple computing partitions for performing operations. As shown in FIG. 9A, the two search operations 920, 940 are placed in the same position and are partitioned by the field "index key". Assigning processing operation 930 to the same partitioned location results in the graph of FIG. 9B, which may be segmented as shown in FIG. 9C. However, this segmentation is incorrect. If the processing operation 930 modifies the index key field in any way, the output of the processing operation should not be passed directly to the second search operation 940 without repartitioning the input. To prevent this, all partitioned locations are considered unique. For example, even if both lookup operations 920, 940 are assigned to the same location, the lookup operations are treated as different locations. Therefore, as shown in FIG. 9D, the processing operation 930 is assigned to one of the unique search locations.

As demonstrated by the above example of partitioned locations, assigning the same location to two operations does not guarantee that the two operations will run together in the same graph. Ensure that two operations assigned to the same partitioned location do not run in the same graph. Although this behavior does not affect the correctness of the program, it can affect its performance. Therefore, the stylized model provides a way for users to co-locate a set of operations together at a specific location. Then ensure that their operations end in the same graph at that location, where other operations may be added to the graph. Whenever operations are designated as co-located, the data sent between these operations will not be repartitioned.

10A to 10B provide an example of common positions. In the program of FIG. 10A, the user has specified that the search operations 1020, 1030 will run together at a given location. After the location assignment and graphic creation, the program will be divided as shown in FIG. 10B.

Summarizing the automatic position assignment and graph segmentation described above, position is a property of an operation rather than a graph. Some operations will occur with assigned positions or position constraints, some operations will have their positions assigned by the user, and some operations will not occur with any assigned positions. The user writes a program with a single operation pattern without worrying about sub-pictures. Each operation provides hints to the stylized model. For example, for each output edge, the percentage of total input data that the operation report will flow on that output edge. This hint helps the stylized model to determine where to assign the operator. In other examples, this information can be calculated automatically for a given program during earlier operations. The stylized model automatically assigns positions to operations, and automatically divides the program into a minimum number of graphics while preserving the nature of no cycles between graphics.

According to an example, the output prompt specified in the operation or the data collected from the previous graphics run can be used to increase the other number of tuples by using the expected number of tuples that will flow on each edge of the program graphics. These locations can thus be assigned to the graph node in a way that minimizes the total number of tuples flowing between the locations. This position assignment can be performed by classifying all edges in the program in descending order of tuple counts and iterating the edges in sorted order. For each edge, the source operator and the destination operator are identified and if neither the source operator nor the destination operator has a position assigned to it, by using the source operator and the destination operator Grouped together and assigned the same position. If one operator has an assigned position, the same position is assigned to the other operator and all other operators that may have been grouped with the operator. This algorithm removes the most expensive edge from the total tuple count, then removes the second most expensive edge, and so on.

11A to 11C illustrate an example of a program with multiple input operations. As shown in FIG. 11A, the program includes a first input argument 1110 input to a first lookup operation 1130 via the edge 1115. The program also includes providing input to a second input argument 1120 of a second search operation 1140 via the edge 1125. Lookup operations 1130, 1140 provide streaming to ZipAny operation 1150 via edges 1135, 1145, respectively, and ZipAny operation 1150 provides streaming to selection operation 1160 via edge 1155. An output is provided to the result 1170 via the edge 1165. The edge weight represents the estimated number of tuples flowing along the edge. For example, the edges 1115, 1135, and 1155 have an edge weight of 1M. Edges 1125 and 1145 have an edge weight of 1, and edge 1165 has a weight of 3. Location SL is partitioned, while location L is not partitioned.

The automatic segmentation of this program can produce the graph of FIG. 11B, where both ZipAny 1150 and selection 1160 are assigned to position SL. This location assignment will work, provided that the second lookup operation 1140 running at location L broadcasts its tuple to all partitioned instances of ZipAny 1150.

As shown in FIG. 11C, the ZipAny operation is replaced with an interleaving operation 1180. The program will work, provided that the second lookup operation 1140 running at position L sends its tuple to only one of the interleaved partitioned instances. Although this solution is operationally specific, it can also solve the problem more generally. For example, all multiple input operations can be marked as indivisible. If an operation can be split into separate operations without changing the functionality of the operation, the operation can be splittable. For example, if there are three streams S1, S2, S3 fed to an operation OP, then if OP (UNION (S1, S2, S3)) == UNION (OP (S1), OP (S2), OP (S3)), then the OP system can be split. An example of a splittable operation is the double-precision floating-point number of the operator mentioned in FIG. 4 that doubles each input value. However, this can cause performance degradation. Another example of a general solution requires the program writer to specify how to partition multiple input operations. However, this will place a burden on the program writer and will eliminate the possibility of dynamically optimizing the program. Yet another example general solution provides a way for an operation writer to customize the split of multiple input operations. In the above example, ZipAny will always want its other inputs to be broadcast, while interleaving will always want its other inputs to be sent to only one location. Although this imposes an additional burden on the operating writer, it is insignificant compared to the potential burden on the program writer and retains the correctness of the program with optimized performance. ii. Graphics segmentation

In addition to position assignment, the new programming model also performs automatic segmentation of graphics in one way that is optimized to reduce overall network traffic and increase the speed and performance of executing programs. The operations in the graph are divided into multiple subgraphs. Each operation in a sub-picture must be assigned to the same location. When performing segmentation, the possibility of establishing a loop among the candidate subgraphs is detected, and the loop is eliminated by further segmenting one of the candidate subgraphs.

Graphic segmentation begins by placing all operations with assigned positions into one of its sub-graphs. For example, the location may be assigned based on the location constraints associated with the operation. For example, certain operations may have specific requirements, where only certain computing devices at specific locations in a decentralized architecture can perform operations according to their limitations. These limitations and capabilities can be identified by the stylized model, which can therefore automatically assign a position to the operation. Segmentation may include reducing the number of subgraphs that have operations assigned to a particular location.

In the process of the segmentation algorithm, the unassigned operations are placed into the sub-graphs with location assignments, and the sub-graphs are merged together as much as possible. When executing the algorithm, a certain number of main constraints are imposed, where these constraints ensure the final graph and the assignment of positions to the operations so that when executing the program represented by the graph, the communication among the operations in the graph is efficient. In particular, all operations must be placed in a subgraph, the algorithm must assign a position to each subgraph, an operation with a position assigned by a programmer must maintain that assignment, and if an The operation has a splittable nature, and the unassigned operation can only be placed into a partitioned location. In addition, if a location is partitioned, all edges in the program whose destination operation is assigned to that location remain unchanged throughout the algorithm. In addition, the graphics must be acyclic.

Graphic segmentation can be performed in two stages. In a first stage, the main segmentation is performed, and in a second stage, the local segmentation is performed. One purpose of the first stage is to determine the assignment of a node to a location in the graph so that when the program represented by the graph is executed, the communication among the locations is minimized. One purpose of local segmentation is to improve and optimize a program and the implementation of operations assigned to the same location. Primary partitioning may include merging partitioned subpictures, expanding partitioned subpictures, assigning unpartitioned locations, and merging unpartitioned subpictures. If all nodes are assigned to the same partitioned location, a subgraph is partitioned. As a result of the main segmentation, each subgraph is assigned a partitioned or unpartitioned position and all nodes in the same subgraph have the same position as the subgraph. Local segmentation can include identifying subgraphs that need to be split, preparing the graph for splitting, constructing merged graphs, merging subgraphs using external incoming edges, and merging subgraphs using unblocking operations.

In the first stage of the main segmentation, a first step merges as many partitioned subgraphs as possible. Therefore, the total number of one of the subgraphs in the program is minimized. The next step is to expand the partitioned subgraphs by folding adjacent unassigned nodes into the partitioned subgraphs. Candidate operations are first checked to determine whether they have been marked as splittable. If not, the candidate operation is not folded. If these candidate operations are splittable, placing their candidates into adjacent subgraphs is restricted by the main partitioning constraints. In one of the next steps of assigning unpartitioned locations, the location is assigned to all unassigned operations by copying the location from the assigned node to its neighbors. A next step involves merging unpartitioned subgraphs, and trying to minimize the total number of subgraphs by merging all possible pairs of unpartitioned subgraphs running at the same location. At some point, further mergers will be impossible. For example, when each operation is assigned to a sub-graph and the number of sub-graphs is minimized, any further merging will introduce one of a cycle or break the constraint into the graph. At this point, the blockade operation can be split into one of its sub-graphs, thereby creating a local graph executed on the same machine. Blocking operations may require input / output operations, and therefore I / O can be performed while retaining a thread to prevent other operations from running.

In the second stage of local segmentation, a position has been assigned. In addition, the partitioned locations can be split just like the unpartitioned locations. For example, if a location includes multiple partitions, the location can be partitioned. However, splitting into multiple local terminal diagrams must satisfy a set of local constraints. This requires that each lock operation must end in one of its own subgraphs. Splitting can use external (non-local) input to generate only one subgraph. And the edges between subgraphs and subgraphs must be acyclic. The split is required to generate only one sub-picture using external input to ensure that the external graphics communicate with a single local graphics, which achieves more transmission / reception optimization and simplifies the agreement.

One of the first steps of local segmentation is to identify the subgraphs that need to be split. These sub-pictures may only be sub-pictures containing blocking operations. In the next step, prepare the graph for splitting. This may include modifying the subgraph to impose local segmentation constraints. For example, the modification may insert a no-op before and after each lock operation. Insert an empty operation before the block operation to ensure that there is no block operation with external input in the subroutine. Insert a blank operation after the block operation to ensure that there is no block operation with external output in the subroutine.

In one of the next steps of local segmentation, a merged graph is constructed, where each operation ends in one of its own subgraphs. Then repeatedly merge these subgraphs together. Specifically, all operations with external incoming edges are merged together into the same subgraph. In addition, all possible sub-picture pairs with non-blocking operations are merged.

Once divided, the graphics can be executed. Each of the subgraphs is executed at a separate position in the subgraph, and each subgraph is executed in a separate single thread. Data transfer along the edges within a subgraph is optimized based on its execution in a single thread environment.

FIG. 12 illustrates an example program. 13A to 13F illustrate the main division of the program, and FIGS. 14A to 14E illustrate an example of local division. The resulting program is shown in Figure 15.

As shown in Figure 12, one of the initial graphics of the program is created. The graph includes a plurality of nodes A to K representing various operations, where edges 1211 to 1222 represent data streams flowing between the nodes. Some of the nodes have predefined positions. For example, the operation of node A is assigned to position C, and the operation of node I is assigned to position L. The operation of each of nodes B, C, E, and F is assigned to the partitioned location SL. During the segmentation, positions are automatically assigned to the remaining nodes D, J, G, H, and K.

In FIG. 13A, each node with a predefined position is placed into its own subgraph. For example, node A is placed in subgraph 1310, node B is placed in subgraph 1312, node C is placed in subgraph 1314, node E is placed in subgraph 1316, and node F is placed in subgraph In Figure 1318 and node I is placed in subgraph 1320. During segmentation, nodes with unassigned positions are placed into these subgraphs 1310 to 1320, and subgraphs are merged as much as possible. This is performed according to the main split constraints mentioned above.

For example, transform FIG. 13A into FIG. 13B by merging partitioned subgraphs as much as possible while following the main partitioning constraints. Candidates for merging include nodes B, C, E and F. Nodes A and I are not candidates because these nodes are not assigned to partitioned locations. Neither node B nor node C can be merged with node E or F because it will introduce a cycle into the graph. It is also impossible to merge subgraphs 1316 and 1318 together, because if the destination node of a sending node in a partitioned location is also in a partitioned location, the sending node and its destination cannot be merged together. Nodes B and C can be merged and merged into the same subgraph 1313.

For example, FIG. 13B is transformed into FIG. 13C by expanding the partitioned subgraph. This expansion includes adding neighboring nodes with unassigned positions to the partitioned subgraph. Nodes D and G are candidates to be folded into the partitioned subgraph because these nodes have unassigned positions and also have edges that are coupled to nodes within the partitioned subgraph. Determine whether nodes D and G have been marked as splittable. If not, the nodes are discarded as candidates. If the nodes are marked as splittable, the nodes are placed in adjacent subgraphs. Operation D cannot be placed in subgraph 1316 with node E due to the following constraint: the destination operation is assigned to a group of edges in a partitioned location and must remain unchanged. Add operation D to subgraph 1313. If nodes B and C have not been previously merged into the same subgraph, this will not be possible because it will not obey the main split constraints.

By adding node D to subgraph 1313, the operation of node D is effectively partitioned. Therefore, a new operation D 'is added to the graph to merge the results of the partitioned instances from D. Similarly, the operation of node G is placed in subgraph 1318 with node F, and a new operation G 'is added to the graph. The new operations D ’and G’ are inseparable. There is no other operation in FIG. 13C that can be placed into the partitioned location.

FIG. 13D illustrates assigning positions to all unassigned positions. This can be performed by copying the location from the assigned node to its neighbors. For example, nodes D ', J, G', H, and K have unassigned positions in Figure 13C. Nodes G ', H, and K include neighbors of subgraph 1320 of node I. Therefore, the position L assigned to the node I is also assigned to the nodes G ', H, and K. Nodes D 'and J do not have any adjacent unpartitioned subgraphs, and therefore nodes D' and J are assigned to controller C.

13E to 13F illustrate one example of merging unpartitioned subgraphs. The total number of subgraphs is minimized by merging together possible unpartitioned subgraph pairs at the same location. There are three subgraphs (1310, 1315, 1317) assigned to position C. Moreover, the subgraphs of nodes G ', H, I, and K are all assigned to position L. All sub-graphs assigned to position L can be merged into a new sub-graph 1322 without introducing a cycle. It is not possible to merge subgraph 1310 containing node A with subgraph 1315 or 1317 without introducing a cycle. However, the subgraph 1315 including the node D 'and the subgraph 1317 including the node J may be merged. A resulting graph is illustrated in Figure 13F. This graphic cannot be merged further. Each operation has been assigned to a subgraph, and the number of subgraphs has been minimized. Any further merger will break one of the main split constraints.

The blocking operation can be split into one of its own subgraphs, thereby creating a local graph executed locally on the same machine. During local segmentation, a position has been assigned. In addition, no special consideration is needed for the partitioned locations, which can be split during the local segmentation stage. Local segmentation must follow the local segmentation constraints mentioned above. These local segmentation constraints require that each lock operation ends in one of its own subgraphs. The split subgraph can use external / non-local input to generate only one subgraph, and the graph must remain acyclic. Ensure that splitting uses external input to generate only one sub-picture to achieve more transmission and reception optimization, and simplify the stylized protocol. In the graph, an external / non-local input is represented by an edge between nodes that have been assigned different positions. During the execution of the program, an external edge causes possible communication between the nodes.

In FIG. 14A, the sub-picture containing the block operation is identified. In this example, operations B and D are the only blocking operations in the program. Therefore, the subgraph 1413 including nodes B, C, and D will be split.

In FIGS. 14B to 14C, the sub-graph 1413 of FIG. 14A is modified so as to impose local segmentation constraints. One of the first modifications shown in FIG. 14B ensures that there is no blocking operation with external input in the sub-graph. Multiple blocking operations with external inputs will make it difficult or impossible to impose local segmentation constraints. These local segmentation constraints require that each block operation ends in one of its own subgraphs, and the graph remains non-pulmonary. The first modification only inserts a no-operation before the blocking operation. An empty operation or "no-op" is an operation that does not change the semantics of the program if it is inserted between two operations. An example of one-off operation is staggered. Interleaving transfers data from a node before it to a node after it. Since the blocking operation B has an external input from the node A, a No-op operation 1432 is inserted between the nodes A and B.

One of the second modifications shown in FIG. 14C ensures that there is no blocking operation with external output in the sub-graph. This prepares the subgraph for a final segmentation step, in which sending and receiving operations are inserted along their outputs, and ensuring that the sending operation does not end in the same subgraph as the blocking operation. Therefore, the second modification inserts another No-op operation 1434 between nodes D and D '.

Figure 14D illustrates the construction of a merged graph, where each operation ends in one of its own subgraphs. As shown, the merged graph 1450 includes subgraphs 1452, 1453, 1454, 1456, and 1458, each of which includes an operation.

In FIG. 14E, operations with external incoming edges are identified and merged together into the same subgraph. Since both node C and the first No-op operation have external edges that are passed in from node A outside merge graph 1450, sub-graphs 1452 and 1454 of FIG. 14D are merged into sub-graph 1455 of FIG. 14E.

As far as possible, merge sub-graphs with non-blocking operations. In FIG. 14E, there are two subgraphs 1455, 1458 containing Noop operations. However, merging the two sub-graphs 1455, 1458 will introduce a cycle, and therefore is not permitted. Since the subgraph 1453 including node B and the subgraph 1456 including node D have blocking operations that cannot be merged with any other subgraph, the local segmentation is completed.

Figure 15 illustrates the final program after the main split and the local split. Several locations have been assigned to each location in such a way as to minimize traffic sent from one location to another across the network. In addition, preventive measures have been taken to ensure efficiency by, for example, preventing circulation between nodes and optimizing transmission and reception between nodes.

FIG. 16 provides a flowchart illustrating an example method 1600 for graph creation and segmentation of an established program. Some parts of the method 1600 are explained in further detail in conjunction with FIGS. 17-18. Each of the methods set forth herein includes portions that can be performed in a different order or simultaneously, and can include additional portions, where other portions can be omitted.

At block 1610, a directed acyclic graph is created, which contains nodes representing the operation of the program. The nodes in the graph are joined by edges, which represent the flow of data from one operation to another. Some of the operations can have predefined positions. For example, these locations can be determined by the nature of the operation, the capabilities of computing devices in a distributed environment, programmer assignments, or any other information.

In block 1620, the location is assigned to any operation that does not have a predefined location. The location may be assigned based on a first set of constraints, such as the main segmentation constraints set forth above. In some instances, the first set of constraints requires that all operations be placed in a sub-graph, a position is assigned to each sub-graph, and an operation with a position assigned by a programmer must maintain that assignment and if one has not If the assigned operation has a splittable property, then the unassigned operation can only be placed into a partitioned location. In addition, if a location is partitioned, all edges in the program whose destination operation is assigned to that location remain unchanged throughout the algorithm. In addition, the graphics must be acyclic. For example, the location may be assigned based on neighboring nodes. For example, operations with unassigned positions can be added to adjacent partitioned subgraphs according to constraints. Any other operation with unassigned locations may be assigned locations that match adjacent unpartitioned nodes. The location assignment can be part of the graphics segmentation.

At block 1630, the graph is divided into a plurality of sub-graphs, where operations in a sub-graph are assigned to the same location. The division of block 1630 is explained in further detail in conjunction with FIG.

In block 1640, local segmentation is performed for individual subgraphs based on a second set of constraints, such as the local segmentation constraints discussed above. The local segmentation is further explained in conjunction with FIG. 18.

At block 1650, each sub-picture is executed at its respective location. Run individual subgraphs in a single, separate thread. The program execution is discussed more fully in the next chapter IV.

FIG. 17 provides a flow chart illustrating one of the methods 1700 of primary graphics segmentation. In block 1710, the partitioned subgraphs are merged as much as possible while following the main segmentation constraints. An example is discussed above in connection with FIG. 13B.

In block 1720, nodes with unassigned positions are added to adjacent partitioned subgraphs as much as possible, while following the main partitioning constraints. An example is discussed above in conjunction with FIG. 13C. In some instances, this may include establishing an additional operation. For example, in the case where a node with an unassigned position is effectively added to an adjacent partitioned subgraph, a new operation is added to the graph outside the subgraph to merge the The result of the partition operation.

In block 1730, the location is assigned to any remaining nodes with unassigned locations. The location may be assigned based on the location previously assigned to the neighboring node while following the main split constraint. An example is discussed above in connection with FIG. 13D.

At block 1740, merge unprovisioned subgraph pairs that may be running at the same location. An example is discussed above in connection with FIGS. 13E to 13F.

FIG. 18 provides a flowchart illustrating an example method 1800 of local segmentation. At block 1810, the subgraphs that need to be split are identified. For example, if the subgraph contains one or more blocking operations, the subgraphs may be split.

At block 1820, the identified subgraph is prepared for splitting. For example, preparation may include making modifications to ensure that there is no blocking operation with external input to the subgraph, and that there is no blocking operation with external output in the subgraph. Such modifications may include adding operations to sub-graphs, such as discussed above in connection with FIGS. 14B-14C.

At block 1830, a merged graph is constructed, where each operation ends in a separate subgraph. An example is discussed above in connection with FIG. 14D.

At block 1840, the individual subgraphs are repeatedly merged without breaking one of the local segmentation constraints until no further merge can be performed. This method can be repeated for each related subgraph in the graph. IV. Run a program across the network

Once divided, the graphics can be executed. Each of the subgraphs is executed at a separate position in the subgraph, and each subgraph is executed in a separate single thread. Data transfer along the edges within a subgraph is optimized based on its execution in a single thread environment.

When executing the program, it is determined whether a stream sent via graphics has been completed. To make this determination, an end node receives a token from each other node that sent the tuple to the end node, the token indicating that other nodes have completed providing input. The other node may be, for example, a partitioned node, or simply a partition. The end node adds tokens together, and when one sum of tokens equals the number of other nodes providing input, the end node determines that the stream has been completed.

When the program is submitted for execution, one or more launches of each graphic are created. There is a start for a unique graphic. A unique graph contains multiple nodes that each run exactly once. An example of a unique figure is provided in FIG. In this example, each of nodes A, B *, C *, and D runs once, where the stream from A is input to B *, the stream is input to C *, and the stream is input to D.

A non-unique graph can cause any number of startup replicas to execute, so that the input will be split and sent to any of these executions and the output will be merged together. An example of a non-unique figure is provided in FIG. 20. There are multiple replicas of operations B and C, where the input from node A is split among the replicas of node B, and so on. For example, nodes B1, B2, and B3 are the same operation B start. Similarly, the nodes C1 and C2 are activated by the same operation C.

When the initialization is started, each node locally tracks a certain number of upstream (transmitting nodes) and downstream nodes (receiving nodes) it is connected to. The node between the initial sending node and the final receiving node can be used as both a sending node and a receiving node, thereby receiving information from one or more nodes, performing an operation, and transmitting the information to one or more other nodes. Each value sent through a stream is a tuple. Different operations in the same program can be run on different machines. The stylized model coordinates the execution of these operators on different machines and propagates data from one operator to another.

Since the operators are running on different machines and therefore at different nodes of the graph, parts of the program run in parallel. To determine whether a particular stream is complete, the destination node sums one of the number of tokens received from the upstream operator of the partition. For example, when the input to a sending node ends, the sending node transmits a token value (for example, 1) to each node to which the sending node has transmitted information. When the destination node receives a token value whose total number reaches one of the number of sending nodes to which it is connected, the destination node determines that the streaming has ended. Therefore, the destination node can take an action, such as generating an end signal or marking the stream as completed. In one example, the destination node sends a completion token to other downstream nodes.

Figure 21 illustrates an example of sending token values to signal the completion of a stream. Each of nodes B0, B1, and B2 receives input from a node A. When node A has finished sending the stream, node A sends a token value (such as 1) to each of the connected downstream nodes B0, B1, B2. Nodes B0, B1, B2 will wait to receive a token value equal to the number of senders (in this case, 1). Each of nodes B0, B1, B2 then sends the stream to a single destination node C. Destination node C knows that it receives input from three different nodes B0, B1, B2, and therefore the destination node C waits to receive a token value equal to 3. When the nodes B0, B1, and B2 finish sending the stream, the nodes send a token value to the connected downstream node, that is, the destination node C. The destination node C sums the received symbol values and compares the sum with the number of nodes from which the destination node C receives input. When the number is equal, node C marks itself as completed.

FIG. 22 illustrates an example in which a sending node only sends a stream to a subset of receiving nodes to which the sending node is connected. For example, nodes B and C are partitioned, but only some of the partition pairs communicate. B0 may only contact the partition C0 without touching the partition C1, and the partition B1 only contacts the partition C1 without touching the partition C0. In this scenario, the sending node may generate a list of all the receiving partitions it has communicated with, and may further provide this list to a controller 2250. For example, the list may be included in a message indicating that the sending partition has completed its transmission of the stream. The controller tracks all receiving partitions that have started processing. If a specific partition has started but does not exist in the list, the controller is responsible for that specific partition and sends a token value representing the sending partition to the specific partition.

FIG. 23 illustrates an example in which some sending nodes may not start processing. For example, nodes C and D may be skipped by node B, and tuples are directly provided to destination node E from a sending node before the skipped node. Another possibility is that nodes B, C, and D are all skipped by node A, which provides tuples directly to destination node E. In either case, the controller takes over responsibility for delivering all tokens from the graphics. The controller simulates the execution of skipped nodes B, C, and D, and delivers the token value to downstream receivers (node E) representing these uninitiated sending nodes. The downstream receiver may sum up the token values to determine whether the stream has been completed.

As mentioned above, the graphics used for the program can be unique or non-unique, one of the partitions of a unique graphics runs once and a non-unique graphics allows any number of copies to be executed for one or more partitions. For example, in a non-unique graph, the input is split among the replicas, and the output is merged together. In a graph where each sending node is not uniquely paired with a receiving node, the receiving node can locally track a certain number of sending nodes, and when the receiving node receives a token number equal to the number of sending nodes, the The receiving node determines that a stream has been completed.

Each sending operation partition may have its own non-unique recipient. In this example, the sender only sends a 1 and the corresponding non-unique receiver completes when it receives a 1. One optimization can be introduced that shares the same execution across multiple senders with a non-unique receiver. Then, the certain number of senders is tracked locally by the receiver, and the non-unique receiver completes when it has received a token equal to the number of senders.

In other examples, a non-unique sender may send the stream to a unique receiver. Each non-unique sender sends a token value of 1 to the unique receiver and the unique receiver waits for a total of token values equal to the number of non-unique senders. When the non-unique sender completes, it sends a list of receiving partitions to which the non-unique sender has sent tokens and the controller is responsible for delivering the remaining tokens to each partition. In other examples, the controller may deliver all tokens for non-unique senders to each receiving partition.

In some instances, certain streams can be broadcast to all activations of a graphic. However, the startup set is not known at a time when the program is constructed, and the set is gradually built as the program execution continues. FIG. 24 illustrates an example in which a non-unique graph G receives a regular input X on partition R1 that can cause multiple activations of G. For example, G can be partitioned (including partitions R1 and R2), or senders S1 through R1 can have multiple activations to different replicas of G. The other input Y should be broadcast to each copy of G. For each of these broadcasts, a dynamic transmission operation S2 is introduced. The dynamic transmission operation S2 has two input streams—a data input stream from Y and a start input stream from one of the controllers. The data input stream should be sent to all normal tuple streams of the destination graph G. When a new activation of the destination graphic is detected, the activation input stream contains the activation on which the tuple reached. For example, when a copy of a specific graphic is executed, an identifier is sent to the controller, which will initiate routing to the appropriate dynamic sending operation. The dynamic transmission operation maintains a set activated by temporary storage, and also maintains a buffer of all input data received. When the data input is completed, an end signal is sent from the dynamic sending operation to all the temporary start and new start, and also to the new start that arrives later. When the start input ends, the input data buffer can be discarded.

FIG. 25 provides a flowchart illustrating an example method 2500 for executing a program via a decentralized network. The program is represented by a graph including a plurality of nodes representing operations, where an edge represents the Data stream of interconnected nodes. As in the above example, the sub-portions of method 2500 may be reordered, supplemented, or reduced.

In block 2510, the operation is performed by one or more first partitions. For example, the first partition may be a sending partition in a graphic.

In block 2520, the tuple based on the performed operation is sent from one or more first partitions to at least one second partition, such as a receiving partition.

In block 2530, when one or more first partitions have finished sending tuples, each of the one or more first partitions sends a token value to at least one second partition. For example, the symbol value may be 1. One or more first partitions can further locally note the completion of the tuple transmission.

In block 2540, at least one second partition sums the received token values, and determines whether the sum matches one of the number of one or more first partitions. For example, at least one second partition may be aware that it receives input from three sending partitions. Therefore, at least one second partition waits until it receives a total of 3 tokens before it considers the stream to be completed.

In block 2550, at least one second partition takes an action in response to the sum of the determination token values matching the number of one or more first partitions. The action may be, for example, making a local note, sending a message and / or token value to another downstream node, and so on.

The techniques described above provide fast and efficient program execution. In addition, the techniques described can be adapted to various types of partitioned and pipelined programs. Furthermore, these techniques can be applied during the writing of a program, and therefore dynamically adapt to changes in the program. The new stylized model supports an unlimited number of partitions, and each partition transfers tuples across databases to other partitions. Although only certain subsets of the partitions can actually be run for a specific program, the controller compensates for partitions that are not running, without putting a significant burden on the controller.

Unless stated otherwise, the aforementioned alternative examples are not mutually exclusive, but may be implemented in various combinations to achieve unique advantages. Since these and other changes and combinations of the features discussed above can be utilized without departing from the subject matter defined by the scope of the patent application, it should be defined by the scope of the patent application by way of illustration rather than by limitation. The foregoing description of the embodiment is understood in the manner of the subject matter. In addition, the provision of the examples set forth in this article and the clauses such as "such as", "including" and the like should not be interpreted as limiting the subject matter of the patent application scope to specific examples; The illustration illustrates only one of many possible embodiments. In addition, the same element symbols in different drawings can identify the same or similar elements.

110‧‧‧Client

120‧‧‧ processor

130‧‧‧Memory

132‧‧‧Command

134‧‧‧Information

136‧‧‧Application

150‧‧‧ Internet

160‧‧‧Data Center

162‧‧‧Computer

164‧‧‧ computing device

170‧‧‧Data Center

172‧‧‧ computing device

180‧‧‧Data Center

181‧‧‧ computing device

182‧‧‧ computing device

183‧‧‧ computing device

184‧‧‧ computing device

185‧‧‧ Computing device

186‧‧‧ computing device

190‧‧‧Controller

192‧‧‧Memory

194‧‧‧ Information

196‧‧‧Command

198‧‧‧ processor

210‧‧‧ Input operation

215‧‧‧Stream

220‧‧‧ListImages operation / ListImages

225‧‧‧Stream

226‧‧‧ Search operation in cache

227‧‧‧Stream

228‧‧‧Stream

230‧‧‧Find Operator / Find

235‧‧‧Stream

240‧‧‧ Thumbnail operation

245‧‧‧ generated thumbnail

250‧‧‧Output

500‧‧‧Type inference method

510‧‧‧ block

520‧‧‧ block

530‧‧‧ block

540‧‧‧ block

550‧‧‧ block

560‧‧‧ block

610‧‧‧Argument

620‧‧‧Find

630‧‧‧filter

640‧‧‧ result

710‧‧‧Argument

720‧‧‧Find

730‧‧‧filter

740‧‧‧Result

815‧‧‧ edge

825‧‧‧edge

835‧‧‧edge

920‧‧‧Find

930‧‧‧Process

940‧‧‧Find

1020‧‧‧Find

1030‧‧‧Find

1110‧‧‧ First input parameter

1115‧‧‧Edge

1125‧‧‧edge

1130‧‧‧First search operation / search operation

1135‧‧‧edge

1140‧‧‧Second search operation / search operation

1145‧‧‧ Edge

1150‧‧‧ZipAny operation / ZipAny

1155‧‧‧ Edge

1160‧‧‧select operation / select

1165‧‧‧edge

1170‧‧‧Result

1211‧‧‧edge

1212‧‧‧Edge

1213‧‧‧Edge

1214‧‧‧Edge

1215‧‧‧edge

1216‧‧‧edge

1217‧‧‧edge

1218‧‧‧Edge

1219‧‧‧Edge

1220‧‧‧Edge

1221‧‧‧Edge

1222‧‧‧Edge

1310‧‧‧Sub-picture

1312‧‧‧Sub-picture

1313‧‧‧Sub-picture

1315‧‧‧Sub-picture

1316‧‧‧Sub-picture

1317‧‧‧Sub-picture

1318‧‧‧Sub-picture

1320‧‧‧Sub-picture

1322‧‧‧New sub-picture

1413‧‧‧Sub-picture

1432‧‧‧No-op operation

1434‧‧‧No-op operation

1450‧‧‧ merged graphics

1452‧‧‧Sub-picture

1453‧‧‧Sub-picture

1454‧‧‧Sub-picture

1455‧‧‧Sub-picture

1456‧‧‧Picture

1458‧‧‧sub-picture

1600‧‧‧Method

1610‧‧‧ block

1620‧‧‧ block

1630‧‧‧ block

1640‧‧‧ block

1650‧‧‧ block

1700‧‧‧Method

1710‧‧‧ block

1720‧‧‧ block

1730‧‧‧ block

1740‧‧‧ block

1800‧‧‧method

1810‧‧‧ block

1820‧‧‧ block

1830‧‧‧ block

1840‧‧‧ block

2500‧‧‧Method

2510‧‧‧ block

2520‧‧‧ block

2530‧‧‧ block

2540‧‧‧ block

2550‧‧‧ block

A‧‧‧Node

B‧‧‧node / operation / block operation

B0‧‧‧node / downstream node

B1‧‧‧Node / Downstream Node / Division

B2‧‧‧node / downstream node

B3‧‧‧Node

C‧‧‧Destination node / node / location / controller / operation

C0‧‧‧Division

C1‧‧‧ partition / node

C2‧‧‧Node

D‧‧‧node / operation

D’ ‧‧‧ New operation / node

E‧‧‧node / destination node

F‧‧‧ Node

G‧‧‧node / non-unique graph / destination graph

G’‧‧‧New operation / node

G0‧‧‧Individual graphics / graphics

G1‧‧‧Separate graphics / graphics

G2‧‧‧separate graphics

G3‧‧‧separate graphics

H‧‧‧ Node

I‧‧‧ Node

J‧‧‧ Node

K‧‧‧node

L‧‧‧Location

R1‧‧‧ Division

S1‧‧‧Streaming / Sender

S2‧‧‧Streaming / Dynamic sending operation

SL‧‧‧Location / Divisional location

X‧‧‧Regular input

Y‧‧‧Input

FIG. 1 is a block diagram of an exemplary system according to an aspect of the present invention.

2A to 2B illustrate an example of creating a program using a stylized model according to aspects of the present invention.

FIG. 3 is a diagram showing an example of built-in operations of the stylized model according to the aspect of the present invention.

FIG. 4 is a diagram showing examples of output type annotations for operations according to aspects of the present invention.

FIG. 5 provides a flowchart illustrating one example type inference method according to aspects of the present invention.

6A to 6C illustrate an example of position assignment during program creation according to aspects of the present invention.

7A to 7B illustrate an example of automatic position assignment during pattern creation according to aspects of the present invention.

8A to 8C illustrate an example of dividing a figure according to the aspect of the present invention to minimize the number of one of the sub-pictures.

9A-9D provide an example of graphical segmentation for partitioned locations according to aspects of the present invention.

10A to 10B provide an example of common positions according to aspects of the present invention.

11A to 11C illustrate an example of a program having multiple input operations according to aspects of the present invention.

FIG. 12 illustrates an example program according to aspects of the present invention.

13A to 13F illustrate an example of the main division of the program of FIG. 12 according to the aspect of the present invention.

14A to 14E illustrate an example of local segmentation of the program of FIG. 12 according to aspects of the present invention.

FIG. 15 illustrates the program of FIG. 12 after performing main division and local division according to the aspect of the present invention.

16 provides a flowchart illustrating an example method of graph creation and segmentation according to aspects of the present invention.

FIG. 17 provides a flowchart illustrating one of the methods of main graphics segmentation.

FIG. 18 provides a flowchart illustrating one example method of local segmentation.

FIG. 19 is a graphical illustration of an example of a unique figure according to the aspect of the invention.

Figure 20 is a graphical illustration of one of the exemplary non-unique graphics according to one aspect of the present invention.

FIG. 21 illustrates an example of transmitting token values according to the aspect of the present invention to signal completion of a stream.

FIG. 22 illustrates an example in which a sending node only sends a stream to a subset of receiving nodes to which the sending node is connected.

FIG. 23 illustrates an example of determining completion of a stream when there is a sending node that has not been initiated.

FIG. 24 illustrates an example of determining the completion of a stream when there are multiple activations of a graphic.

FIG. 25 provides a flowchart illustrating an example method 2500 for executing a program over a distributed network.

Claims (20)

A method for executing a program in a decentralized architecture includes: performing one or more operations by one or more first partitions of the decentralized architecture; tuples from the one or more first partitions Sent to at least one second partition, the tuples are part of a stream and based on the one or more operations; when the sending of the tuples in the stream is completed, from the one or more Each of the partitions sends a token value to the at least one second partition; the second partition determines whether a total of the token values matches one of the one or more first partitions; and A first action is taken in response to determining that the total number of the token values matches the number of the one or more first partitions. The method of claim 1, wherein the at least one second partition is one of the one or more first partitions receiving the partition, the method further comprises: the one of the one or more first partitions It generates a list of the receiving partitions with which the one or more first partitions communicate; and transmits the list to a controller by the one of the one or more first partitions. The method of claim 2, further comprising: tracking, by the controller, all receiving partitions that have initiated processing; determining, by the controller, whether one or more of the initiated processing in the receiving partitions does not exist in the In the list; and for each receiving partition that has been initiated and does not exist in the list, the controller sends a token value representing the one of the one or more first partitions to the receiving Partition. The method of claim 1, further comprising: determining by a controller whether any partitions have not been processed; determining by the controller whether the partitions that have not been processed are intentionally skipped by the design of the program; and by the The controller sends a token value to the second partition on behalf of any partition that has been deliberately skipped. The method of claim 1, wherein taking the first action includes marking the stream as completed or generating at least one of a message indicating that the stream is completed. The method of claim 1, further comprising: constructing a graph, wherein each node of the graph represents a partition; and verifying based on the graph whether the program will be accurately executed across the decentralized architecture. The method of claim 6, which further includes dynamically starting the graph when the program is executed. The method of claim 7, further comprising: sending a data input stream to all activations of a destination graph by a dynamic sending operation; receiving a new tuple from a controller at the dynamic sending operation, the The new tuple is received when an additional activation of the destination graphic is detected. The method of claim 6, wherein the graphic is not unique. The method of claim 1, wherein performing the one or more operations is part of a pipelined data processing stream. A system includes: one or more first partitions in a distributed computing environment; and at least one second partition in the distributed computing environment, the at least one second partition is remote from the one or more first partitions ; Wherein the one or more first partitions are configured to: perform one or more operations; send tuples to at least one second partition, the tuples are part of a stream and are based on the one or more Operation; when the sending of the tuples in the stream is completed, sending a token value to the at least one second partition; and wherein the at least one second partition is configured to: determine the tokens Whether the total number of values matches one of the one or more first partitions; and in response to determining that the total number of tokens matches the number of the one or more first partitions, a first action is taken. The system of claim 11, further comprising a controller, wherein the at least one second partition is a receiving partition of one of the one or more first partitions, and wherein the one or more first partitions further It is configured to: generate a list of the receiving partitions with which the one or more first partitions communicate; and transmit the list to the controller. As in the system of claim 12, wherein the controller is configured to: track all received partitions that have initiated processing; determine whether one or more of the initiated processing in those received partitions does not exist in the list; And for each receiving partition that has started processing and does not exist in the list, a token value representing the one of the one or more first partitions is sent to the receiving partition. As in the system of claim 11, one of the controllers is configured to: determine whether any partitions have not been processed; determine whether the partitions that have not started processing are intentionally skipped by a program design; Any partition that was intentionally skipped will send a token value to the second partition. The system of claim 11, wherein taking the first action includes marking the stream as completed or generating at least one of a message indicating that the stream is completed. The system of claim 11, further comprising a client device communicating with at least one of the one or more first partitions, the at least one second partition, or the controller, the client device configured to : Construct a graph, where each node of the graph represents a partition; and verify whether a program will be accurately executed across the decentralized architecture based on the graph. The system of claim 16, wherein the client device is further configured to dynamically construct the start of the graphic while executing the program. The system of claim 17, further comprising performing a dynamic sending operation on a computing device in the distributed architecture, wherein the dynamic sending operation: sends a data input stream to all activations of a destination graphic; New tuples are received from the controller, and these new tuples are received when an additional activation of the destination graphic is detected. As in the system of claim 16, the graphic is not unique. The system of claim 11, wherein performing the one or more operations is part of a pipelined data processing flow.