WO2021035397A1

WO2021035397A1 - Method and apparatus for data-move task optimizing

Info

Publication number: WO2021035397A1
Application number: PCT/CN2019/102268
Authority: WO
Inventors: Yuqing Wang; Youben YE; Weiming Zhao; Peng Zhou; Weifeng Zhang
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2019-08-23
Filing date: 2019-08-23
Publication date: 2021-03-04
Anticipated expiration: 2022-02-23
Also published as: CN114041116B; CN114041116A

Abstract

Methods and devices for optimizing data-move tasks associated with a machine learning model. The method can include identifying a data-move task in a program executable in a device, identifying one or more operations that are associated with the data-move task, determining hardware capabilities of the device and requirements of optimizing data-move tasks, and in response to a determination that the hardware capabilities of the device satisfy the requirements, mapping the identified data-move task to data writing of the identified one or more operations that are associated with the data-move task.

Description

METHOD AND APPARATUS FOR DATA-MOVE TASK OPTIMIZING

TECHNICAL FIELD

The present disclosure is generally related to the field of computation optimization, and in particular, to methods and apparatus for data-move task optimizing.

BACKGROUND

Due to growing disparity of speed between a CPU and memory external to the CPU chip, reducing computation time of data-move operations, which are widely present in tasks of computer software, becomes a focus. Operations such as copy and move can be time-consuming. Acceleration of such operations can greatly improve performance of computation and can lead to acceleration of model learning in machine learning for heterogeneous devices (e.g. FPGA and NPU) . In an example of calculation operations followed by a data-move operation, optimization of data-move operations provided by conventional solutions are separate from optimization of the calculation operations, performance improvement generated by such optimization is limited. The conventional solutions such as using manual assembler code and copying data with multiple threads have not fully addressed the problem. There is a need for a more efficient data-move operation optimization.

SUMMARY

Embodiments of the present disclosure provide a method for optimizing data-move tasks associated with a machine learning model. The method can include identifying a data-move task in a program executable in a device, identifying one or more operations that are associated with the data-move task, determining hardware capabilities of the device and requirements of accelerating data-move tasks, and in response to a determination that the hardware capabilities of the device satisfy the requirements, mapping the identified data-move task to data writing of the identified one or more operations that are associated with the data-move task.

Embodiments of the present disclosure also provide an apparatus for optimizing data-move tasks associated with a machine learning model. The apparatus can comprise a memory storing a set of instructions, and one or more processors configured to execute the set of instruction to cause the apparatus to perform: identifying a data-move task in a program executable in a device, identifying one or more operations that are associated with the data-move task, determining hardware capabilities of the device and requirements of accelerating data-move tasks, and in response to a determination that the hardware capabilities of the device satisfy the requirements, mapping the identified data-move task to data writing of the identified one or more operations that are associated with the data-move task.

Embodiments of the present disclosure also provide a non-transitory computer readable medium that stores a set of instructions that is executable by at least one processor of a computer to cause the computer to perform a method for optimizing data-move tasks associated with a machine learning model. The method can comprise identifying a data-move task in a program executable in a device, identifying one or more operations that are associated with the data-move task, determining hardware capabilities of the device and requirements of accelerating data-move tasks, and in response to a determination that the hardware capabilities of the device satisfy the requirements, mapping the identified data-move task to data writing of the identified one or more operations that are associated with the data-move task.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings described herein are used to provide further understanding of the present disclosure and constitute a part of the present disclosure. Exemplary embodiments of the present disclosure and descriptions of the exemplary embodiments are used to explain the present disclosure and are not intended to constitute inappropriate limitations to the present disclosure. In the accompanying drawings:

FIG. 1 illustrates a schematic diagram of an exemplary deep learning accelerator system, consistent with embodiments of the present disclosure.

FIG. 2 illustrates a block diagram of exemplary components of a system including an apparatus for optimizing data-move tasks, consistent with embodiments of the present disclosure.

FIG. 3A illustrates a schematic diagram of an exemplary calculation model before being optimized, consistent with embodiments of the present disclosure.

FIG. 3B illustrates a schematic diagram of an exemplary calculation model after being optimized, consistent with embodiments of the present disclosure.

FIG. 4 illustrates a flow chart of an exemplary method of optimizing data-move tasks, consistent with embodiments of the present disclosure.

FIG. 5 illustrates a flow chart of an exemplary method of optimizing data-move tasks, consistent with embodiments of the present disclosure.

FIG. 6A illustrates a schematic diagram of an exemplary data move optimizing of merged calculation processes, consistent with embodiments of the present disclosure.

FIG. 6B illustrates a schematic diagram of data move optimizing of interleaved calculation processes, consistent with embodiments of the present disclosure.

DETAILED DESCRIPTION

To facilitate understanding of the solutions in the present disclosure, the technical solutions in some of the embodiments of the present disclosure will be described with reference to the accompanying drawings. It is appreciated that the described embodiments are merely a part of rather than all the embodiments of the present disclosure. Consistent with the present disclosure, other embodiments can be obtained without departing from the principles disclosed herein. Such embodiments shall also fall within the protection scope of the present disclosure.

In order to improve computing performance, some current solutions focus on improving processes of data-move tasks, for example, by using manual assembler code and copying data with multiple threads. The present disclosure addresses the problem by merging the data-move tasks into operations (e.g., calculation operations) that are performed before the data-move tasks and redirecting output operands of the data-move tasks to offsets generated by a function such that the data-move tasks can be eliminated.

In one example, the optimization of data-move tasks can be used in machine learning systems in a Point-of-Sale (POS) machine that accelerates model learning using field programmable gate arrays (FPGAs) . It is appreciated that the embodiments of the present disclosure can be used in many systems, including autonomous driving systems, voice recognition systems and identity recognition systems, which benefit from computation acceleration. The optimization can be applied in heterogeneous devices such as FPGAs and neural processing units (NPUs) . It is appreciated that the optimization can be used in any type of compiler and computation framework. It is also appreciated that the embodiments can be used in a CPU and any type of accelerator (e.g., FPGAs, NPU, GPU, GRU and ASIC) .

FIG. 1 illustrates a block diagram of an exemplary deep learning accelerator system 100, according to embodiments of the disclosure. Deep learning accelerator system 100 may include a neural network processing unit (NPU) 102, a NPU memory 104, a host CPU 108, a host memory 110 associated with host CPU 108, and a disk 112. Accelerator system 100 can perform data-move optimizing. It is appreciated that while FIG. 1 shows the accelerator system as using an NPU, any type of accelerator can be used.

As illustrated in FIG. 1, NPU 102 may be connected to host CPU 108 through a peripheral interface (not shown) . As referred to herein, a neural network processing unit (e.g., NPU 102) may be a computing device for accelerating neural network computing tasks. In some embodiments, NPU 102 may be configured to be used as a co-processor of host CPU 108.

In some embodiments, NPU 102 may comprise a compiler (not shown) . The compiler may be a program or a computer software that transforms computer code written in one programming language into NPU instructions to create an executable program. In machining applications, a compiler may perform a variety of operations, for example, pre-processing, lexical analysis, parsing, semantic analysis, conversion of input programs to an intermediate representation, code optimization, code generation, or combinations thereof.

In some embodiments, the compiler may be on a host unit (e.g., host CPU 108 or host memory 110 of FIG. 1) , configured to push one or more commands to NPU 102. Based on these commands, a task manager (not shown) of NPU 102 may assign any number of tasks to one or more processing elements of NPU 102. Some of the commands may instruct a DMA unit to load instructions and data from host memory into a global memory. The loaded instructions may then be distributed to each processing element assigned with the corresponding task, and the one or more processing elements may process these instructions.

It is appreciated that the first few instructions received by the processing element may instruct the processing element to load/store data from the global memory into one or more local memories of the processing element (e.g., a memory of the processing element or a local memory for each active processing element) . Each processing element may then initiate the instruction pipeline, which involves fetching the instruction (e.g., via a fetch unit) from the local memory, decoding the instruction (e.g., via an instruction decoder) and generating local memory addresses (e.g., corresponding to an operand) , reading the source data, executing or loading/storing operations, and then writing back results.

Host CPU 108 may be associated with host memory 110 and disk 112. In some embodiments, host memory 110 may be an integral memory or an external memory associated with host CPU 108. Host memory 110 may be a local or a global memory. In some embodiments, disk 112 may comprise an external memory configured to provide additional memory for host CPU 108.

FIG. 2 illustrates a block diagram of exemplary components of a system including an apparatus for optimizing data-move tasks, consistent with embodiments of the present disclosure. Apparatus 200 for optimizing data-move tasks can be implemented within a system. In some embodiments, the system can be a neural network accelerator system 100 of FIG. 1. In some embodiments, apparatus 200 can be a complier on host CPU 108 or host memory 110 shown in FIG. 1. It is appreciated that while the compiler is used to perform data-move optimizing, optimizing data-move tasks can also be implemented in any framework that employs optimization (e.g., TensorFlow for deep learning) . The framework can be an abstraction in which software providing generic functionality can be selectively changed by additional user-written code, thus providing application-specific software. Apparatus 200 can include a data-move task finder 210, a task analyzer 220, a storage allocator 230, an address redirector 240 and a data-move task remover 250.

Data-move task finder 210 can identify a data-move task (DMT) in a program executable in a device (e.g., deep learning accelerator system 100 as shown in FIG. 1) . The data-move task can contain an operation of moving or copying data. Data-move task finder 210 can also identify one or more operations that are associated with the data-move task. The associated operations include operations that are performed before the data-move task. The operations can be calculation operations such as multiplication and addition. In an example that apparatus 200 is the complier on host CPU 108 or host memory 110 shown in FIG. 1, input of the compiler can be model definitions and model data files. The module definition describes primary inputs of the model, intermediate computations and outputs of the model. The model data files can include weights, biases, parameters and coefficients. The compiler can first analyze structure and operations of the model, recognize the part that can be optimized and conduct optimization, convert the model definitions to Intermediate Representation (IR) , and generate binary instructions according to the instruction set architecture (ISA) of a device. Input and output of the model can be tensors (e.g., input tensors and output tensors shown in FIG. 3A and FIG. 3B) . Data-move task finder 210 shown in FIG. 2 can recognize operations of a data-move type such as concatenate operations (e.g., Concat as shown in FIG. 3A) , and Split operations when analyzing the operations of the model. In the example of a Concat operation, the hardware may be required to concatenate two 3-dimensional input data along one dimension. For instance, data matrix [10] [20] [30] and data matrix [10] [40] [30] are concatenated along the second dimension to generate a result of [10] [60] [30] . In the example of a Split operation, a multi-dimensional data can be split along one dimension to generate multiple output data. For instance, data matrix [2] [3] [4] can be split along the third dimension to generate four outputs of [2] [3] . The above exemplary operations can be identified as data-move types.

A schematic diagram of an exemplary calculation model before being optimized is illustrated in FIG. 3A, consistent with embodiments of the present disclosure, and a schematic diagram of an exemplary calculation model after being optimized is illustrated in FIG. 3B, consistent with embodiments of the present disclosure. As shown in FIG. 3A, blocks of various sizes represent the exemplary inputs tensors and output tensors. It is appreciated that while the optimization can be used in deep learning, computation using any data block can employ the data-move task optimization. In deep learning, the input is model structures and definitions. In an example, a data block as an input can be a matrix, a vector, an integer, or a float. The data block can be defined in granularity. For example, when NPUs and FPGAs are used, the data block as an input can have larger granularity, while when CPU and GPU are used, the data block as an input can represent one integer. The exemplary calculation model shown in FIG. 3A can involve three operations including a matrix-multiplication (MatMul) operation, an addition (Add) operation, and a concatenation (Concat) operation. In the illustrated example, four input tensors are calculated to generate one output tensor. The compiler can identify that Concat operation concatenates two data operands along a second axis.

Now referring back to FIG. 2, task analyzer 220 can determine whether hardware capabilities of the device meet requirements associated with the data-move task. Task analyzer 220 can also determine sizes of output operands (e.g. output tensors shown in FIG. 3A) of the identified operations that are associated with the data-move task. Not all data-move type operations can be optimized, because patterns of some operations cannot be extracted or the hardware are not able to optimize the operations. The requirements are met when a minimum size of data that the hardware is configured to write is equal to or smaller than the minimum size of the output operands. For example, an accelerator such as an NPU can be limited in hardware capabilities, as the ISA of the NPU may not support such optimization. For instance, while the minimum size that the NPU is configured to write can be ten integers, the optimization may require writing one integer each time when the data-move task is merged into the identified operations. Since the minimum size is ten integers, the NPU is not capable of writing one integer, which is less than ten, or skipping one integer to map offsets for optimization. The accelerator can be limited by the hardware capabilities so that the accelerator cannot write results of operations in a certain way that may be required for optimization. If the hardware is capable of optimizing the operations, the elimination of data-move operations can be performed. The data-move task can be merged into a calculation operation that is performed prior to the data-move operation.

Address redirector 240 can map the identified data-move task to data writing of the identified operations that are associated with the data-move task when it is determined that the hardware of the device (e.g., deep learning accelerator system 100 as shown in FIG. 1) meet the requirements of optimizing the data-move task. Address redirector 240 can generate a function that maps input offsets and output offsets. An offset can be an integer indicating distance between a base address and a given operand. The function can infer the output offset of the data-move task when the input offset of the data-move task is given. When data is written in a calculation operation performed immediately prior to the data-move task, an output offset generator can be used to write results of the calculation operation to an output offset. The output offset generator can be adjusted by the function. Address redirector 240 can merge the data-move task into the calculation operation by redirecting the results of the calculation operation to the output offset of the data-move task by adjusting the output offset generator of the calculation operation with the function that infers the output offset of the data-move task.

The exemplary calculation model before optimization and after optimization are illustrated in FIG. 3A and FIG. 3B, in which the optimized calculation model in FIG. 3B is performed without executing a data-move task, Referring back to FIG. 2, address redirector 240 in the exemplary compiler can mark in the matrix-multiplication (MatMul) operation and the addition (Add) operation to instruct writing the results in distance-based intervals, and adjust instructions for the matrix-multiplication operation and the addition operation based on offset change of the results. The result of the matrix-multiplication operation is a 2*4 sized Temporary Tensor (shown as 2*4 blocks in FIG. 3A) , and the result of the addition operation is a 2*1 sized Temporary Tensor (shown as 2*1 blocks in FIG. 3A) . In the matrix-multiplication operation, after every four units of data are written in a memory of the device, a length of one unit of data is skipped, the skipped space is reserved for the results of the addition operation. In the addition operation, after every one unit of data is written, a length of four units of data is skipped, the skipped space is reserved for the results of the MatMul operation. Then the compiler can generate instructions to merge the calculation operations and the data-move operation. As shown in FIG. 3B, the optimized calculation model can perform two calculation operations without performing a data-move operation. Accordingly, when the compiler employs the optimization of data-move operations by combining the calculation operations with the data-move task, computation performance can be improved.

Referring back to FIG. 2, storage allocator 230 is provided ahead of the operations (e.g., the calculation operations) , and storage allocator 230 can allocate storage that is determined to be sufficient to store the operations before the operations are performed. Storage allocator 230 can work in combination with address redirector 240 for optimizing the data-move tasks.

Data-move task remover 250 can eliminate the data-move task that has been mapped. When the data-move task is merged into the operations that are prior to the data-move task, performing the data-move task is replaced by redirecting and writing the results to the output offset generated by the function based on sizes of output operands of the operations. As shown in FIG. 3B, the results of MatMul operation and Add operation are concatenated without performing a concatenate operation. Accordingly, the data-move task can be eliminated. Therefore, resources that are dedicated for the data-move tasks before optimization can be released after optimization. The overall computation of the accelerator is improved.

FIG. 4 illustrates a flow chart of an exemplary method 400 of optimizing data-move tasks, consistent with embodiments of the present disclosure. Method 400 can include the following steps. Method 400 can be performed by a compiler on a processing device (e.g., host CPU 108 or NPU 102 in FIG. 1) .

In step 401, a data-move task is identified in a program executable in a device such as an accelerator. The data-move task can be an operation of moving or copying data. For example, a compiler can determine if an operation is a data-move task when the compiler can analyze behaviors of the operation. In examples of a concatenate operation (as shown in FIG. 3A) and a split operation, the compiler can determine how input operands are concatenated along a dimension to generate output operands for the concatenate operation and how input operands are split along a dimension to generate output operands for the split operation. The compiler can recognize that the concatenate operation and the split operation are data-move tasks. The number of the input operands of the data-move task can vary. The number of the output operands of the data-move task can also vary.

In step 402, one or more operations that are associated with the data-move task are identified. The one or more operations that are associated with the data-move task include operations that are performed prior to the data-move task. The operations can be calculation operations such as multiplication and addition. The compiler can identify the operations that are immediately prior to the data-move tasks for merging the data-move tasks and the operations.

In step 403, hardware capabilities of the device and requirements of optimizing data-move tasks are determined. If the hardware capabilities satisfy the requirements, method 400 proceeds to step 404. If the hardware capabilities do not satisfy the requirements, method 400 ends. Step 403 can include determining sizes of output operands (e.g. output tensors shown in FIG. 3A) of the identified operations that are associated with the data-move task. The compiler can determine whether optimization can be performed on the operations based on the definitions of the operations and capabilities of the hardware. The requirements may be met when a minimum size of data that the hardware is configured to write is equal to or less than the minimum size of the output operands. For example, an accelerator such as an NPU can be limited in hardware capabilities, as the ISA of the NPU may not support optimization. For instance, the minimum size that the NPU is configured to write can be ten integers, the optimization may require writing one integer, which is less than ten, and skipping one integer when the data-move task is merged into the identified operations. Since the minimum size is ten integers, the NPU is not capable of writing one integer or skipping one integer to map offset for the optimization.

In step 404, if the hardware capabilities of the device meet the requirements of optimizing the data-move task, the identified data-move task is mapped to data writing of one or more operations that are associated with the data-move task. For example, the operations can include calculation operations such as matrix-multiplication and addition. Mapping the identified data-move task to data writing of the identified one or more operations that are associated with the data-move task is further described in FIG. 5. Step 404 can further include step 501 and step 502 shown in FIG. 5. FIG. 5 illustrates a flow chart of an exemplary method 500 of optimizing data-move tasks, consistent with embodiments of the present disclosure. Method 500 can include the following steps.

In step 501, a function that maps an output offset of the identified data-move task in response to an input offset of the identified data-move task is generated. An offset can be an integer indicating distance between a base address and a given operand. The base address can be an address serving as a reference point for other addresses. The base address can represent a starting address of a storage space. An output offset generator can be used to write results of the identified operations associated with the data-move task to a certain offset. The function can map offset addresses using the following approaches. The function can modify a page table of an operating system. The page table is a data structure mapping between virtual addresses used in a program and physical addresses used in hardware. The function can construct and maintain a lookup table of offset mapping. The function can call a designed formula. The function can also directly modify how the offset changes, e.g., from increasing 1 increment each time to increasing 2, 2, 5 increments periodically. The increments can be determined based on the sizes of the output operands of the operations associated with the data-move task. Any of the above approaches can be used to convert an old address to a new address (e.g., address 100 to address 300) for writing results of operations. The physical address for writing the results is obtained by adding the base address with the offset.

In step 502, the output offset of the identified data-move task is redirected using the function based on the sizes of the output operands of the identified operations associated with the identified data-move tasks. The function can be used to adjust the output offset generator such that the results of the identified operation are redirected and written to the output offset generated by the function. The output offset can be adjusted to obtain an actual address for writing the results of the data-move task.

Now referring back to FIG. 4, in step 405, the data-move task that has been mapped is eliminated. After the identified data-move task is mapped, the data-move task is merged into the associated operations. As shown in FIG. 3A and FIG. 3B, after optimization, Concat operation of two temporary tensors in FIG. 3A is eliminated. In FIG. 3B, the results of MatMul operation and Add operation are concatenated without performing a concatenate operation. Therefore, the mapped data-move task can be removed such that the computation is optimized.

In step 406, it is determined whether there are any more identified data-move tasks to be processed. If another identified data-move task exists to be processed, method 400 proceeds to step 403. If there are no other identified data-move tasks to be processed, method 400 ends.

FIG. 6A illustrates a schematic diagram of an exemplary data move optimizing of merged calculation processes, consistent with embodiments of the present disclosure. As shown in FIG. 6A, calculation process 1 and calculation process 2 are merged sequentially by a merge operation. When being optimized, results of calculation process 1 and calculation process 2 are merged sequentially without performing a merge operation. Accordingly, using the optimization techniques consistent with the disclosed embodiments, the data-move task associated with the calculation processes is eliminated.

FIG. 6B illustrates a schematic diagram of data move optimizing of interleaved calculation processes, consistent with embodiments of the present disclosure. As shown in FIG. 6B, alternatively calculation process 1 and calculation process 2 are interleaved by an interleave operation. When being optimized, the results of calculation process 1 and calculation process 2 are interleaved without performing an interleave operation. The data-move task is eliminated.

It is appreciated that calculation process 1 and calculation process 2 as shown in FIG. 6A and FIG. 6B can run on different hardware (e.g., on two separate GPUs, or calculation process 1 on CPU while calculation process 2 on GPU) in the condition that the CPU and the GPU can access shared memory where the CPU and the GPU can both write results to address offset reserved based on the result sizes of calculation process 1 and calculation process 2. Calculation process 1 and calculation process 2 can also be executed by one accelerator such as a GPU. When calculation process 1 and calculation process 2 are executed on CPU or GPU, the CPU and the GPU are capable of optimizing the operations according to the embodiments of the present disclosure. When any process runs on FPGAs or NPUs, there may be a determination whether the hardware of FPGAs or NPUs are capable of performing such optimization, because ISA of the hardware may not support such optimization.

In some embodiments, a non-transitory computer-readable storage medium including instructions is also provided, and the instructions may be executed by an apparatus (such as a compiler on CPU 108 or NPU 102 in FIG. 1) , for performing the above-described methods. Common forms of non-transitory media include, for example, a floppy disk, a flexible disk, hard disk, solid state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM or any other flash memory, NVRAM, a cache, a register, any other memory chip or cartridge, and networked versions of the same. The device may include one or more processors (such as CPUs and processing accelerators) , an input/output interface, a network interface, and/or a memory.

It should be noted that, the relational terms herein such as “first” and “second” are used only to differentiate an entity or operation from another entity or operation, and do not require or imply any actual relationship or sequence between these entities or operations. Moreover, the words “comprising, ” “having, ” “containing, ” and “including, ” and other similar forms are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items.

As used herein, unless specifically stated otherwise, the term “or” encompasses all possible combinations, except where infeasible. For example, if it is stated that a component may include A or B, then, unless specifically stated otherwise or infeasible, the component may include A, or B, or A and B. As a second example, if it is stated that a component may include A, B, or C, then, unless specifically stated otherwise or infeasible, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

It is appreciated that the above described embodiments can be implemented by hardware, or software (program codes) , or a combination of hardware and software. If implemented by software, it may be stored in the above-described computer-readable media. The software, when executed by the processor can perform the disclosed methods. The computing units and other functional units described in this disclosure can be implemented by hardware, or software, or a combination of hardware and software. One of ordinary skill in the art will also understand that multiple ones of the above described modules/units may be combined as one module/unit, and each of the above described modules/units may be further divided into a plurality of sub-modules/sub-units.

In the foregoing specification, embodiments have been described with reference to numerous specific details that can vary from implementation to implementation. Certain adaptations and modifications of the described embodiments can be made. Other embodiments can be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims. It is also intended that the sequence of steps shown in figures are only for illustrative purposes and are not intended to be limited to any particular sequence of steps. As such, those skilled in the art can appreciate that these steps can be performed in a different order while implementing the same method.

In the drawings and specification, there have been disclosed exemplary embodiments. However, many variations and modifications can be made to these embodiments. Accordingly, although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

A method for optimizing data-move tasks associated with a machine learning model, comprising:

identifying a data-move task in a program executable in a device;

identifying one or more operations that are associated with the data-move task;

determining hardware capabilities of the device and requirements of optimizing data-move tasks; and

in response to a determination that the hardware capabilities of the device satisfy the requirements, mapping the identified data-move task to data writing of the identified one or more operations that are associated with the data-move task.
The method of claim 1, wherein determining the hardware capabilities of the device and the requirements of optimizing data-move tasks comprises

determining sizes of output operands of the identified one or more operations that are associated with the data-move task.
The method of claim 2, wherein the hardware capabilities satisfy the requirements when a minimum size that the device is configured to write is equal to or smaller than a minimum size of the output operands.
The method of any one of claim 2-3, wherein mapping the identified data-move task to writing data of operations that are associated with the data-move task comprises:

generating a function that maps an output offset of the identified data-move task in response to an input offset of the identified data-move task being given; and

redirecting the output offset of the identified data-move task using the function based on the sizes of the output operands.
The method of claim 4, wherein the function maps offset by one of modifying a page table of an operation system, constructing and maintaining a lookup table, calling a formula, or modifying how an offset changes.
The method of claim 1, further comprising:

providing a storage allocator prior to performing the identified one or more operations that are associated with the data-move task.
The method of claim 1, further comprising:

eliminating the data move task that has been mapped.
An apparatus for optimizing data-move tasks associated with a machine learning model, the apparatus comprising:

a memory storing a set of instructions; and

one or more processors configured to execute the set of instruction to cause the apparatus to perform:

identifying a data-move task in a program executable in a device;

identifying one or more operations that are associated with the data-move task;

determining hardware capabilities of the device and requirements of optimizing data-move tasks; and

in response to a determination that the hardware capabilities of the device satisfy the requirements, mapping the identified data-move task to data writing of the identified one or more operations that are associated with the data-move task.
The apparatus of claim 8, wherein determining the hardware capabilities of the device and the requirements of optimizing data-move tasks comprises

determining sizes of output operands of the identified one or more operations that are associated with the data-move task.
The apparatus of claim 9, wherein the hardware capabilities satisfy the requirements when a minimum size that the device is configured to write is equal to or smaller than a minimum size of the output operands.
The apparatus of any one of claim 9-10, wherein mapping the identified data-move task to writing data of operations that are associated with the data-move task comprises:

generating a function that maps an output offset of the identified data-move task in response to an input offset of the identified data-move task being given; and

redirecting the output offset of the identified data-move task using the function based on the sizes of the output operands.
The apparatus of claim 11, wherein the function maps offset by one of modifying a page table of an operation system, constructing and maintaining an lookup table, calling a formula, or modifying how an offset changes.
The apparatus of claim 8, wherein the one or more processors are configured to execute the set of instructions to cause the apparatus to further perform:

providing a storage allocator prior to performing the identified one or more operations that are associated with the data-move task.
A non-transitory computer readable medium that stores a set of instructions that is executable by at least one processor of a computer to cause the computer to perform a method for optimizing data-move tasks associated with a machine learning model, the method comprising:

identifying a data-move task in a program executable in a device;

identifying one or more operations that are associated with the data-move task;

determining hardware capabilities of the device and requirements of optimizing data-move tasks; and

in response to a determination that the hardware capabilities of the device satisfy the requirements, mapping the identified data-move task to data writing of the identified one or more operations that are associated with the data-move task.
The computer readable medium of claim 14, wherein determining the hardware capabilities of the device and the requirements of optimizing data-move tasks comprises

determining sizes of output operands of the identified one or more operations that are associated with the data-move task.
The computer readable medium of claim 15, wherein the hardware capabilities satisfy the requirements when a minimum size that the device is configured to write is equal to or smaller than a minimum size of the output operands.
The computer readable medium of any one of claim 15-16, wherein mapping the identified data-move task to writing data of operations that are associated with the data-move task comprises:

generating a function that maps an output offset of the identified data-move task in response to an input offset of the identified data-move task being given; and

redirecting the output offset of the identified data-move task using the function based on the sizes of the output operands.
The computer readable medium of claim 17, wherein the function maps offset by one of modifying a page table of an operation system, constructing and maintaining an lookup table, calling a formula, or modifying how an offset changes.
The computer readable medium of claim 14, wherein the set of instructions that is executable by at least one processor of the computing device to cause the computing device to further perform:

providing a storage allocator prior to performing the identified one or more operations that are associated with the data-move task.
The computer readable medium of claim 1, 4 wherein the set of instructions that is executable by at least one processor of the computing device to cause the computing device to further perform:

eliminating the data move task that has been mapped.