CN115167865A

CN115167865A - Algorithm pipeline arranging method and device, electronic equipment and storage medium

Info

Publication number: CN115167865A
Application number: CN202210706892.1A
Authority: CN
Inventors: 王璟; 陈铭涛
Original assignee: Alibaba China Co Ltd
Current assignee: Alibaba China Co Ltd
Priority date: 2022-06-21
Filing date: 2022-06-21
Publication date: 2022-10-11
Also published as: WO2023246801A1

Abstract

The application provides an algorithm pipeline arranging method, an algorithm pipeline arranging device, electronic equipment and a storage medium, and the technical scheme is as follows: acquiring a task description file, wherein the task description file comprises a plurality of module description information; calling a plurality of general modules in a module library for processing based on a plurality of module description information to obtain a plurality of application modules; and obtaining an algorithm pipeline corresponding to the task description file based on the plurality of application modules. According to the embodiment of the application, the workload of algorithm development can be reduced, and the speed of algorithm deployment can be accelerated.

Description

Algorithm pipeline arranging method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of artificial intelligence, and in particular, to a method and an apparatus for arranging an algorithm pipeline, an electronic device, and a storage medium.

Background

AI (Artificial Intelligence) technology based on DP (Deep Learning) algorithm has been rapidly developed in recent years. Enabling various IoT (Internet of Things) scenarios with AI has also become a popular application in recent years. From intelligent household products related to individuals to applications such as vehicle traffic perception, site safety helmet detection, life jacket detection and the like related to security production, AI is becoming an important means for liberating manpower and improving productivity.

In an IoT scenario, after an algorithm model is trained, hardware platforms running at various levels of different computational power need to be deployed according to application requirements. When the algorithm is deployed, different data processing modules need to be organized and connected in series to form an algorithm Pipeline (Pipeline). In the related art, for different types of algorithm tasks or algorithm tasks operated by different hardware platforms, an algorithm pipeline is generally obtained in a manual hard coding mode, and the problem of huge workload exists.

Disclosure of Invention

The embodiment of the application provides an algorithm pipeline arranging method, an algorithm pipeline arranging device, electronic equipment and a storage medium, so as to solve the problems in the related art, and the technical scheme is as follows:

in a first aspect, an embodiment of the present application provides an algorithm pipeline arrangement method, including:

acquiring a task description file; the task description file comprises a plurality of module description information;

calling a plurality of general modules in a module library for processing based on a plurality of module description information to obtain a plurality of application modules;

and obtaining an algorithm pipeline corresponding to the task description file based on the plurality of application modules.

In a second aspect, an embodiment of the present application provides an algorithmic pipelining apparatus, where the apparatus includes:

the application layer is used for acquiring the task description file; the task description file comprises a plurality of module description information;

and the framework layer is used for calling a plurality of general modules in the module library for processing based on the plurality of module description information to obtain a plurality of application modules and obtaining an algorithm production line corresponding to the task description file based on the plurality of application modules.

In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory, where the processor, when executing the computer program, implements the method provided in any embodiment of the present application.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements a method provided in any embodiment of the present application.

According to the technical scheme of the embodiment of the application, a plurality of general modules can be pre-configured in the module library, and the general modules are called by writing the task description file containing the description information of the modules so as to process the application modules suitable for specific algorithm tasks and obtain the algorithm production line. Therefore, for tasks on different types or different hardware platforms, an algorithm pipeline adaptive to the tasks can be obtained by multiplexing the above modes as long as different task description files are written. Namely, a modular mode is utilized, the development of low coding is achieved, the development workload is reduced, and the algorithm deployment speed is accelerated.

The foregoing summary is provided for the purpose of description only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the present application will be readily apparent by reference to the drawings and following detailed description.

Drawings

In the drawings, like reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily to scale. It is appreciated that these drawings depict only some embodiments in accordance with the disclosure and are not to be considered limiting of its scope.

FIG. 1 is a schematic diagram of an exemplary application scenario in accordance with an embodiment of the present application;

FIG. 2 is a flowchart of an algorithm pipeline arrangement method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an algorithm pipelining framework in one embodiment of the present application;

FIG. 4 is a visualization rendering of an algorithm pipeline in an embodiment of the present application;

FIG. 5 is a block diagram of an algorithm pipeline orchestration device according to an embodiment of the present application;

fig. 6 is a block diagram of an electronic device for implementing an algorithmic pipelining method according to an embodiment of the present application.

Detailed Description

In the following, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present application. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.

To more clearly show the algorithm pipeline arrangement method provided in the embodiment of the present application, an application scenario that can be used to implement the method is first introduced. Fig. 1 shows a schematic diagram of an exemplary application scenario of an embodiment of the present application. The scenario is an IoT scenario. In an IoT scenario, an algorithm pipeline needs to be generated for an algorithm task, and hardware platforms applying the algorithm pipeline are diverse. As shown in fig. 1, the algorithm pipeline in the embodiment of the present application may be used in hardware platforms such as a server, a computing all-in-one machine, a mobile terminal, and a camera. The computing power of these platforms can vary widely, for example, from edge servers carrying high-computing-power GPUs (Graphics Processing units), to common mobile terminals, to security cameras with extremely limited resources. The hardware architecture is also widely varied, for example, from a general GPU, a CPU (Central Processing Unit), to a heterogeneous system on chip (soc) that simultaneously mounts a CPU, a DPS (Digital Processing Unit), and an NPU (Neural-network Processing Unit).

For example, the algorithm in the embodiment of the present application may refer to an AI algorithm, and accordingly, the algorithm task in the embodiment of the present application may include a computing task implemented based on an AI technology, for example, a computer vision task including a vehicle detection task, an area intrusion detection task, and the like.

In the embodiment of the present application, the algorithm pipeline arranging device is configured to arrange an algorithm pipeline (i.e., pipeline) corresponding to each algorithm task, that is, to implement the algorithm pipeline arranging method provided in the embodiment of the present application. Illustratively, as shown in fig. 1, a developer may write a task description file according to task attributes, and input the task description file to an algorithmic pipelining apparatus, which organizes an algorithmic pipeline based on the task description file, the algorithmic pipeline being applicable to the hardware platform. The task attribute includes a task type, a hardware platform type, and the like. For example, a developer may write a task description file for a license plate recognition task that needs to run on a camera, or for a vehicular traffic awareness task that may run on an edge server, or for a facial recognition task that may run on any platform, etc., to orchestrate a corresponding algorithm pipeline.

For example, the algorithm pipelining device may provide the algorithm pipelining method in the form of a computing framework, that is, the computing framework may be constructed in advance, and the computing framework integrates a plurality of universal and reusable algorithm modules; in practical application, the algorithm modules in the computing framework are instantiated and arranged according to the task attributes of specific algorithm tasks, and an algorithm pipeline which can be transplanted to a specific hardware platform to run is obtained. Namely, a modularization mode is utilized, the development of low coding is achieved, the development workload is reduced, and the algorithm deployment speed is accelerated.

Optionally, the computing framework may also integrate underlying operators of various hardware platforms; in practical application, the corresponding bottom layer operation operator is called for the hardware platform corresponding to the algorithm task, so that the hardware platform can execute the algorithm task according to the algorithm pipeline. Optionally, the computing framework may also include an execution runtime that integrates different inference engines for performing data manipulation of the various algorithmic modules.

So that the manner in which the features and elements of the present embodiments can be understood in detail, a more particular description of the embodiments, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings.

Fig. 2 shows a flowchart of an algorithmic pipelining method according to an embodiment of the present application. As shown in fig. 2, the method may include:

s210, acquiring a task description file; the task description file comprises a plurality of module description information;

s220, calling a plurality of general modules in a module library for processing based on the plurality of module description information to obtain a plurality of application modules;

and S230, obtaining an algorithm pipeline corresponding to the task description file based on the plurality of application modules.

In the embodiment of the application, the task description file can be used for describing the algorithm task. Illustratively, the task description file may include identification information of the algorithmic task, such as the name, category, etc. of the algorithmic task. The task description file may further include a plurality of module description information respectively corresponding to a plurality of data processing modules (or data processing links) of the algorithm task, where each module description information is used to describe a corresponding data processing module. For example, each module description information may include a module name, a module category, a module type, a module parameter, input-output information, and the like.

In practical applications, the algorithmic tasks described in different task description files may include different data processing modules. Illustratively, the data processing module included in the algorithmic task may be set based on task attributes of the algorithmic task, such as a task type of the algorithmic task, a type of a hardware platform executing the algorithmic task, and the like. For example, the facial recognition tasks deployed on the edge server may include video decoding, video framing, color space conversion, face detection, face tracking, face quality scoring, face preference filtering, facial feature extraction, and facial recognition, among other data processing modules. The facial recognition tasks deployed on the panel machine device may include data processing modules such as color space conversion, face detection, face quality scoring, and facial feature extraction.

Since the data processing modules in the algorithmic task are determined based on task attributes, the task description file may be associated with task attributes. Specifically, the developer may determine task description information and module description information in the task description file based on the task attributes, including a module category, a module type, a module parameter, input and output information, and the like. For example, a plurality of data processing modules may be determined based on task attributes, and a task description file may be written based on the plurality of data processing modules and preset rules. Here, the preset rule may include preset writing requirements such as a data format, field information to be filled in, and the like. Based on the method, the related information of each data processing module can be analyzed from the task description file based on the preset rule.

In practical application, different algorithm tasks can be summarized, a series of general data processing modules are disassembled and abstracted from the different algorithm tasks, and the general data processing modules are stored in a module library. For example, computer vision tasks generally include subtasks such as image processing, video processing, model inference, application data processing, and the like, where the image processing may include various data processing modules such as color space conversion, image compression, and the like, the video processing may include various data processing modules such as video decoding, video framing, and the like, and the model inference may include various data processing modules such as target detection, classification, regression, multi-target tracking, feature extraction, recognition, and the like.

In different algorithm tasks, the processing flow of the same data processing module on data is the same, for example, the same function/operator is included, but there may also be differences in configuration, for example, different parameters, different input and output information, and the like. For example, in the face detection task and the vehicle detection task, the processing flow of the image processing module is the same, but detection frames outputting different sizes may be set according to actual needs. In embodiments of the present application, the generic module may comprise an uninstantiated data processing module determined based on the module class and type, e.g. a data processing module containing the determined function/operator but not configured with the relevant parameters.

Optionally, each module description information in the task description information may include information such as a module type, and a parameter, so that the general module may be called in the module library based on the information such as the module type, and the like, and the general module may be instantiated based on configuration information such as the parameter, and the like, that is, the general module is configured with attributes such as the parameters, and the application module is obtained. That is to say, the application module in the embodiment of the present application may include an instantiated module obtained by configuring a general-purpose module, for example, a data processing module that includes the determined function/operator and is configured with parameters. Because the application modules are instantiation modules, the application modules are organized and connected in series, and an algorithm pipeline can be obtained.

Therefore, according to the method provided by the embodiment of the application, a plurality of general modules are pre-configured in the module library, and the task description file containing a plurality of module description information is compiled to call the general modules, so that the application modules suitable for specific algorithm tasks are obtained through processing, and an algorithm production line is obtained. Therefore, for tasks on different types or different hardware platforms, an algorithm pipeline adaptive to the tasks can be obtained by multiplexing the above modes as long as different task description files are written. The method has the characteristics of low coding, modularization and the like.

Optionally, in an exemplary embodiment, in step S220, based on a plurality of module description information, a plurality of general modules are called in a module library for processing, so as to obtain a plurality of application modules, including:

determining a plurality of general modules in a module library based on the module types contained in each module description information in the plurality of module description information;

and instantiating the universal modules based on the configuration information contained in the module description information to obtain the application modules.

That is, each module description information includes a module type, configuration information, and the like. Wherein the module type may be used to determine a generic module in a module library; the configuration information may be used to instantiate a generic module.

In the embodiment of the present application, the module type is the type of the data processing module described by the module description information, such as video decoding, video frame extraction, color space conversion, face detection, face tracking, and the like. Optionally, the module description information may also include the class of data processing modules described, such as image processing, video processing, model reasoning, application data processing, etc. Accordingly, the generic module may be called in the module library based on the module type and category in the module description information, for example, according to the video processing category and the video decoding type, the corresponding video decoding generic module may be called in the module library accurately.

The configuration information may comprise algorithm parameters, i.e. one or more parameters of the data processing module described by the module description information. For example, for an object detection module, the algorithm parameters may include a detection box length, a detection box width, and the like. For batch modules, the algorithm parameters may include batch size.

Optionally, in an exemplary embodiment, the step S230 of obtaining, based on a plurality of application modules, an algorithm pipeline corresponding to the task description file includes:

obtaining an arrangement order of the plurality of application modules based on the order information of the plurality of module description information;

determining the data type among the plurality of application modules based on the input and output information contained in each module description information in the plurality of module description information;

and obtaining an algorithm pipeline based on the arrangement order of the application modules and the data types among the application modules.

That is, the task description file further includes order information of the plurality of module description information for determining the arrangement order of the corresponding plurality of application modules. Alternatively, the order information may include indication information of whether or not parallel between the data processing modules, serial order, and the like. Illustratively, the order between the modules may be explicitly described by data in the task description file, or the module description information may be sequentially written in the task description file according to the order between the modules, so that the order information between the module description information is implicit in the task description file.

According to the above mode, each module description information also includes input and output information for determining the data type between modules, such as image, image set, numerical value, etc. Therefore, based on the information, a plurality of application modules can be accurately arranged to obtain an algorithm production line.

For example, in an embodiment of the present application, the task description file may include a JSON (JSON Object Notation) file, where the JSON file includes task description information input according to a preset field layout structure and a plurality of module description information. The JSON file is a lightweight data exchange format and is easy to analyze and generate by a machine. The field arrangement structure in the JSON file, such as writing positions of the task description information and the module description information, types of fields included, and the like, may be defined in advance, and then the task description information and the module description information are input based on the field arrangement structure, so that the algorithm pipeline arranging apparatus can accurately parse the task description information and the module description information from the JSON file based on the field arrangement structure.

Optionally, in an exemplary embodiment, the method further includes:

and adding corresponding universal modules in the module library according to the universal module registration information input by the user.

That is to say, the embodiment of the application supports a user to register a self-defined module in a module library, and by registering the self-defined module in the module library, an algorithm pipeline which can be arranged by a device can be extended, that is, an application range based on the method is extended without changing a core code for implementing the method.

Optionally, in an exemplary embodiment, the method further includes:

responding to a first hardware platform execution algorithm pipeline, and calling a bottom layer operation operator corresponding to the hardware type from a plurality of bottom layer operation operators based on the hardware type of the first hardware platform;

providing a bottom layer operation operator for the first hardware platform through a preset interface; the preset interface is used for being called by hardware platforms corresponding to a plurality of hardware types.

The first hardware platform is a hardware platform for executing an algorithm pipeline. The first hardware platform may be a hardware platform associated with a task description file, for example, written for an algorithmic task running on the first hardware platform. In some scenarios, the first hardware platform may also be an arbitrary hardware platform, for example, the algorithm task corresponding to the task description file may be implemented on an arbitrary hardware platform, and then the adaptation of the bottom-layer operator is performed according to the hardware platform that actually executes the algorithm task.

In the above exemplary embodiment, the hardware type may refer to the hardware architecture of the hardware platform, including information of processor type, model, and the like. According to the mode, the algorithm pipeline arranging device defines a preset interface as a common processing interface of different hardware platforms. The device can call the corresponding bottom layer operation operator based on the hardware type, and does not need each hardware platform to realize task operation through a special interface, namely, the uniform hardware programming interface shields the realization detail difference of different hardware data calculation, thereby providing the algorithm pipeline arrangement capability with high efficiency, flexibility and multi-platform hardware acceleration.

In order to more clearly present the technical idea of the present application, a specific application example is provided below. In the present application example, the above method is implemented in the form of a computing framework (hereinafter, pipeline framework). FIG. 3 shows a schematic diagram of an algorithmic pipelining framework. As shown in fig. 3, the frame includes: a pipeline Layer, a Layer (data processing link) Layer, a framework Layer, a hardware abstraction Layer and a hardware platform Layer.

The pipeline layer is the application layer of the framework. For different application scenes, pipelines suitable for the scenes are often connected in series according to processing links, such as face detection pipelines, vehicle detection pipelines, regional intrusion detection pipelines and the like. The framework can organize the algorithm pipeline through the JSON file, for landing of the algorithm, a developer only needs to compile the JSON file according to rules, the pipeline framework can conduct operation editing through JSON file description, each flow of the algorithm is executed, and a final output result is obtained. The JSON is used for describing algorithm pipeline, so that low-code development of algorithm engineering can be realized, and landing iteration of the algorithm is accelerated.

The Layer is a library of data processing modules of the pipeline framework. Layer, a basic component unit of algorithm pipeline, is a node (link) for data processing. Different layers are connected in series to form algorithm pipeline with different functions. The Layer component Layer mainly comprises an image processing related Layer, a video processing related Layer, a model reasoning related Layer, an application data processing Layer and a custom Layer at present. The model inference Layer mainly realizes general inference task modules of computer vision scenes, such as target detection, classification, regression, multi-target tracking, feature extraction, identification and the like, and the modules are realized as a general template and are not bound with specific algorithm scenes. The difference between the input and output data of Layer (meaning difference) is different, and the data processing method of Layer has universality. Therefore, in practical use, a Layer can be flexibly applied to different tasks, such as face detection/recognition and vehicle/license plate detection and recognition. In the implementation, the principle of decoupling Layer implementation and hardware details is followed, the code of the Layer does not directly expose the hardware interface details of different platforms, but the hardware difference is shielded through a set of uniform hardware abstraction Layer, so that the Layer module of the Layer has more universality, and algorithm transplanters of different hardware platforms are allowed to share the Layer module of the Layer.

The framework layer contains the core components of the pipeline framework, including the registry, schema parser, execution runtime, and data manager. The registry is used for registering the custom Layer, and if the existing Layer implementation cannot cover some special data processing operations of the algorithm flow, the registry mechanism allows a user to flexibly implement and register the custom Layer without changing the core code of the framework. The diagram analyzer is used for analyzing the JSON file, identifying the Layer configuration in the JSON file and instantiating the Layer, so that an execution diagram of the algorithm pipeline is constructed. And the execution runtime is responsible for executing data operations of different nodes according to different node sequences of the pipeline execution graph, and currently supports integration of different inference engines. And during execution, the multithreading concurrency of the Layer level and the pipeline level can be flexibly carried out, and the parallel capability of different hardware is more fully utilized. The data manager is used for managing input data structures and output data structures of the Layers, the input and the output among different Layers are deconstructed through the data structures, consumption is carried out only when data exist in an input queue of the Layers, and result data are produced in the output queue structure after calculation and are consumed by downstream Layers. Thus, the Layer does not need to explicitly process complex data dependence, and complexity is reduced. The data manager is also responsible for tracking the life cycle of the intermediate data of different links of pipeline and releasing the data.

The Hardware Abstraction Layer (HAL) is a Hardware programming interface with a unified framework, which defines a common processing interface of different Hardware and is used for shielding implementation detail differences of data calculation of different Hardware. According to the running hardware during execution running of pipeline, when Layer operation is running, a bottom Layer operation operator corresponding to the current hardware platform is selected for execution, for example, a corresponding image operator interface, a video operator interface, an inference engine interface and the like are selected. When a developer needs to interface a new hardware platform, the related operator implementation of the HAL on the new platform needs to be added again according to the HAL interface specification.

The hardware platform layer represents a hardware acceleration platform currently supported by a framework, and mainly comprises a platform based on hardware architectures such as a CPU (central processing unit), a GPU (graphic processing unit), a DSP (digital signal processor) and the like, a coverage edge server, a computer all-in-one machine, a consumption-level mobile terminal, a security camera and the like.

To more clearly describe the characteristics of modularity and low coding of the method of the embodiment of the present application, an example of a JSON file of a face target detection task is provided below to show the processing procedure of pipeline. Wherein, the information in the JSON file is exemplified as follows:

the JSON file comprises task description information such as the name and the version of the algorithm task and also comprises module description information. In the above example, module description information is presented by taking jpg (Joint Photographic Group) image decoding as an example, and in practical applications, description information of a plurality of other modules may also be included in the JSON file. In each module description information, a name, a category, a type, input/output information, and the like of the module may be included. For example, the module description information of the jpg image decoding module indicates that the name of the module is jpg image decoding, the category is video processing, the type is jpg image decoding, the input information is jpg image batch data, and the output information is bgr (blue-green-red) batch data. Therefore, in the pipeline layout JSON file, a user can select different Layer modules according to needs and set corresponding parameters for the Layer modules. Different layers are finally combined into the whole algorithm flow, and fig. 4 shows a visual rendering of pipeline according to the JSON file layout in the example. When the method runs, a graph analyzer of the pipeline framework can analyze the JSON file, instantiates the pipeline to execute, and obtains an operation result.

Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application further provides an algorithm pipeline arrangement apparatus 500. Referring to fig. 5, the apparatus 500 may include:

an application layer 510, configured to obtain a task description file; the task description file comprises a plurality of module description information;

and the framework layer 520 is used for calling a plurality of general modules in the module library for processing based on the plurality of module description information to obtain a plurality of application modules, and obtaining an algorithm pipeline corresponding to the task description file based on the plurality of application modules.

The apparatus 500 may further include a module library.

Illustratively, the apparatus 500 may further include:

the hardware abstraction layer is used for responding to the first hardware platform execution algorithm pipeline, calling a bottom layer operator corresponding to the hardware type from the bottom layer operators based on the hardware type of the first hardware platform, and providing the bottom layer operator for the first hardware platform through a preset interface; the preset interface is used for being called by hardware platforms corresponding to a plurality of hardware types.

Illustratively, the framework layer may include a schema parser for determining a plurality of generic modules in the module library based on a module type included in each of the plurality of module description information; and instantiating the universal modules based on the configuration information contained in the module description information to obtain the application modules.

Illustratively, the graph parser in the framework layer may also be used to:

the framework layer may further include a data manager configured to determine a data type between the plurality of application modules based on input and output information included in each of the plurality of module description information, so that the framework layer obtains an algorithm pipeline based on an arrangement order of the plurality of application modules and the data type between the plurality of application modules.

Illustratively, the framework layer may further include a register for adding the corresponding generic module in the module library according to the generic module registration information input by the user.

The framework layer may further include an execution runtime for executing the algorithm pipeline.

Illustratively, the data manager in the framework layer is also used for tracking the life cycle of the intermediate data of each application module in the algorithm pipeline and performing the release work of the data.

Illustratively, the task description file includes a JSON file including task description information and a plurality of module description information input according to a preset field arrangement structure.

The functions of each module in each device in the embodiment of the present application can be referred to the corresponding description in the above method, and have corresponding beneficial effects, which are not described herein again.

The embodiment of the application also provides electronic equipment for realizing the method. Fig. 6 shows a block diagram of an electronic device according to an embodiment of the present application. As shown in fig. 6, the electronic apparatus includes: a memory 610 and a processor 620, the memory 610 having stored therein computer programs executable on the processor 620. The processor 620, when executing the computer program, implements the algorithm pipelining method in the above embodiments. The number of the memory 610 and the processor 620 may be one or more.

The electronic device further includes:

the communication interface 630 is used for communicating with an external device to perform data interactive transmission.

If the memory 610, the processor 620 and the communication interface 630 are implemented independently, the memory 610, the processor 620 and the communication interface 630 may be connected to each other through a bus and perform communication with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 6, but that does not indicate only one bus or one type of bus.

Optionally, in an implementation, if the memory 610, the processor 620, and the communication interface 630 are integrated on a chip, the memory 610, the processor 620, and the communication interface 630 may complete communication with each other through an internal interface.

Embodiments of the present application further provide a computer-readable storage medium, which stores a computer program, and when the program is executed by a processor, the computer program implements the method provided in any embodiment of the present application.

Embodiments of the present application also provide a computer program product comprising a computer program, which when executed by a processor implements the method provided in any of the embodiments of the present application.

The embodiment of the present application further provides a chip, where the chip includes a processor, and is configured to call and execute the instruction stored in the memory from the memory, so that the communication device in which the chip is installed executes the method provided in the embodiment of the present application.

An embodiment of the present application further provides a chip, including: the system comprises an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected through an internal connection path, the processor is used for executing codes in the memory, and when the codes are executed, the processor is used for executing the method provided by the embodiment of the application.

It should be understood that the processor may be a CPU, and may be other general purpose processors, digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or any conventional processor or the like. It is noted that the processor may be a processor supporting an Advanced reduced instruction set machine (ARM) architecture.

Further, optionally, the memory may include a read-only memory and a random access memory, and may further include a nonvolatile random access memory. The memory may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile Memory may include a Read-only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. Volatile Memory can include Random Access Memory (RAM), which acts as external cache Memory. By way of example, and not limitation, many forms of RAM are available. For example, static Random Access Memory (Static RAM, SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), enhanced SDRAM (ESDRAM), SLDRAM (synclink DRAM), and Direct Memory bus RAM (DR RAM).

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions according to the present application are generated in whole or in part when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium.

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.

Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.

Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process. And the scope of the preferred embodiments of the present application includes other implementations in which functions may be performed out of the order shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved.

The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.

It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the method of the above embodiments may be implemented by hardware that is configured to be instructed to perform the relevant steps by a program, which may be stored in a computer-readable storage medium, and which, when executed, includes one or a combination of the steps of the method embodiments.

In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module may also be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. The storage medium may be a read-only memory, a magnetic or optical disk, or the like.

While the present invention has been described with reference to the preferred embodiments, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An algorithm pipeline orchestration method comprising:

calling a plurality of general modules in a module library for processing based on the plurality of module description information to obtain a plurality of application modules;

and obtaining an algorithm production line corresponding to the task description file based on the plurality of application modules.

2. The method of claim 1, wherein the invoking a plurality of generic modules in a module library for processing based on the plurality of module description information to obtain a plurality of application modules comprises:

determining the plurality of general modules in the module library based on the module types contained in each of the plurality of module description information;

3. The method of claim 1 or 2, wherein the deriving an algorithm pipeline corresponding to the task description file based on the plurality of application modules comprises:

and obtaining the algorithm pipeline based on the arrangement sequence of the application modules and the data types among the application modules.

4. The method of claim 1 or 2, wherein the task description file comprises a JSON file comprising the plurality of module description information and task description information input according to a preset field arrangement structure.

5. The method according to claim 1 or 2, wherein the method further comprises:

and adding the corresponding universal module in the module library according to the universal module registration information input by the user.

6. The method according to claim 1 or 2, wherein the method further comprises:

responding to a first hardware platform to execute the algorithm pipeline, and calling a bottom layer operation operator corresponding to the hardware type from a plurality of bottom layer operation operators based on the hardware type of the first hardware platform;

providing the bottom layer operation operator for the first hardware platform through a preset interface; the preset interface is used for being called by hardware platforms corresponding to a plurality of hardware types.

7. An algorithmic pipelining apparatus, wherein the apparatus comprises:

and the framework layer is used for calling a plurality of general modules in the module library for processing based on the plurality of module description information to obtain a plurality of application modules, and obtaining an algorithm pipeline corresponding to the task description file based on the plurality of application modules.

8. The apparatus of claim 7, wherein the apparatus further comprises:

the hardware abstraction layer is used for responding to a first hardware platform to execute the algorithm pipeline, calling a bottom layer operator corresponding to the hardware type from a plurality of bottom layer operators based on the hardware type of the first hardware platform, and providing the bottom layer operator for the first hardware platform through a preset interface; the preset interface is used for being called by hardware platforms corresponding to a plurality of hardware types.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory, the processor implementing the method of any one of claims 1-6 when executing the computer program.

10. A computer-readable storage medium, having stored therein a computer program which, when executed by a processor, implements the method of any of claims 1-6.