CN117193964A

CN117193964A - Task processing method and automatic question-answering method

Info

Publication number: CN117193964A
Application number: CN202310979498.XA
Authority: CN
Inventors: 周文猛; 苏璐岩; 张芃
Original assignee: Alibaba China Co Ltd
Current assignee: Alibaba China Co Ltd
Priority date: 2023-08-04
Filing date: 2023-08-04
Publication date: 2023-12-08

Abstract

The embodiment of the specification provides a task processing method and an automatic question-answering method, wherein the task processing method is applied to a target processing unit in a task processing platform, the task processing platform comprises a first storage unit, a second storage unit and a target processing unit, and the method comprises the following steps: receiving a task processing request, wherein the task processing request carries a task execution configuration file; obtaining a target task processing model corresponding to a task processing request from a first storage unit, wherein the first storage unit comprises a plurality of task processing models stored in advance; analyzing the task execution configuration file by utilizing the analysis file stored in the second storage unit, and determining the operation architecture of the target task processing model; and calling a target task processing model based on the operation framework to process the task processing request, and obtaining a task processing result. The analysis file and the model are stored separately, a user can develop a task processing model without perceiving a parallel architecture of the bottom model, and task processing complexity is simplified.

Description

Task processing method and automatic question-answering method

Technical Field

The embodiment of the specification relates to the technical field of computers, in particular to a task processing method. One or more embodiments of the present specification relate to an automatic question-answering method, a task processing device, an automatic question-answering device, a computing device, a computer-readable storage medium, and a computer program.

Background

With the rapid iterative development of large models, the demands of systemization for the engineering of algorithm infrastructures also manifest themselves as urgency and necessity. Because a single hardware device has bottlenecks in computing performance and running memory, a large model inevitably moves to the development direction of multi-card, multi-machine and cluster scale.

At present, based on the large-scale training of multi-machine multi-process concurrency, the parallel architecture is often seriously coupled with the algorithm implementation, an algorithm development engineer and an engineering architecture design engineer need to pay extra time cost to perform cross-domain code learning, and systematic conflict possibility exists between works, so that the complexity of a task processing process is rapidly increased, and therefore, a task processing scheme with low complexity is needed.

Disclosure of Invention

In view of this, the present embodiment provides a task processing method. One or more embodiments of the present specification relate to an automatic question-answering method, a task processing device, an automatic question-answering device, a computing device, a computer-readable storage medium, and a computer program to solve the technical drawbacks of the related art.

According to a first aspect of embodiments of the present disclosure, there is provided a task processing method applied to a target processing unit in a task processing platform, where the task processing platform includes a first storage unit, a second storage unit, and a target processing unit, the method including:

receiving a task processing request, wherein the task processing request carries a task execution configuration file;

obtaining a target task processing model corresponding to a task processing request from a first storage unit, wherein the first storage unit comprises a plurality of task processing models stored in advance;

analyzing the task execution configuration file by utilizing the analysis file stored in the second storage unit, and determining the operation architecture of the target task processing model;

and calling a target task processing model based on the operation framework to process the task processing request, and obtaining a task processing result.

According to a second aspect of embodiments of the present disclosure, there is provided an automatic question-answering method applied to a target processing unit in a task processing platform, where the task processing platform includes a first storage unit, a second storage unit, and a target processing unit, the method including:

receiving a problem processing request, wherein the problem processing request carries a problem processing configuration file;

Obtaining a target task processing model corresponding to a problem processing request from a first storage unit, wherein the first storage unit comprises a plurality of task processing models stored in advance;

analyzing the problem processing configuration file by utilizing the analysis file stored in the second storage unit, and determining the operation architecture of the target task processing model;

and calling a target task processing model based on the operation framework to process the problem processing request, and obtaining a problem reply result.

According to a third aspect of embodiments of the present specification, there is provided a task processing device applied to a target processing unit in a task processing platform, the task processing platform including a first storage unit, a second storage unit, and the target processing unit, the device including:

the first receiving module is configured to receive a task processing request, wherein the task processing request carries a task execution configuration file;

the first acquisition module is configured to acquire a target task processing model corresponding to a task processing request from a first storage unit, wherein the first storage unit comprises a plurality of task processing models stored in advance;

the first analysis module is configured to analyze the task execution configuration file by utilizing the analysis file stored in the second storage unit and determine the operation architecture of the target task processing model;

The first processing module is configured to call a target task processing model based on the operation framework to process the task processing request, and a task processing result is obtained.

According to a fourth aspect of embodiments of the present disclosure, there is provided an automatic question-answering apparatus applied to a target processing unit in a task processing platform, the task processing platform including a first storage unit, a second storage unit, and a target processing unit, the apparatus including:

the second receiving module is configured to receive a problem processing request, wherein the problem processing request carries a problem processing configuration file;

the second acquisition module is configured to acquire a target task processing model corresponding to the problem processing request from a first storage unit, wherein the first storage unit comprises a plurality of task processing models stored in advance;

the second analysis module is configured to analyze the problem processing configuration file by utilizing the analysis file stored in the second storage unit and determine the operation architecture of the target task processing model;

and the second processing module is configured to call the target task processing model based on the operation framework to process the problem processing request and obtain a problem reply result.

According to a fifth aspect of embodiments of the present specification, there is provided a computing device comprising:

A memory and a processor;

the memory is configured to store computer executable instructions that, when executed by the processor, implement the steps of the methods provided in the first or second aspects above.

According to a sixth aspect of embodiments of the present specification, there is provided a computer readable storage medium storing computer executable instructions which when executed by a processor implement the steps of the method provided in the first or second aspect above.

According to a seventh aspect of embodiments of the present specification, there is provided a computer program, wherein the computer program, when executed in a computer, causes the computer to perform the steps of the method provided in the first or second aspect described above.

The task processing method provided by the embodiment of the specification is applied to a target processing unit in a task processing platform, and the task processing platform comprises a first storage unit, a second storage unit and the target processing unit. Receiving a task processing request, wherein the task processing request carries a task execution configuration file; obtaining a target task processing model corresponding to a task processing request from a first storage unit, wherein the first storage unit comprises a plurality of task processing models stored in advance; analyzing the task execution configuration file by utilizing the analysis file stored in the second storage unit, and determining the operation architecture of the target task processing model; and calling a target task processing model based on the operation framework to process the task processing request, and obtaining a task processing result. By decoupling the engineering framework code and the algorithm model code, the analysis file and the model file are respectively stored in different storage units, so that a user can develop a task processing model without perceiving a parallel architecture of a bottom model, and task processing complexity is simplified.

Drawings

FIG. 1 is an architecture diagram of a task processing system provided in one embodiment of the present description;

FIG. 2 is an architecture diagram of another task processing system provided by one embodiment of the present description;

FIG. 3 is a flow chart of a method of task processing provided in one embodiment of the present disclosure;

FIG. 4 is a schematic version of a target processing unit in a task processing method according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of a configuration file reading path in a task processing method according to an embodiment of the present disclosure;

FIG. 6 is a flow chart of an automatic question-answering method provided by one embodiment of the present disclosure;

FIG. 7 is a block diagram of a task processing system provided in one embodiment of the present disclosure;

FIG. 8 is an interface schematic diagram of an automated question-answering interface provided by one embodiment of the present disclosure;

FIG. 9 is a schematic diagram of a task processing device according to one embodiment of the present disclosure;

fig. 10 is a schematic structural diagram of an automatic question answering device according to one embodiment of the present disclosure;

FIG. 11 is a block diagram of a computing device provided in one embodiment of the present description.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. This description may be embodied in many other forms than described herein and similarly generalized by those skilled in the art to whom this disclosure pertains without departing from the spirit of the disclosure and, therefore, this disclosure is not limited by the specific implementations disclosed below.

The terminology used in the one or more embodiments of the specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of the specification. As used in this specification, one or more embodiments and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any or all possible combinations of one or more of the associated listed items.

It should be understood that, although the terms first, second, etc. may be used in one or more embodiments of this specification to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first may also be referred to as a second, and similarly, a second may also be referred to as a first, without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.

Furthermore, it should be noted that, user information (including, but not limited to, user equipment information, user personal information, etc.) and data (including, but not limited to, data for analysis, stored data, presented data, etc.) according to one or more embodiments of the present disclosure are information and data authorized by a user or sufficiently authorized by each party, and the collection, use, and processing of relevant data is required to comply with relevant laws and regulations and standards of relevant countries and regions, and is provided with corresponding operation entries for the user to select authorization or denial.

In one or more embodiments of the present description, a large model refers to a deep learning model with large scale model parameters, typically including hundreds of millions, billions, trillions, and even more than one billion model parameters. The large Model can be called as a Foundation Model, a training Model is performed by using a large-scale unlabeled corpus, a pre-training Model with more than one hundred million parameters is produced, the Model can adapt to a wide downstream task, and the Model has better generalization capability, such as a large-scale language Model (LLM, large Language Model), a multi-Model pre-training Model and the like.

It should be noted that, the main characteristics of the large model include the parameter number and the depth, for example, the large model with the trillion parameter number may have more than one hundred layers/several hundred layers, and for the complex generation problem, the increase of the depth can obtain more benefits than the increase of the single-layer width, so that the depth of the large model has close relation with the parallel pipeline.

When the large model is actually applied, the pretrained model can be applied to different tasks by fine tuning with a small amount of samples, the large model can be widely applied to the fields of natural language processing (NLP, natural Language Processing), computer vision and the like, and particularly can be applied to the tasks of the computer vision fields such as vision question and answer (VQA, visual Question Answering), image description (IC), image generation and the like, and the tasks of the natural language processing fields such as emotion classification based on texts, text abstract generation, machine translation and the like, and main application scenes of the large model comprise digital assistants, intelligent robots, searching, online education, office software, electronic commerce, intelligent design and the like.

First, terms related to one or more embodiments of the present specification will be explained.

Large-scale language model: large-scale language models refer to neural network models with billions or even trillions of parameters that can be used for natural language processing tasks such as language generation, question-answering, and text classification. These models are typically trained using large amounts of text data and can perform well in a variety of different tasks.

Parallel training: the model training mode of the large model running on a plurality of threads in a multi-machine multi-card environment and performing synchronous iteration is called parallel training.

With the rapid iterative development of large-scale language models based on the structure of a Generative Pre-training model (GPT), the requirements for engineering and systemizing the algorithm infrastructure are also shown as urgency and necessity. Because a single hardware device has bottlenecks in computing performance and running memory, a large model inevitably moves to the development direction of multi-card, multi-machine and cluster scale.

At present, parallel strategies for large model training are mainly divided into three types: data parallelism, tensor parallelism, and pipeline parallelism. For the three parallel modes, the mainstream implementation is global training management based on a large model training framework (megatron), and in order to keep consistency of training and reasoning, the large model training framework is used as a bottom engineering foundation for model slicing and synchronization, multi-process management and the like in the reasoning stage. Code replacement by model training technology (megatron-LM) based on a large model training framework is a low-cost development scheme for algorithm development engineers, but this approach lacks engineering layer abstraction, implying the risk of rapid complexity rise, and because parallel architecture is often severely coupled with algorithm implementation, the algorithm development engineers and engineering architecture design engineers need to pay additional time cost to perform code learning across fields, and there is a possibility of systematic conflict between works. How to separate the algorithm complexity from the engineering complexity, so that the task processing complexity is reduced is a real problem faced in the actual development process.

In practical application, global training and reasoning codes can be taken over by designing an application programming interface, and in order to realize unified abstraction of a distributed training strategy, the practical parallel strategy can be realized and lowered to a fine-granularity operator layer by a Global View (Global View), so that distributed realization details are shielded at a higher algorithm layer, and meanwhile, a certain performance benefit can be obtained for the finer-granularity parallel strategy.

However, in the above scheme, all codes need to be taken over due to the frame positioning, so that development and debugging costs of the migration codes are high for users, and some existing codes cannot be reused and need to be re-developed. Meanwhile, as certain flexibility is required to be reserved, the scheme reserves the position attribute (Placement) for setting the position and the size of the object positioning layer, supports the hardware environment for the actual use of the user-defined model structure, ensures that an algorithm development engineer still needs to grasp the bottom-layer parallel details clearly, and further, the algorithm development engineer needs additional learning cost to grasp the bottom-layer parallel logic. In addition, the design of the scheme for lowering the parallel strategy to the operator layer actually requires to additionally develop a plurality of parallel strategies for the operator structure, and simultaneously is compatible with automatic deduction based on a global view, so that the parallel development is not friendly to external users, and only professional developers in the field can newly add the parallel strategy according to own requirements, so that a use threshold is naturally formed.

In order to solve the above problems, the embodiments of the present disclosure propose a task processing platform focusing on the engineering design and implementation of a model unifying frame based on a large model training frame, so the task processing platform proposed in the embodiments of the present disclosure may also be referred to as a model unifying frame based on a large model training frame, so that an upper algorithm development engineer may not feel to a bottom parallel scheme, may be regarded as developing and testing a model with any parameter in a unified manner, and may omit a hardware environment from a single machine single card to multiple machines multiple cards in an algorithm layer in a development process, thereby providing a unified engineering abstraction foundation for the algorithm development engineer. Moreover, the task processing platform provided by the embodiment of the specification can provide enough flexibility and expandability for users familiar with the large model training framework, and support the users to expand additional layer structure implementation.

Specifically, the embodiment of the specification provides a task processing method, which is applied to a target processing unit in a task processing platform, wherein the task processing platform comprises a first storage unit, a second storage unit and a target processing unit, and comprises the following steps: receiving a task processing request, wherein the task processing request carries a task execution configuration file; obtaining a target task processing model corresponding to a task processing request from a first storage unit, wherein the first storage unit comprises a plurality of task processing models stored in advance; analyzing the task execution configuration file by utilizing the analysis file stored in the second storage unit, and determining the operation architecture of the target task processing model; and calling a target task processing model based on the operation framework to process the task processing request, and obtaining a task processing result. By decoupling the engineering framework code and the algorithm model code, the analysis file and the model file are respectively stored in different storage units, so that a user can develop a task processing model without perceiving a parallel architecture of a bottom model, and engineering complexity and development cost in a distributed large model training process are reduced.

In the present specification, a task processing method, which relates to an automatic question-answering method, a task processing device, an automatic question-answering device, a computing device, a computer-readable storage medium, and a computer program, are provided, and the following embodiments are described in detail one by one.

Referring to fig. 1, fig. 1 illustrates an architecture diagram of a task processing system provided in one embodiment of the present specification, where the task processing system may include a client 100 and a task processing platform 200, and the task processing platform 200 includes a first storage unit 202, a second storage unit 204, and a target processing unit 206;

the client 100 is configured to send a task processing request to the task processing platform 200, where the task processing request carries a task execution configuration file;

a target processing unit 206, configured to obtain a target task processing model corresponding to the task processing request from the first storage unit 202, where the first storage unit 202 includes a plurality of task processing models stored in advance; analyzing the task execution configuration file by utilizing the analysis file stored in the second storage unit 204, and determining the operation architecture of the target task processing model; invoking a target task processing model based on an operation framework to process the task processing request, and obtaining a task processing result; sending a task processing result to the client 100;

The client 100 is further configured to receive a task processing result sent by the task processing platform 200.

By applying the scheme of the embodiment of the specification, the analysis file and the model file are respectively stored in different storage units by decoupling the engineering framework code and the algorithm model code, so that a user can develop a task processing model without perceiving a parallel architecture of a bottom model, and the task processing complexity is simplified.

Referring to FIG. 2, FIG. 2 illustrates an architecture diagram of another task processing system provided by one embodiment of the present description, which may include a plurality of clients 100 and a task processing platform 200. Communication connection can be established between the plurality of clients 100 through the task processing platform 200, in a task processing scenario, the task processing platform 200 is used to provide task processing services between the plurality of clients 100, and the plurality of clients 100 can respectively serve as a transmitting end or a receiving end, so that communication is realized through the task processing platform 200.

The user may interact with the task processing platform 200 through the client 100 to receive data sent by other clients 100, or to send data to other clients 100, etc. In the task processing scenario, it may be that the user issues a data stream to the task processing platform 200 through the client 100, and the task processing platform 200 generates a task processing result according to the data stream and pushes the task processing result to other clients that establish communication.

Wherein, the client 100 and the task processing platform 200 establish a connection through a network. The network provides a medium for communication links between clients 100 and task processing platform 200. The network may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others. Data transmitted by the client 100 may need to be encoded, transcoded, compressed, etc. before being distributed to the task processing platform 200.

The client 100 may be a browser, APP (Application), or web Application such as H5 (HyperText Markup Language, hypertext markup language (htl) 5 th edition) Application, or a light Application (also called applet, a lightweight Application), or cloud Application, etc., and the client 100 may be based on a software development kit (SDK, software Development Kit) of the corresponding service provided by the task processing platform 200, such as a real-time communication (RTC, real Time Communication) based SDK development acquisition, etc. The client 100 may be deployed in an electronic device, need to run depending on the device or some APP in the device, etc. The electronic device may for example have a display screen and support information browsing etc. as may be a personal mobile terminal such as a mobile phone, tablet computer, personal computer etc. Various other types of applications are also commonly deployed in electronic devices, such as human-machine conversation type applications, model training type applications, text processing type applications, web browser applications, shopping type applications, search type applications, instant messaging tools, mailbox clients, social platform software, and the like.

Task processing platform 200 can include servers that provide various services, such as servers that provide communication services for multiple clients, servers for background training that provide support for models used on clients, servers that process data sent by clients, and so forth. It should be noted that, the task processing platform 200 may be implemented as a distributed server cluster formed by a plurality of servers, or may be implemented as a single server. The server may also be a server of a distributed system or a server that incorporates a blockchain. The server may also be a cloud server for cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN, content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms, or an intelligent cloud computing server or an intelligent cloud host with artificial intelligence technology.

It should be noted that, the task processing method provided in the embodiments of the present disclosure is generally executed by the task processing platform, but in other embodiments of the present disclosure, the client may also have a similar function to the task processing platform, so as to execute the task processing method provided in the embodiments of the present disclosure. In other embodiments, the task processing methods provided in the embodiments of the present disclosure may also be executed by the client together with the task processing platform.

Referring to fig. 3, fig. 3 shows a flowchart of a task processing method provided in an embodiment of the present disclosure, where the task processing method is applied to a target processing unit in a task processing platform, and the task processing platform includes a first storage unit, a second storage unit, and a target processing unit, and specifically includes the following steps:

step 302: and receiving a task processing request, wherein the task processing request carries a task execution configuration file.

In one or more embodiments of the present disclosure, a task execution configuration file may be acquired, so that task processing is performed based on the task execution configuration file, and a task processing result is obtained.

In particular, the task processing request may be a task in a different scenario, such as a task in a conference scenario, a task in an e-commerce scenario, and so on. The tasks requested to be processed by the task processing request may also be different types of tasks, such as image generation tasks, recommendation tasks, model training tasks, and so forth.

The target processing unit in the task processing platform can be a large model training framework, the target processing unit can have a plurality of different versions, operators used by different processing versions or parallel implementation have differences, and the scale of the supported task processing models is different.

Referring to fig. 4, fig. 4 is a schematic Version diagram of a target processing unit in a task processing method according to an embodiment of the present disclosure, and as shown in fig. 4, the Version (Version) of the target processing unit may be a large model training framework V1, V3, distributed training (deep) and a distributed implementation (MOE, mixture of Experts) of a hybrid expert system. Different versions include parallel layer implementations (Layers), synchronization logic (maps) at parallel run time, random number synchronization at parallel run time, and recomputation synchronization (random).

The task execution configuration file may be understood as a configuration dictionary for configuring the target processing version and the parallel parameters of the target processing unit. The task execution profile may be generated by linking a third party profile. The task execution configuration file includes, but is not limited to, version information of the target processing unit and parallelism configuration file, and is specifically selected according to practical situations, which is not limited in any way in the embodiment of the present specification.

In practical applications, the manner of receiving the task processing request is various, and is specifically selected according to practical situations, which is not limited in any way in the embodiment of the present disclosure. In one possible implementation manner of the present disclosure, a task processing request sent by a user through a client may be received. In another possible implementation manner of the present disclosure, a task processing request sent by a client in a timing manner may be received.

In an alternative embodiment of the present disclosure, a user may configure the target processing version information in a task execution configuration file, so that the target processing unit switches to the target processing version to perform task processing, that is, the target processing unit includes a plurality of processing versions, and the task execution configuration file includes the target processing version information; after receiving the task processing request, the method may further include the following steps:

and switching to the target processing version corresponding to the task processing request according to the target processing version information.

Specifically, the target processing version information includes, but is not limited to, a target processing version identifier and a target processing version configuration file, and is specifically selected according to practical situations, which is not limited in any way in the embodiments of the present specification.

It should be noted that, for a user who only depends on the target processing unit, before performing task processing, the user only needs to configure the target processing version information in the task execution configuration file, and can call the common application programming interface of the target processing unit through an initialization method, so that the processing of the task processing request is achieved by using the distributed task processing capability provided by the task processing platform, where the initialization method refers to an initialization function running when the target processing unit is called, and can be obtained from the code library.

By applying the scheme of the embodiment of the specification, the target processing version corresponding to the task processing request is switched to according to the target processing version information, so that the target processing unit meets the actual requirement of a user, the task processing request can be ensured to be processed normally, and the flexibility of task processing is improved.

Further, for the user with the customization requirement, in an alternative embodiment of the present disclosure, a parallel policy combination scheme is provided, where the user may perform incremental development on the existing version configuration file through version combination information, so as to implement parallel policy combination implemented based on multiple versions of the target processing unit, that is, the target processing version information includes version combination information; the switching to the target processing version corresponding to the task processing request according to the target processing version information may include the following steps:

screening a plurality of versions to be combined processing corresponding to the version combination information from the plurality of processing versions according to the version combination information;

the method comprises the steps of sending a plurality of version configuration files of versions to be combined and processed to a client, and receiving a target version configuration file sent by the client, wherein the target version configuration file is obtained by a user based on the version configuration file;

And switching to the target processing version corresponding to the task processing request according to the target version configuration file.

Specifically, the version combination information includes identification information for uniquely identifying the version to be combined, such as version numbers V1 and V3 of the version to be combined. Version configuration files include, but are not limited to, version information, data parallelism, tensor parallelism, and are specifically selected according to practical situations, and the embodiment of the present disclosure does not limit the description. The parallel policy combination is a combination among distributed parallel policies such as data parallel, tensor parallel, pipeline parallel, expert parallel and the like.

In practical application, when a plurality of versions to be combined corresponding to the version combination information are screened out from a plurality of processed versions according to the version combination information, the version combination information can be matched with the version information of each processed version, and the plurality of versions to be combined are screened out from the plurality of processed versions according to the matching result.

The version configuration file can be obtained from a database, or can be obtained by receiving the version configuration file sent by the configuration personnel. Further, after the version configuration file is obtained, the version configuration files of the multiple versions to be combined can be sent to the client, the user receives the version configuration files of the multiple versions to be combined through the client, creates a new version based on the version configuration files and independently adds parallel implementation to obtain a target version configuration file, then the user sends the target version configuration file to the target processing unit through the client, and the target processing unit switches to the target processing version corresponding to the task processing request in an incremental development replacement mode according to the target version configuration file.

By applying the scheme of the embodiment of the specification, a plurality of versions to be combined processing corresponding to the version combination information are screened out from the plurality of processing versions according to the version combination information; transmitting version configuration files of a plurality of versions to be combined and processed to a client, and receiving a target version configuration file transmitted by the client; according to the target version configuration file, the task processing request is switched to the target processing version corresponding to the task processing request, so that the flexibility and the expandability of task processing are improved, and the user is supported to expand additional version realization.

In another alternative embodiment of the present disclosure, for a user who needs to customize the underlying parallel implementation, the user may configure the target version development file in the task execution configuration file, so that the target processing unit switches the original processing version of the target processing unit to the target processing version for performing task processing based on the target version development file, that is, the task execution configuration file includes the target version development file; after receiving the task processing request, the method may further include the following steps:

and switching the original processing version of the target processing unit to the target processing version corresponding to the task processing request according to the target version development file.

It should be noted that, the target version development file is a file obtained by developing a bottom parallel version of the user based on actual requirements, and the target version development file includes, but is not limited to, version information, data parallelism, and tensor parallelism. The original processing version of the target processing unit is a processing version set by default for the target processing unit, and is specifically selected according to actual situations, which is not limited in any way in the embodiment of the present specification.

By applying the scheme of the embodiment of the specification, the original processing version of the target processing unit is switched to the target processing version corresponding to the task processing request according to the target version development file, so that the flexibility and the expandability of task processing are improved, a user is supported to combine the existing parallel structure according to actual requirements, and an additional parallel structure is developed and managed by using a custom version.

In an alternative embodiment of the present disclosure, the task processing request includes third party profile information; after receiving the task processing request, the method may further include the following steps:

acquiring an update configuration file from the third party configuration file according to the third party configuration file information;

and updating the task execution configuration file according to the update configuration file to obtain an updated task execution configuration file.

Specifically, the third party profile information refers to information related to a third party profile, and the third party profile information is used to uniquely identify the third party profile.

It should be noted that the target processing unit includes a global parameter management (args variable) that runs through the global, that is, the user may customize members in the variable to transfer training or reasoning parameters, where the member in the custom variable may provide a get_args () interface for the variable, for example, get_args (). The world_size may be referred to as a member, where the member is not limited to an initialization parameter, but may be a structure transferred during runtime. The initialization parameter refers to parameters necessary for parallel operation, such as parallelism.

In practical application, the initialization parameters can be combined with training or reasoning parameters so as to realize linkage with the third party configuration file, wherein the linkage refers to that an update configuration file can be transmitted or transmitted from the third party configuration file according to the information of the third party configuration file, and the task execution configuration file is updated according to the update configuration file to obtain the updated task execution configuration file, so that development convenience is improved.

Referring to fig. 5, fig. 5 shows a schematic diagram of a configuration file reading path in a task processing method according to an embodiment of the present disclosure, according to third party configuration file information, an update configuration file (modelscope Configuration) is obtained from a third party configuration file (configuration. Json), further according to the update configuration file, a task execution configuration file (megatron configuration) is updated, and an updated task execution configuration file is obtained and further added to global parameter management (args).

By applying the scheme of the embodiment of the specification, according to the information of the third party configuration file, the update configuration file is obtained from the third party configuration file; according to the updated configuration file, the task execution configuration file is updated to obtain the updated task execution configuration file, and as the task processing platform provided by the embodiment of the specification is a basic framework of distributed training, the provided interface is limited in the field of multi-process parallel training reasoning, and the task processing platform is compatible with other training frameworks. For users, the existing large model codes can be migrated with less time cost, so that a task processing platform is used for distributed operation support, an external algorithm development engineer is not required to completely know parallel implementation details, repeated development work in the model access process is avoided, and development convenience is improved.

Step 304: and acquiring a target task processing model corresponding to the task processing request from a first storage unit, wherein the first storage unit comprises a plurality of task processing models stored in advance.

In one or more embodiments of the present disclosure, after receiving the task processing request, further, a target task processing model corresponding to the task processing request may be obtained from the first storage unit.

Specifically, the target task processing model is a deep learning model, and the task processing model can be obtained by training based on sample data, or a large model (such as a large-scale language model) obtained by training in advance can be used as the task processing model.

In practical applications, there are various ways of acquiring the target task processing model corresponding to the task processing request from the first storage unit, and the method is specifically selected according to the practical situation, which is not limited in any way in the embodiment of the present specification.

In one possible implementation manner of the present disclosure, a target task processing model corresponding to the task processing request may be obtained from the first storage unit according to a task type of processing requested by the task processing request, for example, if the task type of processing requested by the task processing request is an image generating task, the target task processing model is an image generating model with an image generating function.

In another possible implementation manner of the present disclosure, the obtaining the target task processing model corresponding to the task processing request from the first storage unit may include the following steps:

and acquiring a target task processing model corresponding to the task processing request from the first storage unit based on the target processing version.

Since the model sizes supported by the target processing units of the different processing versions are different, the model sizes of the task processing models can be considered while the model functions are considered when the target task processing models are acquired from the first storage unit. Specifically, a model scale condition corresponding to the target processing version is obtained, the model scale of each task processing model in the first storage unit is compared with the model scale condition, and the target task processing model meeting the model scale condition is obtained from the first storage unit according to a comparison result.

By applying the scheme of the embodiment of the specification, the target task processing model corresponding to the task processing request is obtained from the first storage unit based on the target processing version, so that the target task processing model can process the task processing request, and normal processing of the task processing request is ensured.

Step 306: and analyzing the task execution configuration file by utilizing the analysis file stored in the second storage unit, and determining the operation architecture of the target task processing model.

In one or more embodiments of the present disclosure, after receiving a task processing request and obtaining a target task processing model corresponding to the task processing request from a first storage unit, further, an analysis file stored in a second storage unit may be used to analyze a task execution configuration file to determine an operation architecture of the target task processing model.

Specifically, the parsing file may be parallel code, which is used to parse the task execution configuration file and determine the operation architecture of the target task processing model.

In an alternative embodiment of the present disclosure, the task execution profile includes a parallelism profile; the above-mentioned analyzing the task execution configuration file by using the analysis file stored in the second storage unit, and determining the operation architecture of the target task processing model may include the following steps:

analyzing the parallelism configuration file by utilizing the analysis file stored in the second storage unit, and determining a parallel operation architecture of the target task processing model;

the method for processing the task processing request based on the operation architecture to call the target task processing model to obtain a task processing result can comprise the following steps:

and calling a target task processing model based on the parallel operation architecture to process the task processing request, and obtaining a task processing result.

Specifically, the parallelism profile includes at least one of a data parallelism profile, a tensor parallelism profile, and a pipeline parallelism profile.

By applying the scheme of the embodiment of the specification, the parallel operation architecture of the target task processing model is determined by analyzing the parallelism configuration file by utilizing the analysis file stored in the second storage unit; and calling a target task processing model based on the parallel operation architecture to process the task processing request, obtaining a task processing result, and separating and independently developing the concurrent support of the large model by decoupling engineering framework codes and algorithm model codes.

Step 308: and calling a target task processing model based on the operation framework to process the task processing request, and obtaining a task processing result.

In one or more embodiments of the present disclosure, a task processing request is received, a target task processing model corresponding to the task processing request is obtained from a first storage unit, an analysis file stored in a second storage unit is used to analyze a task execution configuration file, and after an operation architecture of the target task processing model is determined, the target task processing model may be further called based on the operation architecture to process the task processing request, so as to obtain a task processing result.

By applying the scheme of the embodiment of the specification, the analysis file and the task processing model are decoupled, and the analysis file and the model file are respectively stored in different storage units, so that a user can develop the task processing model without sensing the parallel architecture of the bottom model, the task processing complexity is simplified, the code reusability among different models is improved, and the practical working cost in the development and use processes of the large model can be saved.

In an optional embodiment of the present disclosure, the processing the task processing request based on the operation architecture calling the target task processing model to obtain a task processing result may include the following steps:

Analyzing the task processing request and determining target task data corresponding to the task processing request;

and calling a target task processing model based on the operation framework to process target task data, and obtaining a task processing result.

Specifically, the target task data is a task processing object, and the target task data may be model training data or model reasoning data, and is specifically selected according to the actual situation, which is not limited in any way in the embodiment of the present specification.

It should be noted that, the task processing request may directly carry the target task data. In order to reduce the data transmission amount, the task processing request may only carry the data identifier of the target task data, and after the data identifier is obtained, the target task data may be obtained from the task database according to the data identifier. Further, the target task processing model can be called based on the operation architecture to process the target task data, and a task processing result is obtained.

By applying the scheme of the embodiment of the specification, the task processing request is analyzed, and target task data corresponding to the task processing request is determined; and calling a target task processing model based on the operation framework to process target task data, so as to obtain a task processing result, simplify task processing complexity and improve task processing efficiency.

In an optional embodiment of the present disclosure, the above-mentioned task processing request is processed by calling a target task processing model based on an operation architecture, and after obtaining a task processing result, the method may further include the following steps:

and sending the task processing result to the client so that the client displays the task processing result to the user.

It should be noted that, after the task processing request is processed by calling the target task processing model based on the operation architecture to obtain the task processing result, the task processing result may be sent to the client, and the model information of the target task processing model corresponding to the task processing result may also be sent to the client, where the model information includes a model name, a model operation architecture, and so on.

In practical applications, there are various ways in which the client side displays the task processing result to the user, and the method is specifically selected according to the practical situation, which is not limited in any way in the embodiment of the present specification. In one possible implementation manner of the present disclosure, the client may only show the task processing result to the user. In another possible implementation manner of the present disclosure, the client may display the task processing result and the model information of the target task processing model to the user at the same time, so that the user may accurately know the task processing result and the corresponding target task processing model and the model operation architecture.

It should be noted that after the task processing result corresponding to the task processing request is obtained, the task processing result may be checked, if the quality check is passed, the task processing result is sent to the client, and if the quality check is not passed, the task processing is performed again.

By applying the scheme of the embodiment of the specification, the task processing result is sent to the client, so that the client displays the task processing result to the user, the user can accurately obtain the task processing result, the interaction with the user is increased, and the user satisfaction is improved.

In another alternative embodiment of the present disclosure, the task processing request further carries target exhibition requirement information; the above-mentioned task processing request is processed by calling the target task processing model based on the operation architecture, and after the task processing result is obtained, the method may further include the following steps:

and sending the task processing result to the client according to the target display demand information so that the client displays the task processing result to the user.

Specifically, the target presentation requirement information is used for describing requirements of users for viewing task processing results. The target display requirement information includes, but is not limited to, only display task processing results, model information of a target task processing model, and the like, and is specifically set according to actual requirements of users, which is not limited in any way in the embodiment of the present specification.

By applying the scheme of the embodiment of the specification, the task processing result is sent to the client according to the target display demand information, so that the client displays the task processing result to the user, the interaction with the user is increased, and the user satisfaction is improved.

In an optional embodiment of the present disclosure, after the client displays the task processing result to the user, the user may perform data processing according to the task processing result, or may send a post-processing request based on the task processing result displayed by the client to perform multiple rounds of task processing, that is, after the task processing result is sent to the client according to the target display requirement information, the method may further include the following steps:

and receiving an adjustment configuration file sent by a user based on the task processing result, and processing the task processing request according to the adjustment configuration file to obtain an adjusted task processing result.

It should be noted that, the adjustment configuration file may be understood as an adjustment configuration dictionary. After the user obtains the task processing result, if the task processing result is not satisfied, the user can send an adjustment configuration file based on the task processing result, reconfigure the target processing version and the parallel parameters of the target processing unit, and perform secondary task processing. The implementation manner of the "processing the task processing request according to the adjustment configuration file to obtain the adjusted task processing result" is the same as the implementation manner of the task processing method, and the embodiments of the present disclosure will not be described in detail.

By applying the scheme of the embodiment of the specification, the adjustment configuration file sent by the user based on the task processing result is received, the task processing request is processed according to the adjustment configuration file, the adjusted task processing result is obtained, the interaction with the user is increased, and the user satisfaction is improved.

The task processing method provided in the present specification will be further described with reference to fig. 6 by taking an application of the task processing method in the field of intelligent question answering as an example. Fig. 6 shows a flowchart of an automatic question-answering method according to an embodiment of the present disclosure, where the automatic question-answering method is applied to a target processing unit in a task processing platform, and the task processing platform includes a first storage unit, a second storage unit, and a target processing unit, and specifically includes the following steps:

step 602: and receiving a problem processing request, wherein the problem processing request carries a problem processing configuration file.

Step 604: and acquiring a target task processing model corresponding to the problem processing request from a first storage unit, wherein the first storage unit comprises a plurality of task processing models stored in advance.

Step 606: and analyzing the problem processing configuration file by utilizing the analysis file stored in the second storage unit, and determining the operation architecture of the target task processing model.

Step 608: and calling a target task processing model based on the operation framework to process the problem processing request, and obtaining a problem reply result.

It should be noted that, the problem processing configuration file is a task execution configuration file, and the problem reply result is a task processing result. The implementation manners of step 602 to step 608 are the same as those of step 302 to step 308, and the description of the embodiment of the present disclosure will not be repeated.

In practical application, the automatic question answering may be a multi-round question answering, specifically, a question processing request is received, where the question processing request carries a question processing configuration file; obtaining a target task processing model corresponding to a problem processing request from a first storage unit, wherein the first storage unit comprises a plurality of task processing models stored in advance; analyzing the problem processing configuration file by utilizing the analysis file stored in the second storage unit, and determining the operation architecture of the target task processing model; invoking a target task processing model based on an operation framework to process the problem processing request, obtaining a problem reply result, and sending the problem reply result to a user; and receiving an updated problem processing request sent by a user based on the problem reply result, and returning to execute the step of acquiring a target task processing model corresponding to the problem processing request from the first storage unit until a preset stop condition is reached, so as to obtain the problem reply result.

It is worth to say that the automatic question-answering method can be applied to a conversation robot, the conversation robot has an intelligent operation solution, conversation experience similar to a person can be provided for a user, and the conversation robot can converse by accessing knowledge without knowledge processing.

Specifically, the dialogue can be realized by accessing knowledge without knowledge processing: knowledge types supported by conversational robots include, but are not limited to, documents, spreadsheets, and project flows. Document knowledge may be obtained by directly connecting to an enterprise-level website, enterprise knowledge base, etc., or by importing documents. The conversation robot can redefine high frequency questions and answers (Frequent Asked Questions) and provide intelligent operation to the user.

Conversation experience for the class: the language understanding capability of the conversation robot is greatly improved. The combined understanding of different types of knowledge can be realized, and the system has the capabilities of multi-round dialogue and reasoning analysis. The dialogue process of the dialogue robot is more natural and anthropomorphic, the generated dialogue is more coherent and natural, and the original knowledge segments can be cited as the use references in the knowledge question-answering process.

Intelligent operation solution: the dialogue robot has complete new guide and noun explanation, and provides one-stop use introduction, so that step-by-step guidance is realized; the dialogue robot can support labeling of robot replies and support ensuring effects by using high-frequency documents and the like, so that effects can be intervened; the conversation robot can also provide data such as concise and practical conversation overview data, project analysis and the like through the data report, so that feedback on the effect is achieved.

By applying the scheme of the embodiment of the specification, the analysis file and the model file are respectively stored in different storage units through decoupling the engineering framework code and the algorithm model code, so that a user can develop a task processing model without perceiving a parallel architecture of a bottom model, and the complexity of automatic question-answering is simplified.

Referring to FIG. 7, FIG. 7 illustrates a frame diagram of a task processing system provided by one embodiment of the present description, as shown in FIG. 7, the task processing system includes three parts, respectively configured for a user, frame support, and parallel implementation;

user configuration: a user may configure a target processing Version (Version) of the target processing unit and a task execution profile (megatron configuration);

the frame supports: the task processing system framework supports hybrid precision operators (Fused kernel), optimizer packages (optimizers) for supporting model parallelism and hybrid precision training, learning rate policy packages (Lr policies) for support, global variable management (Global variables) for each module transfer state within the library;

parallel implementation: the task processing system comprises a model parallel layer implementation (Layers), a data parallel implementation (LocalDDP), a package (Checkpoint) for recalculating forward intermediate variables in a backward stage of saving a video memory implementation, and a distributed implementation (MoE) of a hybrid expert system;

And, the task processing system also includes a global parameter management (args) to interact with each module and provide a modification interface to the outside.

It should be noted that the task processing system proposed in the embodiments of the present specification does not run independently from specific large model reasoning or training, but independently provides the framework support necessary for the large model reasoning training process. When the task processing is carried out in the operation stage, a task execution configuration file (comprising distributed parallel parameters) can be transferred to be initialized, related parameters can be stored in the global parameter management, a target processing unit can automatically compile operator libraries necessary for operation according to the task execution configuration file, a parallel group of corresponding processes is initialized, and then parallel capacity provided by a task processing platform can be used for operating model codes.

In practical application, as the task processing platform keeps decoupling and support of most of the capabilities in the large model training framework, a developer familiar with the large model training framework can import the components required by the developer without worrying about the influence on other components caused by the calling of the modules. The task processing platform provided by the embodiment of the specification also supports some optimizers and learning rate adjustment strategies supported by the large model training framework, and the model trained based on the large model training framework can be used for independently separating model codes at low cost, so that engineering complexity and development cost in the distributed large model training process are reduced. Moreover, since the embodiment of the specification provides the same interfaces and functions as those of the large model training framework, the method is relatively friendly for internal and external users, the use threshold of a developer is obviously reduced, parallel related code details can be shielded for the user through engineering packaging, and an algorithm engineer can concentrate on iteration of the structure of the model.

Referring to fig. 8, fig. 8 is an interface schematic diagram of an automatic question-answering interface according to one embodiment of the present disclosure. The automatic question-answering interface is divided into a question processing request input interface and a answer result display interface. The problem processing request input interface comprises a problem processing request input box, a determination control and a cancel control. The reply result display interface comprises a reply result display frame.

The method comprises the steps that a user inputs a problem processing request through a problem processing request input box displayed by a client, wherein the problem processing request carries a problem processing configuration file, a 'determination' control is selected, a task processing platform receives the problem processing request sent by the client, and a target task processing model corresponding to the problem processing request is obtained from a first storage unit, wherein the first storage unit comprises a plurality of task processing models stored in advance; analyzing the problem processing configuration file by utilizing the analysis file stored in the second storage unit, and determining the operation architecture of the target task processing model; and calling a target task processing model based on the operation framework to process the problem processing request, obtaining a problem reply result, and sending the problem reply result to the client. The client displays the answer result of the question in the answer result display frame.

In practical applications, the manner in which the user operates the control includes any manner such as clicking, double clicking, touch control, mouse hovering, sliding, long pressing, voice control or shaking, and the like, and the selection is specifically performed according to the practical situation, which is not limited in any way in the embodiments of the present disclosure.

Corresponding to the task processing method embodiment, the present disclosure further provides a task processing device embodiment, and fig. 9 shows a schematic structural diagram of a task processing device provided in one embodiment of the present disclosure. As shown in fig. 9, the task processing device is applied to a target processing unit in a task processing platform, the task processing platform including a first storage unit, a second storage unit, and a target processing unit, the device including:

a first receiving module 902 configured to receive a task processing request, where the task processing request carries a task execution configuration file;

a first obtaining module 904, configured to obtain a target task processing model corresponding to the task processing request from a first storage unit, where the first storage unit includes a plurality of task processing models stored in advance;

a first parsing module 906 configured to parse the task execution configuration file by using the parsing file stored in the second storage unit, and determine an operation architecture of the target task processing model;

The first processing module 908 is configured to call the target task processing model based on the operation architecture to process the task processing request, so as to obtain a task processing result.

Optionally, the target processing unit includes a plurality of processing versions, and the task execution configuration file includes target processing version information; the apparatus further comprises: the first switching module is configured to switch to the target processing version corresponding to the task processing request according to the target processing version information.

Optionally, the target processing version information includes version combination information; the first switching module is further configured to screen a plurality of versions to be combined processing corresponding to the version combination information from the plurality of processing versions according to the version combination information; the method comprises the steps of sending a plurality of version configuration files of versions to be combined and processed to a client, and receiving a target version configuration file sent by the client, wherein the target version configuration file is obtained by a user based on the version configuration file; and switching to the target processing version corresponding to the task processing request according to the target version configuration file.

Optionally, the task execution configuration file includes a target version development file; the apparatus further comprises: and the second switching module is configured to switch the original processing version of the target processing unit to the target processing version corresponding to the task processing request according to the target version development file.

Optionally, the first obtaining module 904 is further configured to obtain, from the first storage unit, a target task processing model corresponding to the task processing request based on the target processing version.

Optionally, the task processing request includes third party profile information; the apparatus further comprises: the updating module is configured to acquire an updating configuration file from the third party configuration file according to the third party configuration file information; and updating the task execution configuration file according to the update configuration file to obtain an updated task execution configuration file.

Optionally, the task execution profile includes a parallelism profile; the first parsing module 906 is further configured to parse the parallelism configuration file by using the parsing file stored in the second storage unit, and determine a parallel operation architecture of the target task processing model; the first processing module 908 is further configured to invoke the target task processing model to process the task processing request based on the parallel operation architecture, so as to obtain a task processing result.

Optionally, the first processing module 908 is further configured to parse the task processing request, and determine target task data corresponding to the task processing request; and calling a target task processing model based on the operation framework to process target task data, and obtaining a task processing result.

Optionally, the task processing request further carries target display requirement information; the apparatus further comprises: and the sending module is configured to send the task processing result to the client according to the target display demand information so that the client displays the task processing result to the user.

Optionally, the apparatus further comprises: and the third processing module is configured to receive an adjustment configuration file sent by the user based on the task processing result, process the task processing request according to the adjustment configuration file and obtain an adjusted task processing result.

The above is a schematic solution of a task processing device of the present embodiment. It should be noted that, the technical solution of the task processing device and the technical solution of the task processing method belong to the same concept, and details of the technical solution of the task processing device, which are not described in detail, can be referred to the description of the technical solution of the task processing method.

Corresponding to the above-mentioned automatic question-answering method embodiment, the present disclosure further provides an automatic question-answering device embodiment, and fig. 10 shows a schematic structural diagram of an automatic question-answering device provided in one embodiment of the present disclosure. As shown in fig. 10, the automatic question answering apparatus is applied to a target processing unit in a task processing platform including a first storage unit, a second storage unit, and a target processing unit, the apparatus including:

a second receiving module 1002 configured to receive a problem handling request, wherein the problem handling request carries a problem handling configuration file;

a second obtaining module 1004, configured to obtain a target task processing model corresponding to the problem processing request from a first storage unit, where the first storage unit includes a plurality of task processing models stored in advance;

a second parsing module 1006 configured to parse the problem processing configuration file by using the parsing file stored in the second storage unit, and determine an operation architecture of the target task processing model;

and the second processing module 1008 is configured to call the target task processing model based on the operation architecture to process the problem processing request, and obtain a problem reply result.

The above is a schematic scheme of an automatic question answering apparatus of this embodiment. It should be noted that, the technical solution of the automatic question-answering device and the technical solution of the automatic question-answering method belong to the same concept, and details of the technical solution of the automatic question-answering device, which are not described in detail, can be referred to the description of the technical solution of the automatic question-answering method.

FIG. 11 illustrates a block diagram of a computing device provided in one embodiment of the present description. The components of computing device 1100 include, but are not limited to, a memory 1110 and a processor 1120. Processor 1120 is coupled to memory 1110 via bus 1130, and database 1150 is used to hold data.

The computing device 1100 also includes an access device 1140, the access device 1140 enabling the computing device 1100 to communicate via one or more networks 1160. Examples of such networks include public switched telephone networks (PSTN, public Switched Telephone Network), local area networks (LAN, local Area Network), wide area networks (WAN, wide Area Network), personal area networks (PAN, personal Area Network), or combinations of communication networks such as the internet. The access device 1140 may include one or more of any type of network interface, wired or wireless, such as a network interface card (NIC, network Interface Card), such as an IEEE802.11 wireless local area network (WLAN, wireless Local Area Networks) wireless interface, a worldwide interoperability for microwave access (Wi-MAX, world Interoperability for Microwave Access) interface, an ethernet interface, a universal serial bus (USB, universal Serial Bus) interface, a cellular network interface, a bluetooth interface, a near field communication (NFC, near Field Communication) interface, and so forth.

In one embodiment of the present description, the above components of computing device 1100, as well as other components not shown in FIG. 11, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device illustrated in FIG. 11 is for exemplary purposes only and is not intended to limit the scope of the present description. Those skilled in the art may add or replace other components as desired.

Computing device 1100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), mobile phone (e.g., smart phone), wearable computing device (e.g., smart watch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or personal computer (PC, personal Computer). Computing device 1100 may also be a mobile or stationary server.

The processor 1120 is configured to execute computer-executable instructions that, when executed by the processor, implement the steps of the task processing method or the automatic question-answering method described above.

The foregoing is a schematic illustration of a computing device of this embodiment. It should be noted that, the technical solution of the computing device belongs to the same concept as the technical solutions of the task processing method and the automatic question-answering method, and details of the technical solution of the computing device, which are not described in detail, can be referred to the description of the technical solutions of the task processing method or the automatic question-answering method.

An embodiment of the present disclosure also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the task processing method or the automatic question-answering method described above.

The above is an exemplary version of a computer-readable storage medium of the present embodiment. It should be noted that, the technical solution of the storage medium and the technical solutions of the task processing method and the automatic question-answering method belong to the same concept, and details of the technical solution of the storage medium which are not described in detail can be referred to the description of the technical solutions of the task processing method or the automatic question-answering method.

An embodiment of the present specification also provides a computer program, wherein the computer program, when executed in a computer, causes the computer to perform the steps of the task processing method or the automatic question-answering method described above.

The above is an exemplary version of a computer program of the present embodiment. It should be noted that, the technical solution of the computer program and the technical solution of the task processing method and the automatic question-answering method belong to the same concept, and details of the technical solution of the computer program, which are not described in detail, can be referred to the description of the technical solution of the task processing method or the automatic question-answering method.

The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The computer instructions include computer program code that may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier wave signal, a telecommunication signal, a software distribution medium, and so forth.

It should be noted that, for simplicity of description, the foregoing method embodiments are all expressed as a series of combinations of actions, but it should be understood by those skilled in the art that the embodiments are not limited by the order of actions described, as some steps may be performed in other order or simultaneously according to the embodiments of the present disclosure. Further, those skilled in the art will appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily all required for the embodiments described in the specification.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.

The preferred embodiments of the present specification disclosed above are merely used to help clarify the present specification. Alternative embodiments are not intended to be exhaustive or to limit the invention to the precise form disclosed. Obviously, many modifications and variations are possible in light of the teaching of the embodiments. The embodiments were chosen and described in order to best explain the principles of the embodiments and the practical application, to thereby enable others skilled in the art to best understand and utilize the invention. This specification is to be limited only by the claims and the full scope and equivalents thereof.

Claims

1. A task processing method applied to a target processing unit in a task processing platform, the task processing platform including a first storage unit, a second storage unit and the target processing unit, the method comprising:

acquiring a target task processing model corresponding to the task processing request from the first storage unit, wherein the first storage unit comprises a plurality of task processing models stored in advance;

and calling the target task processing model based on the operation framework to process the task processing request, and obtaining a task processing result.

2. The method of claim 1, the target processing unit comprising a plurality of processing versions, the task execution profile comprising target processing version information;

after receiving the task processing request, the method further comprises:

3. The method of claim 2, the target processing version information comprising version combination information;

the switching to the target processing version corresponding to the task processing request according to the target processing version information comprises the following steps:

the method comprises the steps of sending version configuration files of a plurality of versions to be combined to a client, and receiving a target version configuration file sent by the client, wherein the target version configuration file is obtained by a user based on the version configuration file;

4. The method of claim 1, the task execution configuration file comprising a target version development file;

after receiving the task processing request, the method further comprises:

5. The method according to any one of claims 2 to 4, wherein the obtaining, from the first storage unit, the target task processing model corresponding to the task processing request includes:

6. The method of claim 1, the task processing request comprising third party profile information;

after receiving the task processing request, the method further comprises:

7. The method of claim 1, the task execution profile comprising a parallelism profile;

the analyzing the task execution configuration file by using the analysis file stored in the second storage unit, and determining the operation architecture of the target task processing model includes:

the step of calling the target task processing model based on the operation framework to process the task processing request to obtain a task processing result comprises the following steps:

And calling the target task processing model based on the parallel operation architecture to process the task processing request, so as to obtain a task processing result.

8. The method of claim 1, wherein the invoking the target task processing model based on the running architecture processes the task processing request to obtain a task processing result, comprising:

and calling the target task processing model based on the operation framework to process the target task data, and obtaining a task processing result.

9. The method of claim 1, the task processing request further carrying target presentation demand information;

the step of calling the target task processing model based on the operation framework to process the task processing request, and after obtaining a task processing result, the step of further comprises:

and sending the task processing result to the client according to the target display demand information so that the client displays the task processing result to a user.

10. The method according to claim 9, further comprising, after the sending the task processing result to the client according to the target exhibition requirement information:

And receiving an adjustment configuration file sent by the user based on the task processing result, and processing the task processing request according to the adjustment configuration file to obtain an adjusted task processing result.

11. An automatic question-answering method is applied to a target processing unit in a task processing platform, wherein the task processing platform comprises a first storage unit, a second storage unit and the target processing unit, and the method comprises the following steps:

acquiring a target task processing model corresponding to the problem processing request from the first storage unit, wherein the first storage unit comprises a plurality of task processing models stored in advance;

and calling the target task processing model based on the operation framework to process the problem processing request, and obtaining a problem reply result.

12. A computing device, comprising:

a memory and a processor;

the memory is configured to store computer executable instructions, the processor being configured to execute the computer executable instructions, which when executed by the processor, implement the steps of the method of any one of claims 1 to 10 or claim 11.

13. A computer readable storage medium storing computer executable instructions which when executed by a processor implement the steps of the method of any one of claims 1 to 10 or claim 11.