CN117370000A

CN117370000A - Image generation method, device, electronic equipment and storage medium

Info

Publication number: CN117370000A
Application number: CN202311267690.2A
Authority: CN
Inventors: 周超
Original assignee: Netease Hangzhou Network Co Ltd
Current assignee: Netease Hangzhou Network Co Ltd
Priority date: 2023-09-27
Filing date: 2023-09-27
Publication date: 2024-01-09

Abstract

The present disclosure provides an image generation method, apparatus, electronic device, and storage medium, the method comprising: determining an image generation task and an alternative task node; matching the image generation task with the alternative task node, and determining a task execution node corresponding to the image generation task from the alternative task node; and executing the image generation task based on the task execution node to generate the image. According to the scheme provided by the disclosure, the image generation task and the alternative task node are matched, and the task execution node corresponding to the image generation task is determined from the alternative task node, so that the load of the task node is balanced, and the performance bottleneck is broken through to a certain extent.

Description

Image generation method, device, electronic equipment and storage medium

Technical Field

The disclosure relates to the technical field of computer vision, and in particular relates to an image generation method, an image generation device, electronic equipment and a storage medium.

Background

This section is intended to provide a background or context to the embodiments of the disclosure recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.

The Stable diffration model is a large model of a generation formula in the field of computer vision, and can perform image generation tasks such as text generation images and image generation images.

Stable Diffusion Web UI is an AI image generation framework constructed based on the Stable Difference model, which provides a graphical user interface for the Stable Difference model, facilitating the user to more intuitively understand and use the Stable Difference model.

However, the image generation scheme implemented based on Stable Diffusion Web UI in the related art generally employs a single architecture, which has a significant limitation in performance.

Disclosure of Invention

In view of the foregoing, an object of the present disclosure is to provide an image generating method, an image generating device, an electronic device and a storage medium, which at least solve one of the technical problems in the related art to a certain extent.

In view of the above object, a first aspect of exemplary embodiments of the present disclosure provides an image generating method, including:

determining an image generation task and an alternative task node;

matching the image generation task with the alternative task node, and determining a task execution node corresponding to the image generation task from the alternative task node;

And executing the image generation task based on the task execution node to generate the image.

Based on the same inventive concept, a second aspect of exemplary embodiments of the present disclosure provides an image generating apparatus, including:

a task and node determination module configured to determine an image generation task and an alternative task node;

the task and node matching module is configured to match the image generation task with the alternative task nodes, and determine a task execution node corresponding to the image generation task from the alternative task nodes;

and the task execution module is configured to execute the image generation task based on the task execution node to generate the image.

Based on the same inventive concept, a third aspect of exemplary embodiments of the present disclosure provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method according to the first aspect when executing the program.

Based on the same inventive concept, a fourth aspect of the exemplary embodiments of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method according to the first aspect.

As can be seen from the foregoing, the image generating method, apparatus, electronic device and storage medium provided in the embodiments of the present disclosure, the method includes: determining an image generation task and an alternative task node; matching the image generation task with the alternative task node, and determining a task execution node corresponding to the image generation task from the alternative task node; and executing the image generation task based on the task execution node to generate the image. According to the scheme provided by the disclosure, the image generation task and the alternative task node are matched, and the task execution node corresponding to the image generation task is determined from the alternative task node, so that the load of the task node is balanced, and the performance bottleneck is broken through to a certain extent.

Drawings

In order to more clearly illustrate the technical solutions of the present disclosure or related art, the drawings required for the embodiments or related art description will be briefly described below, and it is apparent that the drawings in the following description are only embodiments of the present disclosure, and other drawings may be obtained according to these drawings without inventive effort to those of ordinary skill in the art.

Fig. 1 is a schematic view of an application scenario of an image generation scheme in the related art provided in an exemplary embodiment of the present disclosure;

fig. 2 is another application scenario schematic diagram of an image generation scheme in the related art provided by an exemplary embodiment of the present disclosure;

fig. 3 is a schematic view of an application scenario of an image generating method according to an exemplary embodiment of the present disclosure;

fig. 4 is a schematic view of another application scenario of the image generating method provided in the exemplary embodiment of the present disclosure;

FIG. 5 is a flow chart of an image generation method provided by an exemplary embodiment of the present disclosure;

FIG. 6 is a schematic illustration of image generation parameters in the form of images provided by exemplary embodiments of the present disclosure;

FIG. 7 is another flow chart of an image generation method provided by an exemplary embodiment of the present disclosure;

FIG. 8 is another flow chart of an image generation method provided by an exemplary embodiment of the present disclosure;

fig. 9 is a schematic structural view of an image generating apparatus according to an exemplary embodiment of the present disclosure;

fig. 10 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present disclosure.

Detailed Description

It can be appreciated that before using the technical solutions disclosed in the embodiments of the present application, the user should be informed and authorized by appropriate means of the type, the usage range, the usage scenario, etc. of the personal information related to the present application according to the relevant laws and regulations.

For example, in response to receiving an active request from a user, a prompt is sent to the user to explicitly prompt the user that the operation it is requesting to perform will require personal information to be obtained and used with the user. Therefore, the user can autonomously select whether to provide personal information for software or hardware such as electronic equipment, application programs, servers or storage media for executing the operation of the technical scheme according to the prompt information.

As an alternative but non-limiting implementation, in response to receiving an active request from a user, the manner in which the prompt information is sent to the user may be, for example, a popup, in which the prompt information may be presented in a text manner. In addition, a selection control for the user to select to provide personal information to the electronic device in a 'consent' or 'disagreement' manner can be carried in the popup window.

It will be appreciated that the above-described notification and user authorization acquisition process is merely illustrative and not limiting of the implementation of the present application, and that other ways of satisfying relevant legal regulations may be applied to the implementation of the present application.

It will be appreciated that the data (including but not limited to the data itself, the acquisition or use of the data) involved in the present technical solution should comply with the corresponding legal regulations and the requirements of the relevant regulations.

For purposes of making the objects, technical solutions, and advantages of the present disclosure more apparent, the principle and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are presented merely to enable one skilled in the art to better understand and practice the present disclosure and are not intended to limit the scope of the present disclosure in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

In this document, it should be understood that any number of elements in the drawings is for illustration and not limitation, and that any naming is used only for distinction and not for any limitation.

The principles and spirit of the present disclosure are explained in detail below with reference to several representative embodiments thereof.

In the related art, the Stable distribution model is a large generation model in the field of computer vision, and can perform image generation tasks such as text generation images and image generation images. The principle of the Stable difference model is to add noise to a real image, and then gradually remove the noise using a neural network, and with the gradual removal of the noise, the real image is gradually restored.

As a specific example, referring to fig. 1, there is shown a Stable Diffusion Web UI of the related art, in which when the Stable Diffusion Web UI is started, the system performs initializing logic and renders a front-end page according to a predetermined setting, provides a logically clear and interactive graphical user interface for a user, and the user can input parameters through the graphical user interface, for example, input text through a text box displayed with "please input text", input image through a text box displayed with "please input image", and further specify style through a text box displayed with "please specify style", the style refers to, for example, "water and ink", "cartoon" or "simulation", etc., after inputting parameters and clicking a "generate image" button, stable Diffusion Web UI generates an image based on parameters input by the user and returns the generated image to the front-end through Stable Diffusion Web UI and displays it on the screen to the user.

However, the inventors of the present disclosure found that the image generation scheme implemented based on Stable Diffusion Web UI in the related art generally employs a single architecture, which has a significant limitation in performance, and the reason why the performance bottleneck in the related art is more significant is that:

referring to fig. 2, the single architecture includes a client and a service logic layer, where the service logic layer includes a single server, and it can be appreciated that no matter how powerful the single server performs, when there is always a resource exhaustion, therefore, performance bottlenecks are encountered for the Stable Diffusion Web UI-based image generation scheme implemented based on the single architecture, no matter how optimized.

In order to solve the above-mentioned problems, the present disclosure provides an image generation scheme, specifically including: determining an image generation task and an alternative task node; matching the image generation task with the alternative task node, and determining a task execution node corresponding to the image generation task from the alternative task node; and executing the image generation task based on the task execution node to generate the image. According to the scheme provided by the disclosure, the image generation task and the alternative task node are matched, and the task execution node corresponding to the image generation task is determined from the alternative task node, so that the load of the task node is balanced, and the performance bottleneck is broken through to a certain extent.

Referring to fig. 3, the architecture of the image generation scheme provided in the present disclosure includes: the system comprises a client, a scheduler, a distributed business logic layer and a distributed database layer, wherein the distributed business logic layer comprises a plurality of servers, the distributed database layer comprises a plurality of databases, and the same service is expanded to different servers by expanding the number of the servers so as to improve the overall performance. Specifically, requests from clients are distributed to corresponding servers through a scheduler by a certain rule.

Having described the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are specifically described below.

Referring to fig. 4, an application scenario diagram of an image generating method according to an exemplary embodiment of the present disclosure is shown.

The application scenario includes a local terminal device 410 and a server 420. The local terminal device 410 and the server 420 may be connected through a wired or wireless communication network, so as to implement data interaction.

The local terminal device 410 may be an electronic device with data transmission, multimedia input/output functions near the user side, such as a desktop computer, a mobile phone, a mobile computer, a tablet computer, a media player, a car-mounted computer, an intelligent wearable device, a personal digital assistant (personal digital assistant, PDA), or other electronic devices capable of achieving the above functions, etc. The electronic device may include a processor for presenting a graphical user interface that may display a music game interface, and a display screen having a touch input function for processing the music game data, generating the graphical user interface, and controlling the display of the graphical user interface on the display screen.

The server 420 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs (Content Delivery Network, content delivery networks), basic cloud computing services such as big data and artificial intelligent platforms, and the like.

In some exemplary embodiments, the image generation method may run on the server 420.

When the image generation method is operated on the server 420, the server 420 is configured to provide an image generation service to a user of a terminal device, in which a client in communication with the server 420 is installed, and the user may input an image generation parameter through the client, generate an image generation request based on the image generation parameter, the image generation request corresponds to an image generation task, and send the image generation request to the server 420. The server 420 determines an image generation task and an alternative task node; matching the image generation task with the alternative task node, and determining a task execution node corresponding to the image generation task from the alternative task node; and executing the image generation task based on the task execution node to generate the image. The server 420 may also send the generated image to a client, which presents the generated image to the user. Wherein the terminal device may be the aforementioned local terminal device 410.

When the image generation method is run on the server 420, the method may also be implemented and executed based on a cloud interaction system.

The cloud interaction system comprises a client device and a cloud image generation server.

In some example embodiments, various cloud applications may be run under the cloud interaction system, such as: cloud image generation. Taking cloud image generation as an example, cloud image generation refers to an image generation manner based on cloud computing. In the operation mode of cloud image generation, an operation main body of an image generation program and an image generation picture presentation main body are separated, storage and operation of a control method of a moving state in image generation are completed on a cloud image generation server, and a client device is used for receiving and sending data and presenting an image generation picture, for example, the client device can be a display device with a data transmission function, such as a mobile terminal, a television, a computer, a palm computer and the like, which is close to a user side; but the cloud image generation server for cloud is used for information processing. When the image generation is carried out, a player operates the client device to send an operation instruction to the cloud image generation server, the cloud image generation server runs image generation according to the operation instruction, data such as an image generation picture are encoded and compressed, the data are returned to the client device through a network, and finally, the data are decoded through the client device and the image generation picture is output.

In the above embodiments, the description has been given taking an example in which the image generating method is run on the server 420, but the present disclosure is not limited thereto, and in some exemplary embodiments, the image generating method may also be run on the local terminal device 410.

When the image generation method is run on the local terminal device 410, the local terminal device 410 stores an image generation program and is used to present an image generation screen. The local terminal device 410 is used to interact with the player through a graphical user interface, i.e. conventionally, the image generation program is downloaded and installed and run through the electronic device. The manner in which the local terminal device 410 provides the graphical user interface to the player may include a variety of ways, for example, it may be rendered for display on a display screen of the terminal, or it may be provided to the player by holographic projection. For example, the local terminal device 410 may include a display screen for presenting a graphical user interface including an image generation screen, and a processor for running the image generation method, generating the graphical user interface, and controlling the display of the graphical user interface on the display screen.

When the image generation method is run on the local terminal device 410, the local terminal device 410 may include a display screen and a processor. The local terminal device 410 is provided with a client, and a user can input image generation parameters through the client, generate an image generation request based on the image generation parameters, wherein the image generation request corresponds to an image generation task, and send the image generation request to a processor. The processor determines an image generation task and an alternative task node; matching the image generation task with the alternative task node, and determining a task execution node corresponding to the image generation task from the alternative task node; and executing the image generation task based on the task execution node to generate the image. The processor may also send the generated image to a client, which presents the generated image to a user via a display screen.

An image generation scheme according to an exemplary embodiment of the present disclosure is described below in connection with the application scenario of fig. 4. It should be noted that the above application scenario is only shown for the convenience of understanding the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in any way in this respect. Rather, embodiments of the present disclosure may be applied to any scenario where applicable.

Referring to fig. 5, a flowchart of an image generating method according to an exemplary embodiment of the present disclosure is shown.

An image generation method comprising the steps of:

step S510, determining an image generation task and an alternative task node.

In the present exemplary embodiment, determining the image generation task includes at least two of:

receiving the image generation task;

or alternatively, the first and second heat exchangers may be,

and generating the image generation task.

In specific implementation, the receiving the image generation task includes:

and receiving the image generation task generated at the client and sent by the client.

In specific implementation, the generating the image generating task includes:

the image generation task is generated at a server or processor.

In the present exemplary embodiment, in the same manner in which the client, the server, or the processor generates the image generation task, the generation manner of the image generation task will be described below:

In this exemplary embodiment, the generation manner of the image generation task includes:

determining an image generation parameter to be input;

and generating the image generation task based on the image generation parameters.

In this exemplary embodiment, the image generation parameters include at least one of:

text, image.

In particular implementations, the image generation parameters to be input, i.e., the image generation parameters input by the user, such as text and/or images, characterize the elements contained in the image that the user desires to generate.

As a specific example, a user is provided with a graphical user interface-based user interface in the form of a Web page. The user can open the user operation interface through the browser, click the image generation button after filling in the required image generation parameters, generate an image generation task and send a request for executing the image generation task.

The graphical User Interface comprises a User Interface (UI) and a game picture, wherein the UI is a medium for human-computer interaction and information exchange between a system and a User, and the UI can display system information in a human acceptable form, so that the User can conveniently and effectively operate the computer to achieve bidirectional human-computer interaction. The user operation interface may be composed of visual elements such as controls, text, graphics, images, icons, input boxes, and the like. In alternative embodiments, the user interface may include collection controls, research controls, characterization controls, and the like. In an alternative embodiment, the game screen is a display screen corresponding to the virtual scene displayed by the terminal device, and the game screen may include a controlled virtual object for executing game logic in the virtual scene, and the like.

As a specific example, referring to fig. 6, assuming that the user desires to generate an image similar to that shown in fig. 6 including "trees", "houses", "streets" and "clouds", the image generation parameters entered by the user may be: including text of "trees", "houses", "streets" and "clouds".

Wherein the language used by the text is not limited.

The image generation parameters input by the user may also be: similar to the image of fig. 6 including "trees", "houses", "streets" and "clouds" therein.

Wherein, the format adopted by the image is not limited; also, the number of images is not limited, that is, the images input by the user are not limited to one, but may be plural.

The image generation parameters entered by the user may also be: text including "trees", "houses", "streets" and "sky" and images similar to fig. 6 including "trees", "houses", "streets" and "clouds" therein.

In this exemplary embodiment, the generating the image generating task based on the image generating parameter includes:

carrying out serialization processing on the image generation parameters to obtain image generation parameter vectors;

The image generation task includes the image generation parameter vector.

In the specific implementation, the text and/or the image are subjected to serialization processing to obtain text and/or image vectors;

the text and/or image vectors are included in the image generation task.

In the above-described embodiment, the image generation parameters (e.g., text and/or image) are preprocessed (e.g., serialized) at the stage of determining the image generation task (i.e., before step S510), but the present disclosure is not limited thereto, and in other embodiments, the image generation parameters (e.g., text and/or image) may be preprocessed (e.g., serialized) at the stage of generating the image based on the image generation task (i.e., after and after step S510), in which case not the image generation parameter vector (e.g., text and/or image vector) but the image generation parameter itself (e.g., text and/or image itself) is included in the image generation task, specifically:

the image generation task includes the image generation parameters.

In specific implementation, the text and/or the image are included in the image generation task.

In the above-described embodiment, the determination manner of the image generation task is described, and the determination manner of the alternative task node will be described below:

monitoring a change event of the alternative task node, and maintaining a schedulable alternative task node list. The change event of the task node comprises: newly adding an alternative task node event, modifying the alternative task node event and deleting the alternative task node event.

In specific implementation, the processing of the alternative task node change event includes:

and for the event of the newly added candidate task node, adding the information of the newly added candidate task node into the maintained candidate task node list, and starting a background cooperative monitoring task list corresponding to the candidate task node.

And for the event of modifying the alternative task node, finding a corresponding alternative task node in the maintained alternative task node list, and modifying the alternative task node information.

For the event of deleting the alternative task node, deleting the alternative task node from the maintained alternative task node list, and not consuming tasks from the alternative task node task queue bound by the alternative task node. If the task queue bound by the candidate task node still has the legacy task, the task is taken out and re-added to the original task queue to wait for being scheduled by other candidate task nodes.

As a specific example, at startup, the candidate task node needs to register its own information into the ETCD memory, and schedule the task to execute on the candidate task node according to the information in the ETCD. When the alternative task node changes the meta information or the alternative task node exits, the information of the alternative task node is synchronized into the ETCD for scheduling decision.

And step S520, matching the image generation task with the alternative task nodes, and determining a task execution node corresponding to the image generation task from the alternative task nodes.

In this exemplary embodiment, the matching the image generating task and the candidate task node, and determining, from the candidate task node, a task execution node corresponding to the image generating task includes:

determining task estimated time length and an image generation model corresponding to the image generation task;

determining a task execution duration range corresponding to the alternative task node and a loaded image generation model;

matching the estimated time length of the task with the execution time length range of the task, matching the image generation model with the loaded image generation model, and determining the task execution node corresponding to the image generation task from the candidate task nodes.

Next, a task estimated time length corresponding to the image generation task and a determination manner of the image generation model will be described first:

in the present exemplary embodiment, the task estimated time length corresponding to the image generation task refers to the approximate time length required for executing the image generation task;

the image generation model corresponding to the image generation task refers to an image generation model required to perform the image generation task, for example, whether an image model is generated for text or an image.

In this exemplary embodiment, the determining a task estimated duration and an image generation model corresponding to the image generation task includes:

and determining the task estimated time length and the image generation model corresponding to the image generation task based on the image generation parameters.

In this exemplary embodiment, the determining, based on the image generation parameter, the task estimated duration corresponding to the image generation task includes:

determining at least one of size information, magnification information and/or quantity information of an image corresponding to the image generation task based on the image generation parameters;

and determining the task estimated time length corresponding to the image generation task based on at least one of size information, magnification information and/or quantity information.

In particular implementations, the size information of the image includes the length and width of the image.

In specific implementation, determining the task estimated duration corresponding to the image generation task based on at least one of size information, magnification information and/or quantity information includes:

and taking the product of the size information, the magnification information and/or the quantity information as the estimated time length of the task.

In this exemplary embodiment, the determining, based on the image generation parameters, the image generation model corresponding to the image generation task includes:

determining whether an image is included in the image generation parameters;

in response to determining that the image generation parameters do not include images, determining that the image generation model corresponding to the image generation task is a text generation image model;

or alternatively, the first and second heat exchangers may be,

and in response to determining that the image generation parameters comprise images, determining that the image generation models corresponding to the image generation tasks generate image models for the images.

In the above embodiment, the generation manner of the image generation task, the task estimated time length corresponding to the image generation task, and the determination manner of the image generation model are described respectively, where the task estimated time length corresponding to the image generation task and the image generation model may be determined after the image generation task is generated. However, the disclosure is not limited thereto, and in other embodiments, the task estimated time length and the image generation model corresponding to the image generation task may be determined when the image generation task is generated, and the task estimated time length and the information of the image generation model corresponding to the image generation task may be stored in the image generation task, specifically:

In this exemplary embodiment, the image generating task further includes:

the task estimated time length and the image generation model corresponding to the image generation task.

In the above embodiment, the task estimated time length corresponding to the image generation task and the determination manner of the image generation model are described, and next, the task execution time length range corresponding to the candidate task node and the determination manner of the loaded image generation model will be described:

in this exemplary embodiment, determining a task execution duration range and a loaded image generation model corresponding to the candidate task node includes:

and storing the task execution duration range corresponding to the alternative task node and the information of the loaded image generation model in the alternative task node.

In specific implementation, the task execution duration range corresponding to the candidate task node represents the range of the duration of the task which is configured by the candidate task node and is suitable for execution; the loaded image generation model characterizes the candidate task node's current loaded image generation model.

In the above embodiment, the determination manners of the task estimated time length and the image generation model corresponding to the image generation task and the determination manners of the task execution time length range and the loaded image generation model corresponding to the candidate task node are described, and the specific matching manners of the image generation task and the candidate task node will be described on the basis of the description manners:

In this exemplary embodiment, the matching the task estimated duration and the task execution duration range, and the matching the image generation model and the loaded image generation model, and determining, from the candidate task nodes, the task execution node corresponding to the image generation task includes:

for any of the image generation tasks and any of the alternative task nodes,

and determining that the candidate task node is the task execution node corresponding to the image generation task in response to the fact that the task estimated time length is within the task execution time length range and the image generation model is the same as the loaded image generation model.

As a specific example, first, an alternative task node is selected for which the task predicted duration matches the task execution duration range. According to the task execution duration range, the alternative task nodes are divided into different types: an alternative task node of the image generation task with longer processing time and an alternative task node of the image generation task with shorter processing time. According to the task estimated time length type, an alternative task node list with the matched task estimated time length and task execution time length range can be filtered. Then, an alternative task node is selected for which the image generation model matches. And traversing an alternative task node list with matched task estimated time length and task execution time length range, and selecting an alternative task node with the same first currently loaded image generation model as the image generation task image generation model, so that the time cost of switching the drawing model can be avoided.

If there is no candidate task node for which the image generation model matches, a first free candidate task node is selected from the list of candidate task nodes as the most candidate task node. If the optimal candidate task node is found, adding the image generation task to a task list corresponding to the optimal candidate task node; if the most candidate task node does not exist, the image generation task is rolled back to the front list to wait for the next round of scheduling.

Step S530, executing the image generating task based on the task execution node to generate the image.

In this exemplary embodiment, the executing the image generation task based on the task execution node to generate the image includes:

determining an image generation interface corresponding to the image generation task based on the image generation model;

and calling the image generation interface on the task execution node, and executing the image generation task to generate the image.

When the method is implemented, if the image generation model is the text generation image model, determining that the image generation interface corresponding to the image generation task is a text generation image interface;

And if the image generation model generates an image model for the image, determining the image generation interface corresponding to the image generation task as an image generation image interface.

In particular implementations, the task execution node is configured to provide image generation services, such as text-generated images and/or image-generated image interfaces, in the form of APIs, and may load corresponding image generation models in the local GPUs and execute the image generation tasks.

In this exemplary embodiment, after the task generating task is performed based on the task performing node to generate the image, the method further includes:

and displaying the generated image.

In the above embodiments, the description was made taking the example in which the generated image is shown after the image is generated, however, since the image generation task is generally time-consuming longer, in some exemplary embodiments, the state of the task may be shown at the front end. After the front end submits the request for the image generation task, the state of the image generation task, such as whether the task is completed or a specific progress percentage, can be obtained through the task interface.

Specifically, it can be divided into two phases: task scheduling and task execution.

When the task is scheduled, the task state is stored in a database, so that the front end can inquire the queuing condition of the task in the queue through the task interface.

When a task is scheduled to be executed by a specific candidate task node, the candidate task node corresponding to the task can be searched, and the execution progress is obtained through a task state interface of the candidate task node, so that the task state is returned to the front end for displaying the task state.

As can be seen from the foregoing, the image generating method provided by the embodiment of the present disclosure includes: determining an image generation task and an alternative task node; matching the image generation task with the alternative task node, and determining a task execution node corresponding to the image generation task from the alternative task node; and executing the image generation task based on the task execution node to generate the image. According to the scheme provided by the disclosure, the image generation task and the alternative task node are matched, and the task execution node corresponding to the image generation task is determined from the alternative task node, so that the load of the task node is balanced, and the performance bottleneck is broken through to a certain extent.

Referring to FIG. 7, in some exemplary embodiments, node allocation and node execution of image generation tasks are performed based on a distributed task queue approach, in particular:

step 710, determining a task estimated duration corresponding to an image generation task.

receiving the image generation task;

or alternatively, the first and second heat exchangers may be,

and generating the image generation task.

determining an image generation parameter to be input;

text, image.

the image generation task includes the image generation parameter vector.

In this exemplary embodiment, the determining a task estimated duration corresponding to the image generating task includes:

And determining the task estimated time length corresponding to the image generation task based on the image generation parameters.

determining size information, magnification factor information and quantity information of the image corresponding to the image generation task based on the image generation parameters;

and determining the task estimated time length corresponding to the image generation task based on the size information, the magnification factor information and the quantity information.

Step 720, determining whether the estimated duration of the task is smaller than a preset duration threshold.

In specific implementation, the estimated time length of the task and the preset time length threshold value are subjected to numerical comparison to determine whether the estimated time length of the task is smaller than the preset time length threshold value.

In this exemplary embodiment, an image generation model corresponding to the image generation task is determined, and the preset duration threshold is determined based on the image generation model.

In a specific implementation, the determining the image generation model corresponding to the image generation task includes:

and determining the image generation model corresponding to the image generation task based on the image generation parameters.

determining whether an image is included in the image generation parameters;

or alternatively, the first and second heat exchangers may be,

In a specific implementation, the determining the preset duration threshold based on the image generation model includes:

in response to determining that the image generation model generates an image model for the text, determining that the preset duration threshold is a first preset duration threshold;

or alternatively, the first and second heat exchangers may be,

in response to determining that the image generation model generates an image model for the image, determining that the preset duration threshold is a second preset duration threshold;

wherein the first preset duration threshold is greater than the second preset duration threshold.

Step S730, in response to determining that the task estimated duration is less than the preset duration threshold, adding the image generating task to a first task queue.

In implementation, the first task queue is configured to schedule the image generation task with the shorter task estimated duration. Wherein shorter is relative to the preset duration threshold.

Step S740, in response to determining that the estimated time length of the task is greater than or equal to the preset time length threshold, adding the image generating task to a second task queue.

In specific implementation, the second task queue is configured to schedule the image generation task with the longer task estimated duration. Wherein longer is relative to the preset duration threshold.

Step S750, adding the image generating task from the first task queue or the second task queue to the node task queue corresponding to the task executing node.

In particular implementations, the node task queue is configured to control execution of the task execution node corresponding to the image generation task.

In this exemplary embodiment, the adding the image generating task from the first task queue or the second task queue to the node task queue corresponding to the task executing node includes:

determining whether the task execution node is in a high-load state;

And in response to determining that the task execution node is not in the high-load state, adding the image generation task from the first task queue or the second task queue to the node task queue corresponding to the task execution node.

In specific implementation, the value corresponding to the high load state may be preconfigured.

Step S760, executing the image generation task based on the node task queue to generate the image.

In specific implementation, the image generation task is executed based on the sequential position of the image generation task in the node task queue to generate the image.

Referring to fig. 8, in the present exemplary embodiment, an image generation method provided by the present disclosure is described taking as an example the implementation of the AI image generation framework based on Stable Diffusion Web UI:

The Stable distribution model is a large generation model in the field of computer vision, and can perform image generation tasks such as text generation images and image generation images. The principle of the Stable difference model is to add noise to a real image, and then gradually remove the noise using a neural network, and with the gradual removal of the noise, the real image is gradually restored.

In the related art, the image generation scheme implemented based on Stable Diffusion Web UI generally adopts a single architecture, including a client and a service logic layer, where the service logic layer includes a single server, and it can be understood that no matter how powerful the single server performs, when there is always a resource exhaustion, therefore, performance bottlenecks are encountered for the image generation scheme implemented based on the single architecture based on Stable Diffusion Web UI, regardless of optimization. Thus, image generation schemes implemented in the related art based on Stable Diffusion Web UI of the monolithic architecture typically have significant performance bottlenecks.

In order to break through the performance bottleneck of the image generation scheme implemented based on Stable Diffusion Web UI of the monomer architecture in the related art, the disclosure divides Stable Diffusion Web UI into a front-end module, a Scheduler module and a back-end module, and when implemented, stable Diffusion Web UI may be divided into WebUI front-end nodes, scheduler schedulers and WebUI back-end nodes.

In the present exemplary embodiment, the WebUI front-end node is configured to receive an image generation parameter input by a user, generate an image generation task based on the image generation parameter, and send a request for executing the image generation task to the Scheduler based on the image generation task.

In particular implementations, the image generation parameters include at least one of:

text or images.

Wherein user-entered image generation parameters, such as text and/or images, characterize the elements contained in the image that the user desires to generate.

In specific implementation, the method for generating the image generation task based on the image generation parameters comprises the following steps:

the image generation task includes the image generation parameter vector.

As a specific example, the WebUI front end node provides a graphical operator interface to the user in the form of a Web page. The user opens the operation interface through the browser, clicks the image generation button after filling in the required image generation parameters, generates an image generation task, and sends a request for the image generation task. The request for the image generation task is intercepted and the Scheduler is requested by means of an API call instead of using a local function call. After serialization, the request for generating the graph is submitted to a Scheduler for the next processing, and the result is displayed after the processing is completed.

In this exemplary embodiment, the Scheduler is configured to receive a request for an image generation task sent by the WebUI front-end node, asynchronously process the request by using a plurality of queues, select the most suitable WebUI back-end node to schedule the image generation task, and finally forward the task to the WebUI back-end node for processing, and update the state of the task. In addition, the Scheduler can monitor registration and destruction events of the WebUI backend nodes and maintain a schedulable WebUI backend node list.

As a specific example, there are two kinds of task queues in the Scheduler, one is an original task queue, i.e., a queue of original image generation tasks received from WebUI front-end nodes, denoted as task_queue; the other is a task queue of the WebUI back-end node, namely a task queue to be processed by each WebUI back-end node, and one WebUI back-end node corresponds to one queue and is marked as node_task_queue.

The relationship between these two queues is: the task fetched from the task_queue is added to the node_task_queue corresponding to the WebUI backend node after the optimal WebUI backend node is selected.

In some exemplary embodiments, there are two task_queue types, and tasks are added to different task_queue types according to the difference of the estimated duration of the tasks corresponding to the image generation task.

The Scheduler starts a cooperative monitoring task_queue in the background, and continuously consumes tasks from the original task queue.

And for any image generation task, selecting an optimal WebUI back-end node to execute the image generation task according to the task estimated time length estimated for the image generation task and the image generation model of the image generation task. The step of selecting the optimal WebUI backend node is as follows: first, a WebUI back-end node with matched task estimated duration types is selected. WebUI backend nodes are divided into different types: the method comprises the steps of processing a WebUI back-end node of an image generation task with a longer processing time and processing a WebUI back-end node of an image generation task with a shorter processing time. According to the task estimated duration type, a WebUI back-end node list matched with the task estimated duration type can be filtered; then, a WebUI backend node is selected for image generation model matching. Traversing a WebUI back-end node list matched with the task estimated duration type, and selecting a WebUI back-end node of which the first currently loaded image generation model is the same as the image generation model of the image generation task, so that the time cost of switching the drawing model can be avoided. If there is no image generation model matching WebUI backend node, the first free WebUI backend node is selected from the list of WebUI backend nodes as the optimal WebUI backend node. If the optimal WebUI back-end node is found, adding the image generation task into a node_task_queue corresponding to the optimal WebUI back-end node; if the optimal WebUI backend node does not exist, the image generation task is rolled back to the task_queue to wait for the next round of scheduling.

The Scheduler starts the cooperative monitoring node_task_queue in the background, and continuously consumes tasks from the node task queue.

And respectively calling different interfaces of the reasoning back-end node according to different types of tasks, such as calling a text generating image API interface by a text generating image task, and calling an image generating image API interface by the image generating image task.

In specific implementation, the processing of the WebUI back-end node change event by the Scheduler includes:

the Scheduler monitors the events of the ETCD and performs different processing according to the event types.

For the newly added WebUI back-end node event, adding the information of the newly added WebUI back-end node into a WebUI back-end node list maintained by a Scheduler, and starting a background cooperative monitoring corresponding node_task_queue.

For the modified WebUI back-end node event, the corresponding WebUI back-end node is found in the WebUI back-end node list maintained by the Scheduler, and the WebUI back-end node information is modified.

For deleting the WebUI back-end node event, deleting the WebUI back-end node from the WebUI back-end node list maintained by the Scheduler, and not consuming tasks from the WebUI back-end node task queue node_task_queue bound by the WebUI back-end node. If the task queue bound by the WebUI back-end node still has legacy tasks, the tasks are taken out and re-added into the original task queue task_queue, and waiting to be scheduled by other WebUI back-end nodes.

In specific implementation, the processing of the task state by the Scheduler includes:

often the image generation task is time consuming and requires the status of the task to be presented at the front end. After the front end submits the request for the image generation task, the task interface of the Scheduler may obtain the status of the image generation task, such as whether it is completed or a specific progress percentage.

When the task is scheduled, the Scheduler saves the task state into the database, so that the front end can inquire the queuing condition of the task in the queue through the task interface.

When a task is scheduled to be executed by a specific WebUI back-end node, the Scheduler can find the WebUI back-end node corresponding to the task, acquire the execution progress through the task state interface of the WebUI back-end node, and return to the front end to display the task state.

In the present exemplary embodiment, the WebUI backend node is configured to provide image generation services in the form of APIs, such as text-generated images and/or image-generated image interfaces, and may load corresponding models in the local GPUs and perform image-generating tasks.

When the WebUI back-end node is started, the self information needs to be registered in the ETCD, and a Scheduler schedules tasks to the WebUI back-end node to execute according to the information in the ETCD. When the meta information of the WebUI back-end node is changed or the WebUI back-end node exits, the information of the WebUI back-end node is synchronized into the ETCD for the Scheduler to make scheduling decisions.

From the foregoing, it can be seen that, the image generation scheme provided by the exemplary embodiment of the present disclosure is beneficial to implementing horizontal expansion of the inference back end, so as to support higher-level concurrent processing capability, and implement sharing of computing power, so that a user can more effectively utilize cloud resources, and specifically includes:

has high performance: the large-scale requests of the users are evenly distributed to a plurality of reasoning services through front-end and back-end splitting and load balancing, and the large-scale requests of the users can be easily processed by means of more reasoning service copies.

The resource cost can be saved: the reasoning service can perform elastic dynamic capacity expansion and contraction according to the use condition of the resources, expand capacity at the peak time of business, contract capacity at the low peak time of business, and greatly save the resource cost.

The method has expansibility: the HTTP protocol can be utilized to provide the reasoning service, so that the universal reasoning capability is formed, and the requirements of clients such as a desktop end, a mobile end and the like for using the reasoning service can be met.

The utility model has the advantages of easy use: the original using habit of the WebUI user is not required to be changed, the user still uses the original graphical interface, and the existence of front-end and rear-end separation cannot be perceived.

It should be noted that the method of the embodiments of the present disclosure may be performed by a single device, such as a computer or a server. The method of the embodiment can also be applied to a distributed scene, and is completed by mutually matching a plurality of devices. In the case of such a distributed scenario, one of the devices may perform only one or more steps of the methods of embodiments of the present disclosure, the devices interacting with each other to accomplish the methods.

It should be noted that the foregoing describes some embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments described above and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Based on the same inventive concept, the present disclosure also provides an image generating apparatus corresponding to the method of any of the above embodiments.

Referring to fig. 9, a schematic diagram of a structure of an image generating apparatus according to an exemplary embodiment of the present disclosure is provided.

An image generation apparatus comprising the following modules:

a task and node determination module 910 configured to determine an image generation task and an alternative task node;

a task and node matching module 920, configured to match the image generating task with the candidate task node, and determine a task execution node corresponding to the image generating task from the candidate task node;

a task execution module 930 configured to execute the image generation task based on the task execution node to generate the image.

In some exemplary embodiments, the task and node determination module 910 is configured to:

determining an image generation parameter to be input;

In some exemplary embodiments, the image generation parameters include at least one of:

text or images.

the image generation task includes the image generation parameter vector.

In some exemplary embodiments, task and node matching module 920 is configured to:

determining whether an image is included in the image generation parameters;

or alternatively, the first and second heat exchangers may be,

In some exemplary embodiments, the task execution module 930 is configured to:

adding the image generation task to a node task queue corresponding to the task execution node;

and executing the image generation task based on the node task queue to generate the image.

In some exemplary embodiments, the task execution module 930 is configured to:

determining whether the estimated time length of the task is smaller than a preset time length threshold value;

in response to determining that the task estimated duration is less than the preset duration threshold, adding the image generation task to a first task queue;

Or alternatively, the first and second heat exchangers may be,

and in response to determining that the task estimated time length is greater than or equal to the preset time length threshold, adding the image generation task to a second task queue.

In some exemplary embodiments, the task execution module 930 is configured to:

and adding the image generation task from the first task queue or the second task queue to the node task queue corresponding to the task execution node.

In some exemplary embodiments, the task execution module 930 is configured to:

and calling the image generation interface on the task execution node to execute the image generation task.

For convenience of description, the above devices are described as being functionally divided into various modules, respectively. Of course, the functions of the various modules may be implemented in the same one or more pieces of software and/or hardware when implementing the present disclosure.

The device of the foregoing embodiment is configured to implement the corresponding image generation method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which is not described herein.

Based on the same inventive concept, the present disclosure also provides an electronic device corresponding to the method of any embodiment, including a memory, a processor, and a computer program stored on the memory and capable of running on the processor, where the processor implements the image generating method of any embodiment when executing the program.

Fig. 10 shows a more specific hardware architecture of an electronic device according to this embodiment, where the device may include: a processor 1010, a memory 1020, an input/output interface 1030, a communication interface 1040, and a bus 1050. Wherein processor 1010, memory 1020, input/output interface 1030, and communication interface 1040 implement communication connections therebetween within the device via a bus 1050.

The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit ), microprocessor, application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc. for executing relevant programs to implement the technical solutions provided in the embodiments of the present disclosure.

The Memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory ), static storage device, dynamic storage device, or the like. Memory 1020 may store an operating system and other application programs, and when the embodiments of the present specification are implemented in software or firmware, the associated program code is stored in memory 1020 and executed by processor 1010.

The input/output interface 1030 is used to connect with an input/output module for inputting and outputting information. The input/output module may be configured as a component in a device (not shown in the figure) or may be external to the device to provide corresponding functionality. Wherein the input devices may include a keyboard, mouse, touch screen, microphone, various types of sensors, etc., and the output devices may include a display, speaker, vibrator, indicator lights, etc.

Communication interface 1040 is used to connect communication modules (not shown) to enable communication interactions of the present device with other devices. The communication module may implement communication through a wired manner (such as USB, network cable, etc.), or may implement communication through a wireless manner (such as mobile network, WIFI, bluetooth, etc.).

Bus 1050 includes a path for transferring information between components of the device (e.g., processor 1010, memory 1020, input/output interface 1030, and communication interface 1040).

It should be noted that although the above-described device only shows processor 1010, memory 1020, input/output interface 1030, communication interface 1040, and bus 1050, in an implementation, the device may include other components necessary to achieve proper operation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the embodiments of the present description, and not all the components shown in the drawings.

The electronic device of the foregoing embodiment is configured to implement the corresponding image generating method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which is not described herein.

The memory 1020 stores machine readable instructions executable by the processor 1010, which when the electronic device is running, communicate between the processor 1010 and the memory 1020 over the bus 1030, causing the processor 1010 to execute the following instructions when running:

determining an image generation task and an alternative task node;

In a possible implementation manner, the determining the image generating task in the instructions executed by the processor 1010 includes:

determining an image generation parameter to be input;

In a possible implementation manner, the image generation parameters include at least one of the following in the instructions executed by the processor 1010:

Text or images.

In a possible implementation manner, the generating the image generating task based on the image generating parameter in the instructions executed by the processor 1010 includes:

the image generation task includes the image generation parameter vector.

In a possible implementation manner, in the instructions executed by the processor 1010, the matching the image generating task with the candidate task node, and determining, from the candidate task node, a task execution node corresponding to the image generating task includes:

In a possible implementation manner, in the instructions executed by the processor 1010, the determining a task estimated duration and an image generation model corresponding to the image generation task includes:

In a possible implementation manner, in the instructions executed by the processor 1010, the determining, based on the image generation parameter, the estimated task duration corresponding to the image generation task includes:

In a possible implementation manner, in the instructions executed by the processor 1010, the determining the image generation model corresponding to the image generation task based on the image generation parameter includes:

determining whether an image is included in the image generation parameters;

or alternatively, the first and second heat exchangers may be,

In a possible implementation manner, in the instructions executed by the processor 1010, the executing the image generating task based on the task executing node to generate the image includes:

In a possible implementation manner, after determining the task estimated duration corresponding to the image generation task in the instructions executed by the processor 1010, the method further includes:

or alternatively, the first and second heat exchangers may be,

In a possible implementation manner, in an instruction executed by the processor 1010, the adding the image generating task to a node task queue corresponding to the task executing node includes:

From the above, it can be seen that, by matching the image generating task and the alternative task node, the electronic device provided by the embodiment of the present disclosure determines the task executing node corresponding to the image generating task from the alternative task node, so that the load of the task node is balanced, and the performance bottleneck is broken through to a certain extent.

Based on the same inventive concept, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the image generation method according to any of the above embodiments, corresponding to the method of any of the above embodiments.

The non-transitory computer readable storage media described above can be any available media or data storage device that can be accessed by a computer, including, but not limited to, magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (e.g., CD, DVD, BD, HVD, etc.), and semiconductor storage (e.g., ROM, EPROM, EEPROM, nonvolatile storage (NAND FLASH), solid State Disk (SSD)), etc.

The storage medium of the above embodiments stores computer instructions for causing the computer to perform the image generating method according to any one of the above exemplary method portions, and has the advantages of the corresponding method embodiments, which are not described herein.

Based on the same inventive concept, the present disclosure also provides a computer program product corresponding to the image generation method described in any of the above embodiments, which includes computer program instructions. In some embodiments, the computer program instructions may be executed by one or more processors of a computer to cause the computer and/or the processor to perform the image generation method. Corresponding to the execution subject corresponding to each step in each embodiment of the image generation method, the processor for executing the corresponding step may belong to the corresponding execution subject.

The computer program product of the above embodiment is configured to enable the computer and/or the processor to perform the image generating method according to any one of the above embodiments, and has the advantages of corresponding method embodiments, which are not described herein again.

Those skilled in the art will appreciate that embodiments of the present disclosure may be implemented as a system, method, or computer program product. Accordingly, the present disclosure may be embodied in the following forms, namely: all hardware, all software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software, is generally referred to herein as a "circuit," module, "or" system. Furthermore, in some embodiments, the present disclosure may also be embodied in the form of a computer program product in one or more computer-readable media, which contain computer-readable program code.

Any combination of one or more computer readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive example) of the computer-readable storage medium could include, for example: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer, for example, through the internet using an internet service provider.

It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instruction means which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

Furthermore, although the operations of the methods of the present disclosure are depicted in the drawings in a particular order, this is not required to or suggested that these operations must be performed in this particular order or that all of the illustrated operations must be performed in order to achieve desirable results. Rather, the steps depicted in the flowcharts may change the order of execution. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.

It should be noted that unless otherwise defined, technical or scientific terms used in the embodiments of the present disclosure should be given the ordinary meaning as understood by one of ordinary skill in the art to which the present disclosure pertains. The terms "first," "second," and the like, as used in embodiments of the present disclosure, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof, but does not exclude other elements or items. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", etc. are used merely to indicate relative positional relationships, which may also be changed when the absolute position of the object to be described is changed. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements.

While the spirit and principles of the present disclosure have been described with reference to several particular embodiments, it is to be understood that this disclosure is not limited to the particular embodiments disclosed nor does it imply that features in these aspects are not to be combined to benefit from this division, which is done for convenience of description only. The disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. An image generation method, comprising:

determining an image generation task and an alternative task node;

2. The method of claim 1, wherein the determining an image generation task comprises:

determining an image generation parameter to be input;

3. The method of claim 2, wherein the image generation parameters include at least one of:

text or images.

4. The method of claim 2, wherein the generating the image generation task based on the image generation parameters comprises:

the image generation task includes the image generation parameter vector.

5. The method according to claim 2, wherein the matching the image generation task with the candidate task node, and determining a task execution node corresponding to the image generation task from the candidate task nodes, includes:

6. The method of claim 5, wherein determining a task prediction duration and an image generation model corresponding to the image generation task comprises:

7. The method of claim 6, wherein determining the task prediction duration corresponding to the image generation task based on the image generation parameter comprises:

8. The method of claim 6, wherein the determining the image generation model corresponding to the image generation task based on the image generation parameters comprises:

determining whether an image is included in the image generation parameters;

Or alternatively, the first and second heat exchangers may be,

9. The method of claim 5, wherein the performing the image generation task based on the task execution node to generate the image comprises:

10. The method of claim 9, wherein after determining the task prediction duration corresponding to the image generation task, the method further comprises:

or alternatively, the first and second heat exchangers may be,

11. The method of claim 10, wherein adding the image generation task to a node task queue corresponding to the task execution node comprises:

12. The method of claim 5, wherein the performing the image generation task based on the task execution node to generate the image comprises:

13. An image generating apparatus, comprising:

14. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1 to 12 when the program is executed.

15. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1 to 12.