CN116543075A

CN116543075A - Image generation method, device, electronic equipment and storage medium

Info

Publication number: CN116543075A
Application number: CN202310343838.XA
Authority: CN
Inventors: 曹溪语; 陈璇; 辛永正; 张久金; 苏文嗣; 杨虎; 李国豪; 李伟; 佘俏俏; 刘红星
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2023-03-31
Filing date: 2023-03-31
Publication date: 2023-08-04
Anticipated expiration: 2043-03-31
Also published as: CN116543075B

Abstract

The disclosure provides an image generation method, an image generation device, electronic equipment and a storage medium, relates to the technical field of artificial intelligence, and particularly relates to the image processing and computer vision technologies. The specific implementation scheme comprises the following steps: determining an editing area from an image to be edited; invoking an artificial intelligence AI editing function, and acquiring image editing information based on the AI editing function; editing the content of the editing area based on the image editing information to generate a target image. In the embodiment of the disclosure, any region can be designated as an editing region, the selection of the region is more flexible, the editing region is edited by the AI editing function, the use and the operation are more convenient, and the effect of editing the image is better.

Description

Image generation method, device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of artificial intelligence, and in particular, to image processing and computer vision technologies, and in particular, to an image generating method, an image generating device, an electronic device, and a storage medium.

Background

The general image modification and editing needs to be comprehensively processed by combining complex conception and specialized capability, and only more specialized people can better complete and realize drawing, related or modification of the image, so that the requirement on the use threshold of a user is higher, and the realization effect is completely dependent on the user.

Disclosure of Invention

The disclosure provides an image generation method, an image generation device, electronic equipment and a storage medium.

According to an aspect of the present disclosure, there is provided an image generating method including: determining an editing area from an image to be edited; invoking an artificial intelligence AI editing function, and acquiring image editing information based on the AI editing function; and editing the content of the editing area based on the image editing information to generate a target image.

According to a second aspect of the present disclosure, there is provided an image generating apparatus including: the area determining module is used for determining an editing area from the image to be edited; the information acquisition module is used for calling an artificial intelligence AI editing function and acquiring image editing information based on the AI editing function; and the image generation module is used for editing the content of the editing area based on the image editing information to generate a target image.

According to a third aspect of the present disclosure, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the image generation method of the first aspect of the present disclosure.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the image generation method according to the first aspect of the present disclosure.

According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the steps of the image generation method according to the first aspect of the present disclosure.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

fig. 1 is a flowchart of an image generation method according to an embodiment of the present disclosure;

FIG. 2 is a flow chart of another image generation method provided in accordance with an embodiment of the present disclosure;

FIG. 2a is a schematic diagram of obtaining an image to be edited based on an image selection request provided in accordance with an embodiment of the present disclosure;

Fig. 2b is a schematic illustration of a start of application provided in accordance with an embodiment of the present disclosure;

FIG. 2c is a schematic illustration of a painted editing area provided in accordance with an embodiment of the present disclosure;

FIG. 2d is a schematic diagram of a direct invocation of an AI editing component provided in accordance with an embodiment of the disclosure;

fig. 2e is a functional area schematic diagram of an AI editing function provided according to an embodiment of the present disclosure;

FIG. 2f is a schematic diagram of AI editing via shortcut key invocation provided in accordance with an embodiment of the disclosure;

FIG. 2g is a schematic diagram of various configuration sub-regions in a functional region provided in accordance with an embodiment of the present disclosure;

FIG. 2h is a schematic illustration of a target image provided in accordance with an embodiment of the present disclosure;

FIG. 3 is a flow chart of another image generation method provided in accordance with an embodiment of the present disclosure;

FIG. 4 is a flow chart of another image generation method provided in accordance with an embodiment of the present disclosure;

FIG. 5 is a flow diagram of another image generation method provided in accordance with an embodiment of the present disclosure;

FIG. 5a is a schematic illustration of a sub-display area provided in accordance with an embodiment of the present disclosure;

FIG. 6 is a flow chart of another image generation method provided in accordance with an embodiment of the present disclosure;

FIG. 6a is a schematic diagram of a first generation progress display provided in accordance with an embodiment of the present disclosure;

FIG. 6b is a schematic diagram of a marking task presentation area provided in accordance with an embodiment of the present disclosure;

fig. 7 is a block diagram of an image generating apparatus provided according to an embodiment of the present disclosure;

fig. 8 is a block diagram of an electronic device for implementing the methods of embodiments of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Artificial intelligence (Artificial Intelligence, AI) is a discipline of studying certain mental processes and intelligent behaviors (e.g., learning, reasoning, thinking, planning, etc.) that make computers simulate life, both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include computer vision technologies, speech recognition technologies, natural language processing technologies, and learning/deep learning, big data processing technologies, knowledge graph technologies, and the like.

An Image Processing technique (Image Processing technique) is a technique for Processing Image information with a computer. Mainly comprises image digitizing, image enhancing and restoring, image data encoding, image dividing, image identifying and the like.

Computer Vision (Computer Vision) refers to machine Vision that uses a camera and a Computer to replace human eyes to recognize, track and measure targets, and further performs graphic processing, so that the Computer processing becomes an image more suitable for human eyes to observe or transmit to an instrument for detection. Computer vision is a comprehensive discipline including computer science and engineering, signal processing, physics, applied mathematics and statistics, neurophysiology and cognitive sciences, and the like.

Fig. 1 is a flowchart of an image generating method according to an embodiment of the present disclosure. As shown in fig. 1, the method includes, but is not limited to, the steps of:

s101, determining an editing area from an image to be edited.

It should be noted that, the execution body of the training method of the image editing model in the embodiment of the present disclosure may be a hardware device having data information processing capability and/or software necessary for driving the hardware device to operate. Alternatively, the execution body may include a server, a computer, a user terminal, and other intelligent devices. The user terminal comprises, but is not limited to, a mobile phone, a computer, an intelligent voice interaction device and the like.

Optionally, the image to be edited may be an image selected locally by the user, or may be any image in the history database.

Alternatively, the editing area may be all the areas in the image to be edited, or may be an area designated by the user in the image to be edited.

S102, invoking an artificial intelligence AI editing function, and acquiring image editing information based on the AI editing function.

Alternatively, the method of invoking the artificial intelligence AI editing function can choose to invoke the AI editing component directly or via a shortcut. The artificial intelligence AI editing function may intelligently edit the editing region, wherein editing content of the editing region includes, but is not limited to: new addition of elements, removal of elements, and replacement of content.

The image editing information is information for editing an editing area, and is determined by a user operation in the AI editing function. Alternatively, the user may input text information for describing editing modifications to the editing area in the AI editing function.

For example, the image editing information may include text information input by the user and other parameter configuration information of the user, for example, text information "yellow floret" input by the user, or information such as scale setting of the image, number of generated images, editing content of the image to be edited, and the like when the user actually operates.

When a user inputs text information in the AI editing function, the AI editing function performs content completion or removal of an editing region based on the text information input by the user; when the user does not input text information in the AI editing function, the AI editing function performs repair processing on the editing region.

S103, editing the content of the editing area based on the image editing information to generate a target image.

After the image editing information is acquired, editing the content of the editing area based on the image editing information so as to achieve the purposes of adding, removing or modifying the content of elements in the image. Editing the image after the editing area is modified into a target image, wherein the size of the target image is consistent with the size of the image to be edited.

By way of example, assuming that the image to be edited is a cat image with one blue eye, the user wishes to modify the eyes of the cat to green and let the cat wear a lovely bow tie, so that when determining the editing area in the image to be edited, the user can directly smear the appropriate area in the eyes and neck of the cat as the editing area, further call the AI editing function in which text information can be directly input, the text information can be: a kitten with a green eye of a lovely bow tie; meanwhile, the content of the text information is also image editing information, the AI editing function edits the content of the editing area based on the image editing information, a lovely bow tie is worn at the neck of the kitten, and eyes of the kitten are changed from blue to green, so that a target image required by an end user is obtained.

In the embodiment of the disclosure, an editing area is determined in an image to be edited, image editing information is acquired based on an artificial intelligence AI editing function, and the editing area is edited according to the image editing information to obtain a target image. In the embodiment of the disclosure, when the editing area of the image to be edited is determined, any area in the image to be edited can be designated, the selection of the area is more flexible, the editing of the editing area is more targeted, when the AI editing function is called to acquire the image editing information, the user can directly input text information, edit based on the text information, or not input the text information, the function in the AI editing function edits the editing area, the threshold of the image editing is lower, the use and the operation are more convenient, and the accuracy of the image editing is also high.

Fig. 2 is a flow chart of another image generation method provided according to an embodiment of the present disclosure. As shown in fig. 2, the method includes, but is not limited to, the steps of:

s201, determining an editing area from an image to be edited.

In some implementations, when selecting an image to be edited, an image selection request may be received, an image is selected from an image library according to the image selection request, and the selected image is used as the image to be edited; as shown in fig. 2a, the image selection request is "panda and sunflower", and then an image including panda and sunflower is selected as an image to be edited in the image library according to the image selection request.

In other implementations, the candidate image may also be used as the image to be edited during browsing of the candidate image in response to invoking the AI editing function.

Optionally, after determining the image to be edited, the image to be edited may be displayed in an image display area of the interface; further, monitoring a region selection instruction of the image to be edited, and if the region selection instruction is monitored, tracking the region selection instruction and executing to obtain an edited region.

Optionally, the region selection instruction may be an operation such as smearing or circling the image to be edited by the user, or may be generated by a policy algorithm. It can be appreciated that in the embodiments of the present application, the editing area may be determined from the image to be edited in various manners including, but not limited to, smearing, circling, or policy algorithm. As shown in fig. 2b and 2c, the region in the image to be edited may be subjected to operations such as smearing or circling, and the edited region is obtained from the image to be edited.

In the embodiment of the present disclosure, the implementation manner of step S201 may be implemented in any manner of each embodiment of the present disclosure, which is not limited herein, and is not described herein again.

S202, invoking an artificial intelligence AI editing function, and acquiring image editing information based on the AI editing function.

In some implementations, monitoring a first call operation of a component corresponding to the AI editing function; if the first calling operation is monitored, loading and displaying a function area of the AI editing function in the interface designated area. That is, the AI-editing component is directly invoked, as shown in fig. 2d, and when it is determined that a direct invocation is made, the user may click on the AI-editing component, and in response to monitoring that the user clicks on the AI-editing component, a functional area of the AI-editing function is displayed in the interface-designated area, as shown in fig. 2e, which shows the functional area of the AI-editing function, including a text input box and a number adjustment.

In other implementations, monitoring a second call operation of the call shortcut key of the AI editing function; if the second calling operation is monitored, loading and displaying a function area of the AI editing function in the interface designated area. I.e., clicking the shortcut key invokes the AI edit function.

Alternatively, after the edit area is selected, a shortcut key such as "edit the present picture" may be set, and after the user selects to click on the shortcut key, the AI editing function is invoked, and the function area of the AI editing function is displayed in the interface designated area, as shown in fig. 2f, and the AI editing function is invoked by "edit the present picture".

Alternatively, the functional area may include all the functions of editing the editing area, such as element replacement, removal, and content augmentation.

Optionally, the function area may further include a configuration sub-area of each editing parameter, and the configuration sub-area may include one or more of a text configuration sub-area, an image proportion configuration sub-area, a generation mode configuration sub-area, a generation step number configuration sub-area, and an image generation number configuration sub-area. A schematic diagram of the functional area is shown in fig. 2 g.

Alternatively, in the case of image editing based on text information, an input operation for configuring a sub-region for text may be acquired, and the text information may be determined according to the input operation. For example, the text configuration sub-region may be a text entry box. Wherein the input of the text configuration sub-area is determined by the user according to the editing purpose, for example, input "kitten with green eyes of lovely bow tie". It will be appreciated that the user may not need to perform text input operations in the text configuration sub-area without requiring image editing of the text information.

Optionally, a parameter configuration operation may be performed on the remaining configuration sub-areas in the functional area to generate image editing auxiliary information. The remaining configuration sub-areas of the functional area other than the text configuration area may be subjected to parameter configuration operations including, but not limited to: local generation or global generation, generation steps, image scale, etc.

In the embodiment of the present disclosure, the implementation manner of step S202 may be implemented in any manner of each embodiment of the present disclosure, which is not limited herein, and is not described in detail.

S203, obtaining the complement content of the editing area according to the image to be edited and the image editing information.

Optionally, obtaining a mask image of the image to be edited; inputting an image to be edited, a mask image and image editing information into a pre-trained image editing model, and outputting a target image by the image editing model; or, intercepting a local image from the image to be edited, wherein the local image at least comprises an area corresponding to the mask image; the partial image, the mask image, and the image editing information are input into the image editing model, and the target image is output from the image editing model.

In some implementations, after obtaining the pre-trained latent diffusion model, the pre-trained latent diffusion model may be modified to obtain an image editing model, in this disclosure, the image editing model needs to add a mask image and an input channel of a masked image, and the input channel of a denoising network in a pipeline (pipeline) of a Chinese raw image processing in the latent diffusion model may be expanded from original 4 dimensions to 9 dimensions, where the mask image occupies 1 dimension, the masked image occupies 4 dimensions, and the noisy image vector represents z _T Occupying 4 dimensions. The image editing model may include an image self-encoder, a text encoder, and a denoising network.

Based on the mask image and the image to be edited or the partial image, a masked image is obtained, the masked image is image-coded by an image self-encoder to obtain a first image vector representation, and the mask image is image-coded by the image self-encoder to obtain a second image vector representation of the mask image. Optionally, text information in the image editing information is text-encoded by a text encoder to obtain text vector representation, the first image vector representation, the second image vector representation and the vector representation of Gaussian random noise are spliced to obtain spliced vector representation, the spliced vector representation and the text vector representation are input into a denoising network, and iterative denoising processing and a text-generated diagram are performed by the denoising network to obtain a target image

The mask image is an image for obviously distinguishing the editing area, and in the embodiment of the disclosure, the pixel points of the editing area in the image to be edited can be set to be white, and the pixel points of the non-editing area in the image to be edited are set to be black, so that a black-and-white mask image is obtained.

In some implementations, the image editing model may extract style characteristics of an image to be edited, and obtain, based on the style characteristics of the image to be edited and the image editing information, complement content consistent with a style of the image to be edited, where the complement content is complement content of the editing area, and the complement content may include modifications such as patching or deleting of the image content.

S204, performing content complementation on the editing area based on the supplementary content to generate a target image.

When the AI editing is used for processing the editing area, the AI editing also comprises parameter configuration such as local generation or global generation; thus, different parameter configurations are considered when supplementing the editing area with content based on the supplemental content.

Alternatively, when the parameter configuration is global generation, the image to be edited, the mask image, and the image editing information are input into a pre-trained image editing model, and the edited target image is output by the image editing model. When the parameter configuration is locally generated, a local image is cut from the image to be edited, the local image is a part of the image to be edited, the local image at least needs to comprise a region corresponding to the mask image, namely, the region to be edited in the image to be edited, the local image, the mask image and the image editing information are input into a pre-trained image editing model, and the image editing model outputs an edited target image.

It is understood that the process of outputting the target image using the image editing model includes a process of content-complementing the editing region based on the complement content.

Alternatively, the image editing model may be a text-guided image region editing algorithm, and the training set of the image editing model is formed by a large number of images-texts and automatically constructed mask images, and the image editing model may include a plurality of modules such as an image self-encoder, a text encoder, a denoising network and the like.

Alternatively, style characteristics of the image to be edited or the partial image may be extracted; based on the style characteristics and the image editing information, obtaining the complement content consistent with the style of the image to be edited; and complementing the supplementary content to an editing area, performing smoothing treatment on the connection part of the supplementary content and the reserved content in the image to be edited to generate a target image, deleting elements of 'sunflower' in the editing area shown in fig. 2c, and performing smoothing treatment on the connection part of the deleted content and the reserved content to generate the target image, as shown in fig. 2 h.

According to the embodiment of the disclosure, the editing area in the image to be edited is determined, the image editing information is determined based on the AI editing function, the complement content of the editing area is obtained according to the image to be edited and the image editing information, and the object image is obtained by performing content complement on the editing area according to the complement content. In the embodiment of the disclosure, when the editing area is determined, the editing area can be obtained by monitoring and tracking the area selection instruction of the image to be edited, wherein the area selection instruction comprises operations such as smearing or circling, the operability of determining the editing area is higher, and the determination method of the editing area is more flexible; in addition, the AI editing function can be called by directly calling an AI editing component or by a shortcut key, so that the operation is more convenient; the method has the advantages that the image editing information is used for obtaining the complement content, the editing area is complemented with the content to obtain the target image, the editing of the image to be edited is more convenient, more flexibility and variability are provided for the generation of the target image, the editing effect of the image to be edited is good, and the professional capability requirement of a user is low.

Fig. 3 is a flow chart illustrating another image generation method according to an embodiment of the present disclosure. As shown in fig. 3, the method includes, but is not limited to, the steps of:

s301, determining an editing area from an image to be edited.

In the embodiment of the present disclosure, the implementation manner of step S301 may be implemented in any manner of each embodiment of the present disclosure, which is not limited herein, and is not described herein again.

S302, an artificial intelligence AI editing function is called, and image editing information is acquired based on the AI editing function.

In the embodiment of the present disclosure, the implementation manner of step S302 may be implemented in any manner of each embodiment of the present disclosure, which is not limited herein, and is not described in detail.

S303, editing the content of the editing area based on the image editing information to generate a target image.

In the embodiment of the present disclosure, the implementation manner of step S303 may be implemented by any one of the embodiments of the present disclosure, which is not limited herein, and is not described herein again.

S304, monitoring the re-editing instruction, and if the re-editing instruction is monitored, re-editing the image to be edited based on the image editing information or the first updated image editing information.

In some implementations, the re-edit instruction is monitored, and in the embodiments of the present disclosure, a "re-edit" command is set on the interface, and when the user clicks on "re-edit", this indicates that the re-edit instruction is started at this time.

When the re-editing instruction is monitored, the image needs to be edited again, that is, the image to be edited is edited again. Optionally, when the image to be edited is edited again, the image editing information of the image to be edited last time can be modified, that is, the image editing information can be obtained again for editing according to the first updated image editing information.

In the embodiment of the disclosure, by determining the editing area in the image to be edited, determining the image editing information based on the AI editing function, generating the target image according to the image editing information, monitoring the re-editing instruction after generating the target image, and if the re-editing instruction is monitored, re-editing the image to be edited. In the embodiment of the disclosure, the image to be edited is modified again by modifying the image editing information of the image to be edited last time or acquiring new image editing information, so that the flexibility of editing the image to be edited is improved, and the operation of modifying the image to be edited for multiple times is simpler and the use is convenient.

Fig. 4 is a flowchart illustrating another image generation method according to an embodiment of the present disclosure. As shown in fig. 4, the method includes, but is not limited to, the steps of:

s401, determining an editing area from an image to be edited.

In the embodiment of the present disclosure, the implementation manner of step S401 may be implemented in any manner of each embodiment of the present disclosure, which is not limited herein, and is not described herein again.

S402, invoking an artificial intelligence AI editing function, and acquiring image editing information based on the AI editing function.

In the embodiment of the present disclosure, the implementation manner of step S402 may be implemented in any manner of each embodiment of the present disclosure, which is not limited herein, and is not described in detail.

S403, editing the content of the editing area based on the image editing information to generate a target image.

In the embodiment of the present disclosure, the implementation manner of step S403 may be implemented in any manner of each embodiment of the present disclosure, which is not limited herein, and is not described herein again.

S404, if the AI editing function is called again in the process of displaying the target image, the target image is used as a new image to be edited, and the new image to be edited is edited based on the image editing information or the second updated image editing information.

In the process of displaying the target image, if the AI editing function is called again, the target image is required to be edited again.

When the AI editing function is called again, the edited target image is edited again, optionally, the target image can be used as a new image to be edited, and the generated target image can be edited continuously. Alternatively, the target image may be edited based on the last image editing information. Optionally, new second updated image editing information is generated for the target image, and the target image is edited using the second updated image editing information. For example, the image to be edited is an image a, the image editing information 1, and the image a may be edited based on the image editing information 1 to generate a target image, which is labeled as an image B. In the embodiment of the present application, editing of the image B may be continued using the image editing information 1. The icon B may also be edited based on the image editing information 2, and it is understood that the image editing information 2 is the second updated image editing information.

In the embodiment of the disclosure, the editing area in the image to be edited is determined, the image editing information is determined based on the AI editing function, the target image is generated according to the image editing information, and after the target image is generated, the target image can be edited again based on the operation instruction, so that the secondary editing can be performed on the basis of obtaining the target image. In the embodiment of the disclosure, the image editing information of the image to be edited for the first time can be directly modified, or new image editing information can be obtained to modify the target image, so that the operation time for performing secondary editing on the target image is saved, continuous editing is rapidly performed based on the operation instruction of calling the AI editing function, continuous editing on the image can be realized, and the operation is convenient and rapid.

Fig. 5 is a flowchart illustrating another image generation method according to an embodiment of the present disclosure. As shown in fig. 5, the method includes, but is not limited to, the steps of:

s501, determining an editing area from an image to be edited.

In the embodiment of the present disclosure, the implementation manner of step S501 may be implemented in any manner of each embodiment of the present disclosure, which is not limited herein, and is not described in detail.

S502, invoking an artificial intelligence AI editing function, and acquiring image editing information based on the AI editing function.

In the embodiment of the present disclosure, the implementation manner of step S502 may be implemented in any manner of each embodiment of the present disclosure, which is not limited herein, and is not described in detail.

S503, obtaining the generated quantity of the edited images from the image editing information, dividing the image display areas on the interface based on the generated quantity of the images, and loading the images to be edited in each sub-display area respectively.

In some implementations, the configuration sub-area of each editing parameter in the function area of AI editing may further include a configuration of the number of generated images, that is, the parameter configuration may further include the number of generated images.

Alternatively, the image display area on the interface may be divided based on the number of image generation in the image editing information acquisition parameter configuration, that is, when the number of image generation is set to N, the image display area on the interface is divided into N sub-display areas, N being an integer of 1 or more, as shown in fig. 5 a.

It should be noted that, in order to ensure the display effect of each sub-display area on the interface, the value of the image generation number N should not be too large, for example, N may be 4, 6 or 8.

Further, after dividing each sub-display area based on the image generation number, the image to be edited is loaded and displayed in each sub-display area, respectively.

S504, performing image editing on the image to be edited in each sub-display area based on the image editing information.

And carrying out image editing on the images to be edited in each sub-display area based on the image editing information, namely, each image to be edited in each sub-display area corresponds to one target image.

S505, generating a target image, and editing the target image again based on the operation instruction.

Because each image to be edited in the sub-display area can generate one target image based on the image editing information, that is, a plurality of target images, if the operation instruction indicates that the AI editing function is called again, the target image needs to be edited again, and one target image is selected from the plurality of target images to serve as a new image to be edited.

In the embodiment of the present disclosure, the implementation manner of step S505 may be implemented by any one of the embodiments of the present disclosure, which is not limited herein, and is not described herein again.

In the embodiment of the disclosure, by determining the editing area in the image to be edited and determining the image editing information based on the AI editing function, the image editing information further comprises setting the number of the edited images, dividing the image display area into a plurality of sub-display areas based on different image generation numbers, and editing the image to be edited in each sub-display area based on the image editing information to generate the target image. In the embodiment of the disclosure, a plurality of images to be edited can be edited at the same time, the efficiency of image editing is improved, in the process of editing the plurality of images to be edited, the images to be edited are displayed and edited through each sub-display area, the editing condition of each image to be edited is intuitively reflected, and when the target image is edited again, one image to be edited can be selected from the plurality of target images at random, so that the operability and the flexibility are high.

Fig. 6 is a flow chart of another image generation method provided according to an embodiment of the present disclosure. As shown in fig. 6, the method includes, but is not limited to, the steps of:

S601, determining an editing area from an image to be edited.

In the embodiment of the present disclosure, the implementation manner of step S601 may be implemented by any one of the embodiments of the present disclosure, which is not limited herein, and is not described herein again.

S602, invoking an artificial intelligence AI editing function, and acquiring image editing information based on the AI editing function.

In the embodiment of the present disclosure, the implementation manner of step S602 may be implemented in any manner of each embodiment of the present disclosure, which is not limited herein, and is not described in detail.

S603, editing the content of the editing area based on the image editing information to generate a target image.

Optionally, the image generation instruction may also be monitored, and when the image generation instruction is monitored, the first generation progress of the target image is tracked, and the first generation progress is displayed in a floating manner in the image to be edited, as shown in fig. 6a, and 20% of the first generation progress is displayed in a floating manner on the image to be edited, so that the user can intuitively know the editing progress of the image to be edited, and the user experience of image editing is improved.

In the embodiment of the present disclosure, the implementation manner of step S603 may be implemented by any one of the embodiments of the present disclosure, which is not limited herein, and is not described herein again.

S604, generating an editing task of the image to be edited based on the target image and the image editing information, and displaying the editing task in a task display area.

Because each target image has corresponding image editing information, an editing task of the image to be edited can be generated according to the target image and the image editing information corresponding to the target image, and the editing task comprises all editing history parameters of the image to be edited. The editing task is displayed in the task display area, so that editing parameters of the image to be edited can be intuitively reflected and checked.

Alternatively, the editing task may be presented by a task generation icon, that is, the task generation icon is synchronously generated in the task presentation area during the generation of the target image.

When a plurality of target images are generated, a second generation schedule of task generation is comprehensively determined based on the first generation schedule of the target images.

Alternatively, the method for comprehensively determining the second generation progress according to the first generation progress of the plurality of target images may employ: directly averaging, taking an intermediate value, for example, averaging after removing the fastest progress value and the smallest progress value, or selecting the fastest progress or the smallest progress as the second generation progress.

In some implementations, after an image to be edited passes through an image editing model and generates a corresponding target image based on image editing parameters, an editing task may be generated based on the target image and image editing information, further, after the target image is acquired, a thumbnail of the target image may be generated, and the task generation icon is replaced by the thumbnail of the target image for the editing task corresponding to the target image, and an editing record of the image to be edited may be formed through the editing task. The interface comprises a function area, an image display area and a task display area, and the generated editing task can be displayed in the task display area as shown in fig. 6 b. Alternatively, the editing tasks in the task presentation area may be ordered from top to bottom in the order of generation. And the thumbnail of the target image corresponding to the editing task can be displayed in the task display area so as to conveniently and quickly look up the editing information of the target image.

Optionally, after the editing task of the image to be edited is generated, a viewing request of the editing task may be monitored; when the viewing request is monitored, the target image is presented in the image presentation area, and the image editing information is presented in the function area of the AI editing function. That is, the editing parameters of the image to be edited in the process of generating the target image are recorded in the editing task, and when the viewing request of the editing task is monitored, the target images corresponding to different editing tasks can be directly displayed in the image display area, and various editing parameters, namely image editing information, can be displayed in the functional area of the AI editing function.

In the embodiment of the disclosure, the editing area in the image to be edited is determined, the image editing information is determined based on the AI editing function, the corresponding target image is acquired based on the image editing information, after the target image is generated, the editing task of the image to be edited is generated based on the target image and the image editing information, and the editing task is stored and displayed. In the embodiment of the disclosure, the editing task is generated through the image editing information, the task generating icon and the thumbnail are utilized to display in the task display area, the editing parameters of each time are stored and recorded, the corresponding editing parameters and the target images under different editing tasks can be checked by clicking the editing task, the use efficiency of the user is higher, and the user can operate by hand.

Fig. 7 is a block diagram of an image generating apparatus provided according to an embodiment of the present disclosure. As shown in fig. 7, an image generating apparatus 700 of an embodiment of the present disclosure includes: a region determination module 701, an information acquisition module 702, and an image generation module 703.

A region determining module 701, configured to determine an editing region from an image to be edited;

the information acquisition module 702 is configured to invoke an artificial intelligence AI editing function and acquire image editing information based on the AI editing function;

The image generating module 703 is configured to edit the content of the editing area based on the image editing information, and generate a target image.

It should be noted that the above explanation of the data management method embodiment is also applicable to the data management device of the embodiment of the present disclosure, and specific processes are not repeated here.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related user personal information all conform to the regulations of related laws and regulations, and the public sequence is not violated.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

Fig. 8 illustrates a schematic block diagram of an example electronic device 800 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.

Various components in device 800 are connected to I/O interface 805, including: an input unit 806 such as a keyboard, mouse, etc.; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, or the like. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The computing unit 801 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the respective methods and processes described above, for example, an image generation method. For example, in some embodiments, the image generation method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 800 via ROM 802 and/or communication unit 809. When a computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the image generation method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the image generation method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. An image generation method, comprising:

determining an editing area from an image to be edited;

invoking an artificial intelligence AI editing function, and acquiring image editing information based on the AI editing function;

and editing the content of the editing area based on the image editing information to generate a target image.

2. The method according to claim 1, wherein the image editing information includes at least text information describing an expected effect of the editing region, the editing the content of the editing region based on the image editing information, and generating a target image includes:

Obtaining the complement content of the editing area according to the image to be edited and the image editing information;

and carrying out content complementation on the editing area based on the supplementary content to generate the target image.

3. The method according to claim 2, wherein the obtaining the complement of the editing area according to the image to be edited and the image editing information includes:

acquiring a mask image of the image to be edited;

inputting the image to be edited, the mask image and the image editing information into a pre-trained image editing model, and outputting the target image by the image editing model; or alternatively, the process may be performed,

intercepting a local image from the image to be edited, wherein the local image at least comprises an area corresponding to the mask image;

and inputting the local image, the mask image and the image editing information into the image editing model, and outputting the target image by the image editing model.

4. The method according to claim 2, wherein the obtaining the complement content corresponding to the editing area according to the image to be edited, the mask image and the image editing information includes:

Extracting style characteristics of the image to be edited or the local image;

based on the style characteristics and the image editing information, obtaining the complement content consistent with the style of the image to be edited;

the content complement of the editing area based on the supplemental content, generating the target image, includes:

and complementing the supplementary content to the editing area, and performing smoothing processing on the connection part of the supplementary content and the reserved content in the image to be edited to generate the target image.

5. The method of any of claims 1-4, wherein the invoking the artificial intelligence AI edit function comprises:

monitoring a first calling operation of a component corresponding to the AI editing function;

and if the first calling operation is monitored, loading and displaying the function area of the AI editing function in the interface designated area.

6. The method of any of claims 1-4, wherein the invoking the artificial intelligence AI edit function comprises:

monitoring a second calling operation of the calling shortcut key of the AI editing function;

and if the second calling operation is monitored, loading and displaying the function area of the AI editing function in the interface designated area.

7. The method of any of claims 1-4, wherein after the generating the target image, further comprising:

monitoring the re-editing instruction;

and if the re-editing instruction is monitored, re-editing the image to be edited based on the image editing information or the first updated image editing information.

8. The method of any of claims 1-4, wherein after the generating the target image, further comprising:

and in the process of displaying the target image, if the AI editing function is called again, taking the target image as a new image to be edited, and editing the new image to be edited based on the image editing information or the second updated image editing information.

9. The method of any of claims 1-4, wherein the method further comprises:

acquiring the number of the edited images from the image editing information, dividing the image display areas on an interface based on the number of the edited images, and loading the images to be edited in each sub-display area respectively;

and carrying out image editing on the image to be edited in each sub-display area based on the image editing information.

10. The method of claim 9, wherein the method further comprises:

and when the target images are multiple, selecting one from the multiple target images as the new image to be edited.

11. The method of any of claims 1-4, wherein the method further comprises:

and generating an editing task of the image to be edited based on the target image and the image editing information, and displaying the editing task in a task display area.

12. The method of any of claims 1-4, wherein the method further comprises:

monitoring an image generation instruction, if the image generation instruction is monitored, tracking a first generation progress of the target image, and displaying the first generation progress in a floating mode on the image to be edited.

13. The method of claim 11, wherein the method further comprises:

in the process of generating the target image, synchronously generating task generating icons in the task display area;

and when the plurality of generated target images are generated, comprehensively determining a second generation progress generated by the task based on the first generation progress of the target images.

14. The method of claim 11, wherein after the target image is generated, further comprising:

and acquiring a thumbnail of the target image, and replacing the task generation icon with the thumbnail.

15. The method of claim 13, wherein the generating the editing task of the image to be edited further comprises:

monitoring the viewing request of the editing task;

and when the viewing request is monitored, displaying the target image in an image display area, and displaying the image editing information in a functional area of the AI editing function.

16. The method of any of claims 1-4, wherein the method further comprises:

when the AI editing function is called, a function area of the AI editing function is displayed in a designated area of an interface, wherein the function area comprises configuration subareas of all editing parameters;

acquiring input operation aiming at a text configuration subarea, and determining the text information according to the input operation; and/or the number of the groups of groups,

and carrying out parameter configuration operation on the rest configuration subareas in the functional area to generate image editing auxiliary information.

17. The method according to any one of claims 1-4, wherein before the selecting an edit region from the image to be edited, further comprising:

Receiving an image selection request, selecting an image from an image library according to the image selection request, and taking the selected image as the image to be edited; or alternatively, the process may be performed,

and in the process of browsing the candidate images, responding to the AI editing function, taking the candidate images as the images to be edited.

18. The method of any of claims 1-4, wherein the determining an edit area from the image to be edited comprises;

displaying the image to be edited in an image display area of an interface;

monitoring the region selection instruction of the image to be edited, and if the region selection instruction is monitored, tracking the region selection instruction to obtain the editing region.

19. An image generating apparatus, comprising:

the area determining module is used for determining an editing area from the image to be edited;

the information acquisition module is used for calling an artificial intelligence AI editing function and acquiring image editing information based on the AI editing function;

and the image generation module is used for editing the content of the editing area based on the image editing information to generate a target image.

20. The apparatus of claim 19, wherein the image editing information includes at least text information describing an expected effect of the editing region, and the image generating module includes:

21. The apparatus of claim 20, wherein the image generation module comprises:

acquiring a mask image of the image to be edited;

22. The apparatus of claim 19, wherein the image generation module comprises:

extracting style characteristics of the image to be edited or the local image;

23. The apparatus of any of claims 19-22, wherein the information acquisition module comprises:

24. The apparatus of any of claims 19-22, wherein the information acquisition module comprises:

25. The apparatus of any of claims 19-22, wherein the image generation module is further followed by:

monitoring the re-editing instruction;

26. The apparatus of any of claims 19-22, wherein the image generation module is further followed by:

27. The apparatus of any of claims 19-22, wherein the apparatus further comprises:

28. The apparatus of claim 27, wherein the apparatus further comprises:

29. The apparatus of any of claims 19-22, wherein the apparatus further comprises:

30. The apparatus of any of claims 19-22, wherein the apparatus further comprises:

31. The apparatus of claim 29, wherein the apparatus further comprises:

32. The apparatus of claim 29, wherein after the image generation module, further comprising:

33. The apparatus of claim 31, wherein the apparatus further comprises:

Monitoring the viewing request of the editing task;

34. The apparatus of any of claims 19-22, wherein the apparatus further comprises:

35. The apparatus of any of claims 19-22, wherein prior to the region determination module, further comprising:

36. The apparatus of any of claims 19-22, wherein the region determination module comprises;

displaying the image to be edited in an image display area of an interface;

37. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-18.

38. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-18.

39. A computer program product comprising a computer program which, when executed by a processor, implements the steps of the method according to any of claims 1-18.