CN117333573A - AIGC-based image light and shadow processing method, device, equipment and storage medium - Google Patents

AIGC-based image light and shadow processing method, device, equipment and storage medium Download PDF

Info

Publication number
CN117333573A
CN117333573A CN202311531390.0A CN202311531390A CN117333573A CN 117333573 A CN117333573 A CN 117333573A CN 202311531390 A CN202311531390 A CN 202311531390A CN 117333573 A CN117333573 A CN 117333573A
Authority
CN
China
Prior art keywords
image
shadow
processed
light
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311531390.0A
Other languages
Chinese (zh)
Inventor
王凡祎
苏婧文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to CN202311531390.0A priority Critical patent/CN117333573A/en
Publication of CN117333573A publication Critical patent/CN117333573A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/001Texturing; Colouring; Generation of texture or colour
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/60Editing figures and text; Combining figures or text
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The application discloses an AIGC-based image light and shadow processing method, an AIGC-based image light and shadow processing device, electronic equipment and a storage medium, wherein the AIGC-based image light and shadow processing method comprises the following steps: acquiring an image to be processed and a shadow template image; extracting features of the image to be processed to obtain whole image features, color texture features and contour edge features corresponding to the image to be processed; processing the image to be processed according to the shadow template image to obtain an intermediate image corresponding to the image to be processed; and processing the intermediate image according to the image to be processed, the whole image characteristic, the color texture characteristic and the contour edge characteristic to obtain an output image corresponding to the image to be processed. The method and the device realize that in the image light and shadow processing process, the original image is subjected to light and shadow editing to obtain an intermediate image containing the light and shadow effect, and then the intermediate image is supplemented by the characteristic information contained in the original image, so that the light and shadow editing effect is more real, and meanwhile, the detail characteristics of the original image are also reserved.

Description

AIGC-based image light and shadow processing method, device, equipment and storage medium
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to an AIGC-based image light and shadow processing method, apparatus, electronic device, and storage medium.
Background
Along with the development of photographing and photography culture, in order to meet the demands of users, the use of the element, namely the light and shadow, is more frequent, for example, the editing and increasing of the light and shadow effect are carried out on the image, so that the user can obtain the image with the special light and shadow effect through the processing of the image in a simpler or conventional scene.
However, when editing and increasing the light effect on an image, particularly in the image processing process based on the generated artificial intelligence (Artificial Intelligence Generated Content, AIGC), the light image is usually directly covered on the original image after being processed, and then the obtained image is re-optimized to obtain the image with the light effect.
Disclosure of Invention
An objective of the embodiments of the present application is to provide an AIGC-based image light and shadow processing method, apparatus, electronic device, and storage medium, so as to solve a technical problem that in a related art, original image feature details cannot be guaranteed not to be changed when editing a light and shadow effect.
In a first aspect, an embodiment of the present application provides an AIGC-based image light and shadow processing method, including:
Acquiring an image to be processed and a shadow template image;
extracting features of the image to be processed to obtain whole image features, color texture features and contour edge features corresponding to the image to be processed;
processing the image to be processed according to the shadow mask image map to obtain an intermediate image corresponding to the image to be processed;
and processing the intermediate image according to the image to be processed, the whole image feature, the color texture feature and the contour edge feature to obtain an output image corresponding to the image to be processed.
In a second aspect, an embodiment of the present application provides an AIGC-based image light and shadow processing apparatus, including:
the image acquisition module is used for acquiring an image to be processed and a shadow template image;
the feature extraction module is used for carrying out feature extraction on the image to be processed to obtain integral image features, color texture features and contour edge features corresponding to the image to be processed;
the first processing module is used for processing the image to be processed according to the light and shadow mask image to obtain an intermediate image corresponding to the image to be processed;
and the second processing module is used for processing the intermediate image according to the image to be processed, the whole image characteristic, the color texture characteristic and the contour edge characteristic to obtain an output image corresponding to the image to be processed.
In a third aspect, an embodiment of the present application provides an electronic device, where the electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, where the steps in the AIGC-based image light and shadow processing method described in any one of the foregoing are implemented when the processor executes the computer program.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium storing a computer program, which when executed by a processor, implements the steps in the AIGC-based image light and shadow processing method of any one of the above.
The embodiment of the application provides an AIGC-based image light and shadow processing method, an AIGC-based image light and shadow processing device, electronic equipment and a storage medium, when editing and adjusting the light and shadow effect of an image based on generated artificial intelligence, firstly, an image to be processed and a used light and shadow template image which are edited and adjusted are obtained, then, characteristic extraction is carried out on the image to be processed, different characteristic information corresponding to the image to be processed is obtained, including integral image characteristics, color texture characteristics and contour edge characteristics, and when the image is processed, the image to be processed is firstly processed for the first time by utilizing the light and shadow template image, so that an intermediate image containing light and shadow information is obtained. In the image light and shadow processing process, the original image is subjected to light and shadow editing without considering the characteristic information of the original image, an intermediate image containing the light and shadow effect is obtained, and then the characteristic information contained in the original image is used for supplementing the characteristic information of the intermediate image, so that the light and shadow editing effect is more real, and meanwhile, the detail characteristics of the original image are maintained.
Drawings
Fig. 1 is a schematic flow chart of an AIGC-based image light and shadow processing method according to an embodiment of the present disclosure;
FIG. 2 is a schematic flow chart of the steps for obtaining an intermediate image according to the embodiments of the present application;
FIG. 3 is a schematic flow chart of a step of obtaining an intermediate image according to an embodiment of the present disclosure;
FIG. 4 is a schematic flow chart of a step of obtaining an intermediate image according to an embodiment of the present disclosure;
FIG. 5 is a flowchart illustrating steps for adjusting an AIGC-based image lighting effect according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an AIGC-based image light and shadow processing apparatus according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
fig. 8 is another schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order and/or performed in parallel. Furthermore, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.
The term "including" and variations thereof as used herein are intended to be open-ended, i.e., including, but not limited to. The term "based on" is based at least in part on. The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments. Related definitions of other terms will be given in the description below.
In the related art, when editing and increasing the light and shadow effect on an image, particularly in the image processing process based on the generated artificial intelligence (Artificial Intelligence Generated Content, AIGC), the light and shadow image is generally directly covered on the original image after being processed, and then the image obtained at this time is re-optimized to obtain the image with the light and shadow effect, and the application of this method can cause the characteristics and details of the image to be affected, so as to affect the authenticity of the image.
In order to solve the technical problems in the related art, an embodiment of the present application provides an AIGC-based image light and shadow processing method, please refer to fig. 1, fig. 1 is a schematic flow chart of the AIGC-based image light and shadow processing method provided in the embodiment of the present application, and the method includes steps 101 to 104.
And step 101, acquiring an image to be processed and a shadow template image.
In an embodiment, when adjusting and processing the light and shadow effect, firstly, an image to be processed for adjusting and processing the light and shadow effect is obtained, and a basis for adjusting and processing the light and shadow effect, namely, a light and shadow template image is obtained, and then, when processing, the light and shadow of the image to be processed is adjusted by utilizing the light and shadow template image according to a corresponding processing mode.
Illustratively, during the actual processing, the light and shadow template images may be pre-stored for selection and use in a corresponding template library, or may be uploaded in real time during the light and shadow effect adjustment process. Accordingly, in the process of acquiring an image to be processed and a shadow template image, it includes: receiving an uploaded image to be processed under the condition of receiving an image light and shadow adjustment instruction; and under the condition that a light and shadow template selection instruction is received, selecting in a light and shadow template library, and determining a light and shadow template image for performing light and shadow processing on the image to be processed.
Specifically, in the process of processing, when an image light and shadow adjustment instruction is received, it is determined that the adjustment processing of the light and shadow effect of the image can be performed, at this time, an input image to be processed is received, and when a light and shadow template image used in the process of performing light and shadow adjustment is determined, a selection can be made from a preset light and shadow template library, and when the light and shadow template selection instruction is received, the light and shadow template library is called and template display, and then the light and shadow template image for performing the light and shadow processing is determined by selection.
For a preset shadow template library, a plurality of shadow template images are stored, each shadow template image has corresponding identification information, and when the shadow template images are selected, a user can find the currently required shadow template images from the shadow template library by inputting the corresponding identification information and then matching the identification information.
Further, as can be seen from the foregoing description, the light and shadow template image may be selected in advance, or may be uploaded in real time during use, so that when determining the light and shadow template image, the method further includes: under the condition that a light and shadow template uploading instruction is received, receiving an uploaded first image, analyzing and processing the first image, and determining the image type of the first image, wherein the image type comprises a light and shadow image and a non-light and shadow image; in the case that the image type of the first image is determined to be a shadow image, taking the first image as a shadow template image; and under the condition that the image type of the first image is determined to be a non-shadow image, extracting shadow information from the first image, and taking a second image corresponding to the extracted shadow information as a shadow template image.
That is, in the processing procedure, if it is determined that selection is not currently performed in the light and shadow template library, determination of the light and shadow template image may be completed through a corresponding uploading operation. Specifically, in the case of receiving a light and shadow template uploading instruction, receiving an uploaded first image, wherein the image type of the first image can be a light and shadow image and a non-light and shadow image, then determining whether the first image can be directly used as the light and shadow template image or not by judging the image type of the first image, wherein in the case of determining that the image type of the first image is the light and shadow image, the first image is directly used as the light and shadow template image, and in the case of determining that the image type of the first image is the non-light and shadow image, further processing is needed to obtain the light and shadow template image.
Illustratively, when the light shadow template image is determined according to the image uploaded by the user, and when the uploaded light shadow image needs to be processed to obtain the light shadow template image, the light shadow information contained in the first image is sampled and acquired, and then the currently required light shadow template image is generated according to the obtained light shadow information.
In practical applications, the uploaded first image may be a real image including a shadow effect, and useful information for the first image is shadow information in the first image, and the used shadow information may be acquired in the first image based on a back process of the shadow effect addition, so as to obtain a second image generated based on the shadow information, and the obtained second image is used as a shadow template image.
And 102, extracting features of the image to be processed to obtain whole image features, color texture features and contour edge features corresponding to the image to be processed.
In an embodiment, after the image to be processed and the shadow template image are obtained, the adjustment processing of the shadow effect may be performed on the image to be processed by using the shadow template image. In the actual processing process, firstly, feature extraction is performed on the image to be processed to obtain the whole image feature, the color texture feature and the contour edge feature contained in the image to be processed for subsequent light and shadow adjustment.
For example, when the feature extraction is performed on the image to be processed, the image to be processed may be processed using a feature extraction model trained in advance, and the feature extraction model used may be obtained using a feature extraction model constructed based on a control network (control network) in the case of generating artificial intelligence (Artificial Intelligence Generated Content, AIGC). And for the feature extraction model used at the moment, the control net can be trained and optimized based on different conditions, so that the feature extraction model obtained under different conditions has the capability of extracting different features.
It should be noted that, for the feature extraction model used for feature extraction, the feature extraction model may be composed of a plurality of sub-models, each sub-model is used for extracting one feature of the whole graph feature, the color texture feature and the contour edge feature, and then all the sub-models are integrated in a coding manner to obtain a feature extraction model, so that when the feature extraction is performed, different feature information including the whole graph feature, the color texture feature and the contour edge feature can be obtained for the image output to be processed.
And step 103, processing the image to be processed according to the shadow template image to obtain an intermediate image corresponding to the image to be processed.
In an embodiment, after the image to be processed and the shadow template image are obtained, besides extracting features to obtain feature information under different requirements, the shadow template image is used for processing the image to be processed, and an intermediate image corresponding to the image to be processed is obtained when the processing is completed.
It should be noted that, after the image to be processed and the shadow template image are obtained, the step of extracting the features and the step of obtaining the intermediate image may be performed without distinction, that is, the sequence of the step 102 and the step 103 is not fixed, and may be performed simultaneously except for a certain sequence, which is not specifically limited.
Illustratively, when obtaining the intermediate image, the intermediate image is obtained by performing the first processing using the shadow template image without considering details such as color textures of the image to be processed, and when obtaining the intermediate image, reference may be made to fig. 2, and fig. 2 is a schematic flow chart of steps for obtaining the intermediate image provided in the embodiment of the present application, where the steps include steps 201 to 203.
Step 201, extracting features of the shadow template image to obtain corresponding shadow features of the shadow template image;
step 202, extracting description information of an image to be processed to obtain text description corresponding to the image to be processed;
and 203, carrying out image generation and image processing on the image to be processed according to the shadow characteristics and the text description to obtain an intermediate image corresponding to the image to be processed.
Specifically, when an image to be processed is processed for the first time according to a shadow template image, firstly, extracting features of the shadow template image to obtain shadow features corresponding to the shadow template image, meanwhile, extracting description information of the image to be processed to obtain text description of the image to be processed, and finally, performing the first time processing on the image to be processed by utilizing the shadow features and the text description to obtain an intermediate image corresponding to the image to be processed.
For example, when extracting the shadow features of the shadow template image, the feature extraction model can be trained in advance, and the feature extraction model used at the moment can also be constructed based on the control net, so that the feature extraction of the image to be processed and the shadow template image can be completed through the whole feature extraction model for the whole control net network. Meanwhile, when extracting the text description of the Image to be processed, a multi-mode large model (Bootstrapping Language-Image Pre-tracking, BLIP) may be used to process the Image to be processed, so as to generate the text description of the Image to be processed.
And further, after the light and shadow characteristics and the text description are obtained, the image to be processed is used as an initial image to be processed at the moment, the image to be processed is processed by utilizing the light and shadow characteristics and the text description by utilizing the image generation technology, and a corresponding intermediate image is obtained after the processing is completed. And when the graphic processing is performed, the graphic processing method can be realized based on stable diffusion.
Further, when the image to be processed is processed according to the light and shadow template image, different processing may be performed according to different requirements, for example, the processing manner described in steps 201 to 203, that is, the obtained light and shadow template image is directly given to the image to be processed, but in the actual processing process, the image with the light and shadow effect adjusted may already have the light and shadow effect, at this time, the light and shadow effect may be removed first, and then the required light and shadow effect is added to the image to be processed, so at this time, when the processing is performed, reference may be made to fig. 3, and fig. 3 is a further flowchart of the step of obtaining the intermediate image provided in the embodiment of the present application, where the step includes steps 301 to 303.
Step 301, determining whether to delete the current shadow of the image to be processed according to the received instruction information;
step 302, under the condition that the current light shadow of the image to be processed is determined to be reserved, carrying out image generation and image processing on the image to be processed according to the light shadow characteristics and the text description to obtain an intermediate image;
and 303, under the condition that the current light shadow of the image to be processed is determined to be deleted, performing light shadow deletion processing on the image to be processed, and performing image generation graph processing on the image to be processed after the deletion processing is completed according to the light shadow characteristics and the text description, so as to obtain an intermediate image.
In the processing, whether the current light shadow of the image to be processed is deleted is determined according to the received instruction information, and steps 201 to 203 are directly executed under the condition that the current light shadow of the image to be processed is determined to be reserved, that is, the image to be processed is subjected to image generation processing according to the light shadow characteristics and the text description to obtain the corresponding intermediate image, and under the condition that the current light shadow of the image to be processed is determined to be deleted, the image to be processed is firstly required to be subjected to the light shadow deletion processing, and then the image to be processed after the deletion processing is performed on the image to be processed according to the light shadow characteristics and the text description to obtain the corresponding intermediate image.
In the processing process, the image to be processed can be the real image processed for the first time or the real image processed by multiple times of light shadow editing, so that when the image to be processed is processed, whether the current light shadow contained in the image to be processed is reserved or not can be determined according to the actual requirement of a user, and under the condition that the current light shadow is not required to be reserved, the deletion processing is performed on the current light shadow, otherwise, the current light shadow in the image to be processed is ignored, and direct processing is performed, wherein the neglecting of the current light shadow here means that the light shadow is reserved.
When deleting the current light shadow included in the image to be processed, the method is similar to the method described in the previous step for extracting the second image as the light shadow image from the first image, except that the former is to obtain the light shadow image, and the light shadow image is removed, so that the description is omitted.
In addition, in the case of preserving the current light shadow of the image to be processed, it is explained that the light shadow deletion process is not performed on the image to be processed at this time, and therefore, when the current light shadow of the image to be processed is preserved and the intermediate image is obtained by processing, reference may be made to fig. 4, and fig. 4 is a further flowchart of the step of obtaining the intermediate image provided in the embodiment of the present application, where the step includes steps 401 to 404.
Step 401, under the condition of determining the current shadow of the reserved image to be processed, extracting the current shadow image of the image to be processed, and obtaining a new image to be processed which does not contain the current shadow image;
step 402, extracting image description of a new image to be processed to obtain new text description corresponding to the new image to be processed;
step 403, fusing the current shadow image and the shadow template image to obtain a new shadow template image;
and step 404, carrying out image generation and image processing on the new image to be processed according to the new shadow template image and the text description to obtain an intermediate image.
Specifically, when the intermediate image is obtained by processing, the processing is performed under the condition that the current light shadow of the image to be processed is determined, and then when the processing is performed, the extraction of the light shadow image of the image to be processed is performed first to obtain the current light shadow image corresponding to the image to be processed, then the new image to be processed is obtained by deleting the current light shadow image in the image to be processed, the extraction of the text description of the new image to be processed is performed to obtain the new text description corresponding to the new image to be processed, and meanwhile, the obtained current light shadow image and the light shadow template image obtained in advance can be processed together to obtain the new light shadow template image, and finally the image to be processed is subjected to the image generation processing according to the new light shadow template image and the new text description to obtain the intermediate image.
For example, the text descriptions of different images are different, and at the same time, the text descriptions of the same image with different shadows are also different, so in order to determine the accuracy of the shadow editing and adjustment, when determining that the shadow is included in the image to be processed, the image to be processed needs to be correspondingly processed, so as to obtain the current shadow image and the image to be processed without the shadow. And when the image to be processed does not contain the light shadow, the image to be processed is processed directly by utilizing the light shadow characteristics of the light shadow template image and the text description of the image to be processed, and an intermediate image is generated.
And 104, processing the intermediate image according to the image to be processed, the whole image feature, the color texture feature and the contour edge feature to obtain an output image corresponding to the image to be processed.
In an embodiment, after feature extraction and first processing of the image to be processed are completed, the intermediate image is processed according to all the obtained features, so as to obtain an output image with the light and shadow editing adjustment completed. Specifically, when processing is performed, the intermediate image is used as an initial image to be processed currently, and the image to be processed, the whole image feature, the color texture feature and the contour edge feature are utilized to perform image generation processing on the intermediate image so as to output and obtain an output image corresponding to the image to be processed.
For example, when the intermediate image is processed, besides the related features obtained by extracting the features, the text description corresponding to the image to be processed is used as a processing basis, the intermediate image without considering the color texture details is used as an initial image to be processed at the moment, the contour, texture and color feature information of the obtained image to be processed are used for processing the intermediate image, and the color texture details and the like which are not considered in the first processing are supplemented, so that the detail information of the image to be processed can be completely reserved and not modified.
Further, when the output image is obtained by performing the graphically-generated image processing according to the obtained features, the image may be adjusted by interaction with the user, for example, adjustment of brightness, contrast, and the like, so that when the output image is obtained by performing the processing, the method includes: acquiring text description corresponding to an image to be processed; carrying out image generation processing on the intermediate image according to the text description, the whole image feature, the color texture feature and the contour edge feature to obtain a processed image to be processed; and under the condition that an image adjusting instruction is received, adjusting the processed image to be processed according to the image adjusting instruction to obtain an output image.
When the image adjustment instruction is received, the image obtained by the completion of the supplementation is adjusted in response to the image adjustment instruction, so that an output image is obtained when the adjustment is completed, and when the adjustment is not needed, the image obtained by the completion of the supplementation is directly used as the output image to be output and displayed.
Further, referring to fig. 5, fig. 5 is a flowchart illustrating steps of adjusting an image light effect based on an AIGC according to an embodiment of the present application.
In the processing shown in fig. 5, when it is determined to start adjusting the light effect on the image, it includes:
s1, receiving an input original image and a shadow template image;
s2, obtaining text description of the original image based on BLIP, and obtaining text description corresponding to the original image;
s3, extracting the light and shadow characteristics of the light and shadow template image by utilizing a control net-bright model to obtain the light and shadow characteristics corresponding to the light and shadow template image;
S4, extracting different characteristic information contained in the original image by using different models;
wherein, when extracting the characteristics of the original image, the method comprises the following steps: the whole image feature of the original image is extracted by using a control net-reference model, the color texture feature of the original image is extracted by using a control net-tile model, and the contour edge feature of the original image is extracted by using a control net-linear model. And each model can be performed simultaneously without limiting the sequence when extracting the characteristics.
It should be noted that, the control net-bright model, the control net-reference model, the control net-tile model and the control net-linear model are respectively obtained by training the control net for four conditions (conditions), and different models which are not included in 15 are included in the control net, so that a model combination based on the model 4 for processing is obtained through a large number of experiments, the obtained output image has a better light effect, and the change of the detailed features of the original image can be better avoided.
S5, processing by using a stablediffusion graph algorithm according to the text description and the shadow characteristics, and finishing the first processing to obtain an intermediate image with a shadow effect;
And S6, processing the intermediate image of the characteristic information and the to-be-light shadow effect obtained by processing the original image according to different models by using a stabledifusion graph algorithm, and finishing the second processing to obtain a final output result.
In combination with the steps S5 and S6, when the actual processing is performed, the processing procedure may be determined to be divided into two parts, where the first part is to perform the first processing on the original image according to the light and shadow characteristics and the text description, the specific processing may be performed by using a stabledifusion algorithm, the original image is used as an initial image of the first processing, an intermediate image with the light and shadow effect is obtained without considering the detail characteristics of the original image, and the second part is to perform the second processing on the intermediate image with the light and shadow effect according to the characteristic information and the text description of the original image, and the second part may also perform the processing by using a stabledifusion algorithm, and use the intermediate image as an initial image of the second processing, and supplement the characteristic information obtained based on a control network, such as the color and sharpening of the person, the image, the contour and the edge of the image in the image, and the like, so as to obtain the supplementary image containing the light and shadow effect after completing the supplementary image with all the characteristic information.
It should be noted that, the execution sequence of S2, S3 and S4 is not fixed, and may be performed simultaneously, or may have a certain sequence, but the sequence is not limited.
In summary, the application discloses an image light and shadow processing method based on AIGC, when editing and adjusting the light and shadow effect of an image based on generating artificial intelligence, firstly, obtaining an image to be processed for editing and adjusting and a light and shadow template image used, then, carrying out feature extraction on the image to be processed to obtain different feature information corresponding to the image to be processed, including integral image features, color texture features and contour edge features, and when processing the image, firstly, carrying out first processing on the image to be processed by using the light and shadow template image to obtain an intermediate image containing light and shadow information, and because the first processing does not consider the image features of the image to be processed, after obtaining the intermediate image, carrying out feature supplementation on the intermediate image by using the obtained integral image features, color texture features and contour edge features to obtain an output image corresponding to the image to be processed. In the image light and shadow processing process, the original image is subjected to light and shadow editing without considering the characteristic information of the original image, an intermediate image containing the light and shadow effect is obtained, and then the characteristic information contained in the original image is used for supplementing the characteristic information of the intermediate image, so that the light and shadow editing effect is more real, and meanwhile, the detail characteristics of the original image are maintained.
According to the method described in the above embodiments, the present embodiment will be further described from the viewpoint of an AIGC-based image light and shadow processing apparatus, which may be implemented as a separate entity or may be implemented as an integrated electronic device, such as a terminal, which may include a mobile phone, a tablet computer, or the like.
Referring to fig. 6, fig. 6 is a schematic structural diagram of an AIGC-based image light processing apparatus according to an embodiment of the present application, and as shown in fig. 6, the AIGC-based image light processing apparatus 600 according to an embodiment of the present application includes:
an image acquisition module 601, configured to acquire an image to be processed and a shadow template image;
the feature extraction module 602 is configured to perform feature extraction on an image to be processed to obtain a whole image feature, a color texture feature and a contour edge feature corresponding to the image to be processed;
a first processing module 603, configured to process an image to be processed according to the shadow template image, to obtain an intermediate image corresponding to the image to be processed;
the second processing module 604 is configured to process the intermediate image according to the image to be processed, the whole image feature, the color texture feature, and the contour edge feature, and obtain an output image corresponding to the image to be processed.
In an embodiment, the image acquisition module 601 is further configured to:
receiving an uploaded image to be processed under the condition of receiving an image light and shadow adjustment instruction;
and under the condition that a light and shadow template selection instruction is received, selecting in a light and shadow template library, and determining a light and shadow template image for performing light and shadow processing on the image to be processed.
In an embodiment, the image acquisition module 601 is further configured to:
under the condition that a light and shadow template uploading instruction is received, receiving an uploaded first image, analyzing and processing the first image, and determining the image type of the first image, wherein the image type comprises a light and shadow image and a non-light and shadow image;
in the case that the image type of the first image is determined to be a shadow image, taking the first image as a shadow template image;
and under the condition that the image type of the first image is determined to be a non-shadow image, extracting shadow information from the first image, and taking a second image corresponding to the extracted shadow information as a shadow template image.
In an embodiment, the first processing module 603 is further configured to:
extracting features of the shadow template image to obtain shadow features corresponding to the shadow template image;
extracting description information of the image to be processed to obtain text description corresponding to the image to be processed;
And carrying out graphic generation on the image to be processed according to the light and shadow characteristics and the text description to obtain an intermediate image corresponding to the image to be processed.
In an embodiment, the first processing module 603 is further configured to:
determining whether to delete the current light shadow of the image to be processed according to the received instruction information;
under the condition that the current light shadow of the image to be processed is reserved is determined, carrying out image generation and image processing on the image to be processed according to the light shadow characteristics and the text description to obtain an intermediate image;
and under the condition that the current light shadow of the image to be processed is deleted, performing light shadow deletion processing on the image to be processed, and performing image generation processing on the image to be processed after the deletion processing is completed according to the light shadow characteristics and the text description to obtain an intermediate image.
In an embodiment, the first processing module 603 is further configured to:
under the condition that the current light shadow of the reserved to-be-processed image is determined, extracting the current light shadow image of the to-be-processed image, and obtaining a new to-be-processed image which does not contain the current light shadow image;
extracting the image description of the new image to be processed to obtain a new text description corresponding to the new image to be processed;
fusing the current shadow image and the shadow template image to obtain a new shadow template image;
And carrying out image generation and image processing on the new image to be processed according to the new shadow template image and the new text description to obtain an intermediate image.
In an embodiment, the second processing module 604 is further configured to:
acquiring text description corresponding to an image to be processed;
carrying out image generation processing on the intermediate image according to the text description, the whole image feature, the color texture feature and the contour edge feature to obtain a processed image to be processed;
and under the condition that an image adjusting instruction is received, adjusting the processed image to be processed according to the image adjusting instruction to obtain an output image.
In addition, referring to fig. 7, fig. 7 is a schematic structural diagram of an electronic device provided in an embodiment of the present application, where the electronic device may be a mobile terminal, such as a smart phone, a tablet computer, or the like. As shown in fig. 7, the electronic device 700 includes a processor 701, a memory 702. The processor 701 is electrically connected to the memory 702.
The processor 701 is a control center of the electronic device 700, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device 700 and processes data by running or loading application programs stored in the memory 702, and calling data stored in the memory 702, thereby performing overall monitoring of the electronic device 700.
In this embodiment, the processor 701 in the electronic device 700 loads instructions corresponding to the processes of one or more application programs into the memory 702 according to the following steps, and the processor 701 executes the application program stored in the memory 702, so as to implement any step of the AIGC-based image light and shadow processing method provided in the foregoing embodiments.
The electronic device 700 may implement the steps in any embodiment of the image light and shadow processing method based on the AIGC provided in the embodiments of the present application, so that the beneficial effects that any one of the image light and shadow processing method based on the AIGC provided in the embodiments of the present application can implement are detailed in the previous embodiments, and are not described herein again.
Referring to fig. 8, fig. 8 is another schematic structural diagram of an electronic device provided in an embodiment of the present application, and fig. 8 is a specific structural block diagram of the electronic device provided in the embodiment of the present application, where the electronic device may be used to implement the image light and shadow processing method based on the AIGC provided in the embodiment described above. The electronic device 800 may be a mobile terminal such as a smart phone or a notebook computer.
The RF circuit 810 is configured to receive and transmit electromagnetic waves, and to perform mutual conversion between the electromagnetic waves and the electrical signals, thereby communicating with a communication network or other devices. RF circuitry 810 may include various existing circuit elements for performing these functions, such as an antenna, a radio frequency transceiver, a digital signal processor, an encryption/decryption chip, a Subscriber Identity Module (SIM) card, memory, and so forth. The RF circuitry 810 may communicate with various networks such as the internet, intranets, wireless networks, or other devices via wireless networks. The wireless network may include a cellular telephone network, a wireless local area network, or a metropolitan area network. The wireless network may use various communication standards, protocols, and technologies including, but not limited to, global system for mobile communications (Global System for Mobile Communication, GSM), enhanced mobile communications technology (Enhanced Data GSM Environment, EDGE), wideband code division multiple access technology (Wideband Code Division Multiple Access, WCDMA), code division multiple access technology (Code Division Access, CDMA), time division multiple access technology (Time Division Multiple Access, TDMA), wireless fidelity technology (Wireless Fidelity, wi-Fi) (e.g., institute of electrical and electronics engineers standards IEEE 802.11a,IEEE 802.11b,IEEE802.11g and/or IEEE802.11 n), internet telephony (Voice over Internet Protocol, voIP), worldwide interoperability for microwave access (Worldwide Interoperability for Microwave Access, wi-Max), other protocols for mail, instant messaging, and short messaging, as well as any other suitable communication protocols, even including those not currently developed.
The memory 820 may be used to store software programs and modules, such as program instructions/modules corresponding to the AIGC-based image light processing method in the above-described embodiments, and the processor 880 executes various functional applications and image light processing methods for the AIGC-based image by running the software programs and modules stored in the memory 820.
Memory 820 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 820 may further include memory located remotely from processor 880, which may be connected to electronic device 800 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input unit 830 may be used to receive input numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. In particular, the input unit 830 may include a touch-sensitive surface 831 as well as other input devices 832. The touch-sensitive surface 831, also referred to as a touch screen or touch pad, may collect touch operations thereon or thereabout by a user (e.g., operations of the user on the touch-sensitive surface 831 or thereabout by using any suitable object or accessory such as a finger, stylus, etc.), and actuate the corresponding connection device according to a predetermined program. Alternatively, touch-sensitive surface 831 can include both a touch detection device and a touch controller. The touch detection device detects the touch azimuth of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 880 and can receive commands from the processor 880 and execute them. In addition, the touch-sensitive surface 831 can be implemented using a variety of types, such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch-sensitive surface 831, the input unit 830 may also include other input devices 832. In particular, other input devices 832 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, mouse, joystick, etc.
The display unit 840 may be used to display information entered by a user or provided to a user as well as various graphical user interfaces of the electronic device 800, which may be composed of graphics, text, icons, video, and any combination thereof. The display unit 840 may include a display panel 841, and optionally, the display panel 841 may be configured in the form of an LCD (Liquid Crystal Display ), an OLED (Organic Light-Emitting Diode), or the like. Further, touch-sensitive surface 831 can overlay display panel 841, and upon detection of a touch operation thereon or thereabout by touch-sensitive surface 831, is communicated to processor 880 for determining the type of touch event, whereupon processor 880 provides a corresponding visual output on display panel 841 based on the type of touch event. Although in the figures, touch-sensitive surface 831 and display panel 841 are implemented as two separate components, in some embodiments touch-sensitive surface 831 may be integrated with display panel 841 to implement input and output functions.
The electronic device 800 may also include at least one sensor 850, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor that may adjust the brightness of the display panel 841 according to the brightness of ambient light, and a proximity sensor that may generate an interrupt when the folder is closed or closed. As one of the motion sensors, the gravity acceleration sensor can detect the acceleration in all directions (generally three axes), and can detect the gravity and the direction when the mobile phone is stationary, and can be used for applications of recognizing the gesture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and knocking), and the like; other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that may also be configured with the electronic device 800 are not described in detail herein.
Audio circuitry 860, speakers 861, and microphone 862 may provide an audio interface between the user and the electronic device 800. The audio circuit 860 may transmit the received electrical signal converted from audio data to the speaker 861, and the electrical signal is converted into a sound signal by the speaker 861 to be output; on the other hand, the microphone 862 converts the collected sound signals into electrical signals, which are received by the audio circuit 860 and converted into audio data, which are processed by the audio data output processor 880 and transmitted to, for example, another terminal via the RF circuit 810, or which are output to the memory 820 for further processing. Audio circuitry 860 may also include an ear bud jack to provide communication of peripheral headphones with electronic device 800.
The electronic device 800, via the transmission module 870 (e.g., wi-Fi module), may facilitate user reception of requests, transmission of information, etc., that provides wireless broadband internet access to the user. Although the transmission module 870 is shown in the figures, it is understood that it is not a necessary component of the electronic device 800 and may be omitted entirely as desired within the scope of not changing the essence of the invention.
The processor 880 is a control center of the electronic device 800, connects various parts of the entire cellular phone using various interfaces and lines, and performs various functions of the electronic device 800 and processes data by running or executing software programs and/or modules stored in the memory 820, and calling data stored in the memory 820, thereby performing overall monitoring of the electronic device. Optionally, processor 880 may include one or more processing cores; in some embodiments, processor 880 may integrate an application processor that primarily handles operating systems, user interfaces, applications, and the like, with a modem processor that primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 880.
The electronic device 800 also includes a power supply 890 (e.g., a battery) that provides power to the various components, and in some embodiments, may be logically connected to the processor 880 via a power management system to perform functions such as managing charging, discharging, and power consumption via the power management system. Power supply 890 may also include one or more of any components of a dc or ac power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, etc.
Although not shown, the electronic device 800 further includes a camera (e.g., front camera, rear camera), a bluetooth module, etc., which are not described herein. In particular, in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal further includes a memory, and one or more programs, where the one or more programs are stored in the memory, and configured to be executed by the one or more processors, where the one or more programs implement any step in the AIGC-based image light and shadow processing method provided in the foregoing embodiment.
In the implementation, each module may be implemented as an independent entity, or may be combined arbitrarily, and implemented as the same entity or several entities, and the implementation of each module may be referred to the foregoing method embodiment, which is not described herein again.
Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, the present embodiment provides a storage medium in which a plurality of instructions capable of implementing any one of the steps of the AIGC-based image light processing method provided in the above embodiment when executed by a processor are stored.
Wherein the storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
The steps in any embodiment of the image light and shadow processing method based on the AIGC provided in the embodiments of the present application may be executed due to the instructions stored in the storage medium, so that the beneficial effects that any one of the image light and shadow processing method based on the AIGC provided in the embodiments of the present application may achieve are detailed in the previous embodiments and are not described herein.
The foregoing describes in detail an AIGC-based image light and shadow processing method, apparatus, electronic device and storage medium provided in the embodiments of the present application, and specific examples are applied to illustrate the principles and embodiments of the present application, where the foregoing description of the embodiments is only for helping to understand the methods and core ideas of the present application; meanwhile, as those skilled in the art will vary in the specific embodiments and application scope according to the ideas of the present application, the contents of the present specification should not be construed as limiting the present application in summary. Moreover, it will be apparent to those skilled in the art that various modifications and variations can be made without departing from the principles of the present application, and such modifications and variations are considered to be within the scope of the present application.

Claims (10)

1. An AIGC-based image light and shadow processing method, comprising:
acquiring an image to be processed and a shadow template image;
extracting features of the image to be processed to obtain whole image features, color texture features and contour edge features corresponding to the image to be processed;
processing the image to be processed according to the shadow mask image map to obtain an intermediate image corresponding to the image to be processed;
and processing the intermediate image according to the image to be processed, the whole image feature, the color texture feature and the contour edge feature to obtain an output image corresponding to the image to be processed.
2. The method of claim 1, wherein the acquiring the image to be processed and the shadow template image comprises:
receiving an uploaded image to be processed under the condition of receiving an image light and shadow adjustment instruction;
and under the condition that a light and shadow template selection instruction is received, selecting in a light and shadow template library, and determining a light and shadow template image for carrying out light and shadow processing on the image to be processed.
3. The method of claim 2, wherein the method further comprises:
under the condition that a light and shadow template uploading instruction is received, receiving an uploaded first image, analyzing and processing the first image, and determining the image type of the first image, wherein the image type comprises a light and shadow image and a non-light and shadow image;
In the case that the image type of the first image is determined to be a shadow image, taking the first image as a shadow template image;
and under the condition that the image type of the first image is determined to be a non-shadow image, extracting shadow information from the first image, and taking a second image corresponding to the extracted shadow information as a shadow template image.
4. The method according to claim 1, wherein the processing the image to be processed according to the shadow mask image to obtain an intermediate image corresponding to the image to be processed includes:
extracting features of the shadow template image to obtain shadow features corresponding to the shadow template image;
extracting the description information of the image to be processed to obtain a text description corresponding to the image to be processed;
and carrying out image generation and image processing on the image to be processed according to the light and shadow characteristics and the text description to obtain an intermediate image corresponding to the image to be processed.
5. The method according to claim 4, wherein the performing graphics processing on the image to be processed according to the light and shadow features and the text description to obtain an intermediate image corresponding to the image to be processed includes:
Determining whether to delete the current light shadow of the image to be processed according to the received instruction information;
under the condition that the current light shadow of the image to be processed is reserved is determined, carrying out image generation processing on the image to be processed according to the light shadow characteristics and the text description to obtain an intermediate image;
and under the condition that the current light shadow of the image to be processed is determined to be deleted, performing light shadow deletion processing on the image to be processed, and performing image generation graph processing on the image to be processed after deletion processing according to the light shadow characteristics and the text description to obtain an intermediate image.
6. The method according to claim 5, wherein in the case of determining to preserve the current light shadow of the image to be processed, performing a graphically-generated image processing on the image to be processed according to the light shadow feature and the text description to obtain an intermediate image, comprising:
extracting a current light shadow image of the image to be processed under the condition that the current light shadow of the image to be processed is reserved is determined, and obtaining a new image to be processed which does not contain the current light shadow image;
extracting the image description of the new image to be processed to obtain a new text description corresponding to the new image to be processed;
Fusing the current shadow image and the shadow template image to obtain a new shadow template image;
and carrying out image generation and image processing on the new image to be processed according to the new shadow template image and the new text description to obtain an intermediate image.
7. The method according to claim 1, wherein the processing the intermediate image according to the image to be processed, the whole image feature, the color texture feature, and the contour edge feature, to obtain an output image corresponding to the image to be processed, further comprises:
acquiring a text description corresponding to the image to be processed;
carrying out image generation and image processing on the intermediate image according to the text description, the whole image feature, the color texture feature and the contour edge feature to obtain a processed image to be processed;
and under the condition that an image adjusting instruction is received, adjusting the processed image to be processed according to the image adjusting instruction to obtain an output image.
8. An AIGC-based image light and shadow processing apparatus, comprising:
the image acquisition module is used for acquiring an image to be processed and a shadow template image;
the feature extraction module is used for carrying out feature extraction on the image to be processed to obtain integral image features, color texture features and contour edge features corresponding to the image to be processed;
The first processing module is used for processing the image to be processed according to the light and shadow mask image to obtain an intermediate image corresponding to the image to be processed;
and the second processing module is used for processing the intermediate image according to the image to be processed, the whole image characteristic, the color texture characteristic and the contour edge characteristic to obtain an output image corresponding to the image to be processed.
9. An electronic device comprising a processor, a memory and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the method according to any one of claims 1 to 7 when the computer program is executed.
10. A computer-readable storage medium, characterized in that it stores a computer program which, when executed by a processor, implements the steps in the method according to any one of claims 1 to 7.
CN202311531390.0A 2023-11-16 2023-11-16 AIGC-based image light and shadow processing method, device, equipment and storage medium Pending CN117333573A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311531390.0A CN117333573A (en) 2023-11-16 2023-11-16 AIGC-based image light and shadow processing method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311531390.0A CN117333573A (en) 2023-11-16 2023-11-16 AIGC-based image light and shadow processing method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN117333573A true CN117333573A (en) 2024-01-02

Family

ID=89275896

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311531390.0A Pending CN117333573A (en) 2023-11-16 2023-11-16 AIGC-based image light and shadow processing method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN117333573A (en)

Similar Documents

Publication Publication Date Title
CN107256555B (en) Image processing method, device and storage medium
CN108346175B (en) Face image restoration method, device and storage medium
CN108418969B (en) Antenna feed point switching method and device, storage medium and electronic equipment
CN105989572B (en) Picture processing method and device
CN112581358B (en) Training method of image processing model, image processing method and device
CN111182236A (en) Image synthesis method and device, storage medium and terminal equipment
EP4273745A1 (en) Gesture recognition method and apparatus, electronic device, readable storage medium, and chip
CN111966436A (en) Screen display control method and device, terminal equipment and storage medium
CN112541489A (en) Image detection method and device, mobile terminal and storage medium
CN117576258A (en) Image processing method, device, electronic equipment and storage medium
CN111026457B (en) Hardware configuration method and device, storage medium and terminal equipment
CN114511438A (en) Method, device and equipment for controlling load
CN108829600B (en) Method and device for testing algorithm library, storage medium and electronic equipment
CN110168599A (en) A kind of data processing method and terminal
CN110717486B (en) Text detection method and device, electronic equipment and storage medium
CN111046215B (en) Image processing method and device, storage medium and mobile terminal
CN111355991B (en) Video playing method and device, storage medium and mobile terminal
CN117333573A (en) AIGC-based image light and shadow processing method, device, equipment and storage medium
CN111372001B (en) Image fusion method and device, storage medium and mobile terminal
CN112329909B (en) Method, apparatus and storage medium for generating neural network model
CN114140655A (en) Image classification method and device, storage medium and electronic equipment
CN113489903A (en) Shooting method, shooting device, terminal equipment and storage medium
CN110958392A (en) Shooting method of terminal equipment, terminal equipment and storage medium
CN111402273A (en) Image processing method and electronic equipment
CN110990606B (en) Picture storage method and device, storage medium and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination