CN116664391A - Image processing method, device, electronic equipment and storage medium - Google Patents
Image processing method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116664391A CN116664391A CN202310644495.0A CN202310644495A CN116664391A CN 116664391 A CN116664391 A CN 116664391A CN 202310644495 A CN202310644495 A CN 202310644495A CN 116664391 A CN116664391 A CN 116664391A
- Authority
- CN
- China
- Prior art keywords
- image
- target
- conversion relation
- processed
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 16
- 238000006243 chemical reaction Methods 0.000 claims abstract description 208
- 238000012545 processing Methods 0.000 claims abstract description 52
- 230000008859 change Effects 0.000 claims abstract description 16
- 238000000034 method Methods 0.000 claims description 37
- 238000012549 training Methods 0.000 claims description 32
- 230000003287 optical effect Effects 0.000 claims description 14
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 11
- 230000006870 function Effects 0.000 description 18
- 238000010586 diagram Methods 0.000 description 11
- 238000004590 computer program Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 230000004044 response Effects 0.000 description 4
- 230000009466 transformation Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000013475 authorization Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000011833 dog model Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Image Processing (AREA)
Abstract
Provided are an image processing method, an image processing device, an electronic device and a storage medium, wherein a first image and a second image which is matched with the first image and has a target style are preset, and a second target conversion relation between a target image and the second image is obtained through an image processing model based on a first target conversion relation between an image to be processed and the first image, so that the target image can be generated by utilizing the second image and the second target conversion relation. In this way, the stylized processing of the image to be processed is realized by means of the preset image pair with the target style change relation, according to the target conversion relation of the corresponding image between the image pair image to be processed and the preset pair.
Description
Technical Field
The disclosure relates to the technical field of computers, and in particular relates to an image processing method, an image processing device, electronic equipment and a storage medium.
Background
Image special effects techniques may be used to special effects processing of pictures or videos provided by a user, such as stylizing (e.g., comizing) images. Computer vision models employed by related image special effects techniques are typically trained based on a large amount of paired data. For the training method based on the paired data, if a model with good generalization and good quality is to be obtained, a very large amount of paired data is often required to train, which greatly increases the training cost of the model and reduces the usability.
Disclosure of Invention
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
According to a first aspect, according to one or more embodiments of the present disclosure, there is provided an image processing method including:
acquiring an image to be processed;
determining a first target conversion relation between the image to be processed and a preset first image;
inputting the first target conversion relation into an image processing model to obtain a second target conversion relation;
obtaining the target image based on a preset second image and the second target conversion relation;
wherein the second image is an image having a target style paired with the first image, and the target image is an image having the target style paired with the image to be processed.
In a second aspect, according to one or more embodiments of the present disclosure, there is provided an image processing apparatus including:
an image acquisition unit for acquiring an image to be processed;
A first relation determining unit, configured to determine a first target conversion relation between the image to be processed and a preset first image;
the second relation determining unit is used for inputting the first target conversion relation into the image processing model to obtain a second target conversion relation;
an image generating unit, configured to generate an image based on a preset second image and the second target conversion relationship, so as to obtain the target image;
wherein the second image is an image having a target style paired with the first image, and the target image is an image having the target style paired with the image to be processed.
In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device comprising: at least one memory and at least one processor; wherein the memory is for storing program code, and the processor is for invoking the program code stored by the memory to cause the electronic device to perform a method provided in accordance with one or more embodiments of the present disclosure.
In a fourth aspect, according to one or more embodiments of the present disclosure, there is provided a non-transitory computer storage medium storing program code which, when executed by a computer device, causes the computer device to perform a method provided according to one or more embodiments of the present disclosure.
According to one or more embodiments of the present disclosure, by presetting a first image and a second image paired with the first image and having a target style, a second target conversion relationship between the target image and the second image is obtained through an image processing model based on a first target conversion relationship between the image to be processed and the first image, so that the target image can be generated using the second image and the second target conversion relationship. In this way, the stylized processing of the image to be processed is realized by means of the preset image pair (i.e. the first image and the second image) with the target style change relation, according to the target conversion relation of the corresponding image between the image to be processed (i.e. the image to be processed and the target image) and the preset pair (i.e. the first image and the second image).
Drawings
The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.
FIG. 1 is a schematic diagram of a model training method provided by the related art;
FIG. 2 is a flowchart of an image processing method according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of an image processing method according to another embodiment of the present disclosure;
FIG. 4 is a flowchart of an image processing method according to another embodiment of the present disclosure;
FIG. 5 is a schematic diagram of a training method of a conversion relation determination model according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure have been shown in the accompanying drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.
It should be understood that the steps recited in the embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Furthermore, embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.
The term "including" and variations thereof as used herein are intended to be open-ended, i.e., including, but not limited to. The term "based on" is based at least in part on. The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments. The term "responsive to" and related terms mean that one signal or event is affected to some extent by another signal or event, but not necessarily completely or directly. If event x occurs "in response to" event y, x may be directly or indirectly in response to y. For example, the occurrence of y may ultimately lead to the occurrence of x, but other intermediate events and/or conditions may exist. In other cases, y may not necessarily result in the occurrence of x, and x may occur even though y has not yet occurred. Furthermore, the term "responsive to" may also mean "at least partially responsive to".
The term "determining" broadly encompasses a wide variety of actions, which may include obtaining, calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, a database or another data structure), ascertaining and the like, and may also include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory) and the like, as well as parsing, selecting, choosing, establishing and the like. Related definitions of other terms will be given in the description below. Related definitions of other terms will be given in the description below.
It will be appreciated that the data (including but not limited to the data itself, the acquisition or use of the data) involved in the present technical solution should comply with the regulations of the relevant legal regulations.
It will be appreciated that prior to using the technical solutions disclosed in the embodiments of the present disclosure, the user should be informed and authorized of the type, usage range, usage scenario, etc. of the personal information related to the present disclosure in an appropriate manner according to relevant legal regulations. For example, in response to receiving an active request from a user, prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using personal information to the user, so that the user may autonomously select whether to provide personal information to software or hardware such as an electronic device, an application, a server, or a storage medium that performs the operation of the technical solution of the present disclosure according to the prompt information.
As an alternative but non-limiting implementation, in response to receiving an active request from a user, the prompt information may be sent to the user, for example, in a popup window, where the prompt information may be presented in a text manner. In addition, a selection control for the user to select to provide personal information to the electronic device in a 'consent' or 'disagreement' manner can be carried in the popup window.
It will be appreciated that for images generated in accordance with methods provided by embodiments of the present disclosure, they should be processed in compliance with the regulations of the relevant legal regulations. For example, the identification which does not affect the use of the user is added according to the stipulation of technical measures, or the deep synthesis condition is prompted to the public according to the stipulation of significant identification in reasonable positions and areas.
It will be appreciated that the above-described notification and user authorization process, and image processing, are merely illustrative, and not limiting of the implementations of the present disclosure, as other ways of satisfying relevant legal regulations may be applied to the implementations of the present disclosure.
It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.
It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.
For the purposes of this disclosure, the phrase "a and/or B" means (a), (B), or (a and B).
The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
For the model training method based on the paired data, if a model with good generalization and good quality is to be obtained, a large amount of paired data is often required to train, so that the training cost of the model is greatly increased, and the usability is reduced. For example, referring to fig. 1, the model training method provided by the related art is based on training directly using paired sample image pairs, which specifically employs n pairs of sample image pairs, i.e., sample image pairs (x 1 ，y 1 )、(x 2 ，y 2 )、(x 3 ，y 3 )、......、(x n ，y n ). Wherein the sample image y i Is the same as the sample image x i Paired images of a certain style (where i is a positive integer not greater than n), e.g. sample image x i And y i The image (e.g., cartoon image) of the original image and the image of the original image after special effect processing can be respectively obtained. In the traditional model training method, a sample image x is directly used i As input, sample image y i Model 10 is trained as the desired output so that the trained model can generate an image of a style paired with the input image based on the input image, thereby enabling a stylized process for the input image. However, as described above, the model training method shown in fig. 1 has a high number of required sample image pairs because the sample image pairs are directly used as training objects, and thus the training cost of the model is high.
Referring to fig. 2, fig. 2 shows a flowchart of an image processing method 200 according to an embodiment of the disclosure, and the method 200 includes steps S210 to S220.
Step S210: a plurality of pairs of sample image pairs are acquired, the pairs comprising a sample image and a target sample image paired with the sample image having the target style.
In some embodiments, more than three pairs of sample image pairs may be selected in a preset set of sample images, each pair of sample image pairs comprising a sample image and a target sample image paired with the sample image having the target style.
Wherein, the target style can be used for adding expression special effects or other stylized special effects to the original image. For example, the sample image and the target sample image may be an original image and a cartoon-processed image of the original image, respectively, but the present disclosure is not limited thereto.
Step S220: a third target conversion relationship between the first sample image and the second sample image is acquired.
Step S230: a fourth target conversion relationship between the first target sample image and the second target sample image is acquired.
Step S240: and training the image processing model based on the third target conversion relation and the fourth target conversion relation.
The third target conversion relation is a target conversion relation between the first sample image and the second sample image, and the fourth target conversion relation is a target conversion relation between the first target sample image and the second target sample image.
The first sample image and the first target sample image may constitute a sample image pair (i.e., a first sample image pair), and the first sample image and the first target sample image may constitute a sample image pair (i.e., a second sample image pair). In some embodiments, the first sample image pair and the second sample image pair are any two sample image pairs of the plurality of pairs of sample images.
In some embodiments, the image processing model may be obtained by training, using as input a target conversion relationship (i.e., a third target conversion relationship) between a first sample image in a first sample image pair and a second sample image in a second sample image pair, and using as a desired output (i.e., a fourth target conversion relationship) a target conversion relationship between the first target sample image in the first sample image pair and the second target sample image in the second sample image pair; wherein the first sample image pair and the second sample image pair are any two of the three or more sample image pairs.
Illustratively, referring to FIG. 3, the sample image set contains 3 pairs of sample image pairs, respectively sample image pairs (x 1 ，y 1 )、(x 2 ，y 2 )、(x 3 ，y 3 ). Wherein the target sample image y 1 、y 2 And y 3 Respectively, with the sample image x 1 、x 2 And x 3 Paired images with a target style.
In the model training process, for any two of the 3 pairs of sample images, for example, the first pair of image samples (x 1 ，y 1 ) And a second graphPhoto sample pair (x) 2 ，y 2 ) For example, sample image x 1 And sample image x 2 Target conversion relation f between x As input (e.g. x 1 ＝f x (x 2 ) A) to image the target sample y 1 And the target sample image y 2 Target conversion relation f between y As the desired output (e.g., y 1 ＝f y (y 2 ) To train the image processing model 20.
In one embodiment, the model may be trained in a loss-supervised manner, e.g., converting the target relationship f x Input to the image processing model, and calculate the conversion relation f between the output of the image processing model and the target y And adjusting parameters of the image processing model based on the loss value.
Referring to fig. 3, up to 6 pairs of directed object conversion relation pairs (f x ，f y ) The 6 pairs of directed target conversion relation pairs (f x ，f y ) The image processing model can be trained as a training sample pair. Similarly, if the sample image set contains n pairs of sample images (where n is not less than 3), at most n (n-1) pairs of target conversion relation pairs (f x ，f y ) Training an image processing model as a training sample pair, wherein f x And f y Satisfies the following formula:
x m ＝f x ({x 1 |i≠m，1≤i≤n})
y m ＝f y ({y 1 |i≠m，1≤i≤n})
wherein m is any positive integer not less than 1 and not more than n.
Thus, according to one or more embodiments of the present disclosure, by training an image processing model with a target conversion relationship between sample images in any two sample image pairs in a sample image set as an input and a target conversion relationship between corresponding target sample images as a desired output, the number of training samples of the model is greatly increased compared to a conventional training sample using directly sample images as training samples, and the training quality of the model is ensured while the number of demands on sample images is reduced.
Referring to fig. 4, fig. 4 shows a flowchart of an image processing method 400 according to an embodiment of the present disclosure, and the method 400 includes steps S410 to S440.
Step S410: and acquiring an image to be processed.
In some embodiments, the image to be processed may include a picture or video provided by the user that requires special effects processing of the image. In one embodiment, the image to be processed may include a person image, but the present disclosure is not limited thereto.
Step S420: and determining a first target conversion relation between the image to be processed and a preset first image.
The first image may be pre-stored in the client local or server, which is not limited herein. In some embodiments, the first image may be the same type of image as the image to be processed uploaded by the user. For example, if the image to be processed is a pet dog image, the first image is also a pet dog image or an image with a pet dog model. In one embodiment, the first image may be determined based on the type of image to be processed. By way of example, the type of the image to be processed may be determined based on an instruction of a user, or based on the type to which an entry for uploading the image to be processed belongs, or based on image recognition of the image to be processed, or the like, but the present disclosure is not limited thereto.
The first target conversion relationship is a target conversion relationship between the image to be processed and the first image. In this embodiment, the first target conversion relationship is used to enable the first image to approach the image to be processed.
Step S430: and inputting the first target conversion relation into an image processing model to obtain a second target conversion relation.
Step S440: obtaining the target image based on a preset second image and the second target conversion relation; wherein the second image is an image having a target style paired with the first image, and the target image is an image having the target style paired with the image to be processed.
The image processing model is trained to be able to generate a target conversion relationship between a set of two images for pairing with the two images respectively based on the target conversion relationship between the two images. For example, the training method of the image processing model may be a method as shown in fig. 2 of the present disclosure, which is not described herein.
The second target conversion relationship is a target conversion relationship between the target image and the second image. In the present embodiment, the second target conversion relationship is used to enable the second image to approach the target image.
According to one or more embodiments of the present disclosure, by presetting a first image and a second image paired with the first image and having a target style, a second target conversion relationship between the target image and the second image is obtained through an image processing model based on a first target conversion relationship between the image to be processed and the first image, so that the target image can be generated using the second image and the second target conversion relationship. In this way, the stylized processing of the image to be processed is realized by means of the preset image pair (i.e. the first image and the second image) with the target style change relation, according to the target conversion relation of the corresponding image between the image to be processed (i.e. the image to be processed and the target image) and the preset pair (i.e. the first image and the second image).
In some embodiments, the target conversion relationships may include spatial conversion relationships and color conversion relationships. Wherein the spatial transformation relationship represents the change of the image on the spatial scale, such as translation, turnover, deformation and the like; the color conversion relationship identifies a change in the color scale of the image, embodied as a color change of the image. In this way, the target conversion relationship may reflect the spatial scale change and the color change between the two images.
In some embodiments, when converting one image (i.e., original image) into another image (i.e., target image) through a target conversion relationship, the original image may be spatially converted first, and then the spatially converted image (e.g., an intermediate image) may be color converted. In one embodiment, the targetThe transformation function f is represented by a 2D deformation field, which can be represented as a combination of a Ts function and a Tc function; wherein, the Ts function represents the change of RGB image on space scale, and the concrete form can be an optical flow chart; the Tc function represents the change in the color scale of the RGB image, which may be in the specific form of a series of parameters of affine transformation, but the disclosure is not limited thereto. Applying the target conversion function f to an original image Xi to obtain a target image The process of (1) may include: the original image is processed based on the Ts function to obtain an intermediate image (such as a light flow graph), and the intermediate image obtained in the previous step is further processed based on the Tc function to obtain the target image.
In some embodiments, the determining the first target conversion relationship between the image to be processed and the preset first image includes:
acquiring a light flow graph based on the image to be processed and the preset first image;
obtaining color affine variation based on the image to be processed and the preset first image;
processing the optical flow graph based on the affine change amount of the color to obtain a target conversion relation graph;
and taking the target conversion relation graph as the first target conversion relation.
In some embodiments, the obtaining a light flow map based on the image to be processed and the preset first image includes:
acquiring a spatial scale variation based on the image to be processed and the preset first image;
the optical flow map is obtained based on the spatial scale variation.
In some embodiments, the determining the first target conversion relationship between the image to be processed and the preset first image includes:
Obtaining a conversion relation determining model;
and inputting the image to be processed and the preset first image into a conversion relation determining model to obtain the first target conversion relation.
In one specific embodiment, the conversion relation determination model is obtained by the following steps:
step A1: inputting the sample image to be converted into a conversion relation determining model to obtain an initial target conversion relation;
step A2: obtaining an initial conversion image based on the sample image to be converted and the initial target conversion relation;
step A3: and adjusting parameters of the conversion relation determination model based on the loss between the initial conversion image and the target conversion sample image to obtain a trained conversion relation determination model.
In this way, the trained conversion relation determination model may be used to determine a target conversion relation between a given image to be converted and a target conversion image.
Illustratively, a training flow diagram of the conversion relation determination model is shown with reference to fig. 5. The training samples may include a plurality of training sample pairs, each training sample pair including a sample image X to be converted i And corresponding target conversion sample image X j . When training the conversion relation determination model, the sample image X to be converted is obtained i Inputting a conversion relation determination model Mx to obtain an initial target conversion function f to be matched with a sample image X to be converted i Obtaining an initial conversion image with the target conversion function fAnd based on the initial conversion image->And target conversion sample X j The loss between them adjusts the parameters of the conversion relation determining model Mx such that the target conversion function generated by the conversion relation determining model Mx enables the initial conversion image +.>As close as possible to the target conversion sample image X j . In a toolIn the embodiment of the volume, the initial conversion image +.>And target conversion sample X j The conversion relation determination model Mx is trained by means of reconstruction loss supervision, but the present disclosure is not limited thereto.
In some embodiments, the third target conversion relationship or the fourth target conversion relationship may be determined in the same manner as the first target conversion relationship. For example, the first sample image and the second sample image may be input into the conversion relation determination model to obtain a third target conversion relation, and similarly, the first target sample image and the second target sample image may be input into the conversion relation determination model to obtain a fourth target conversion relation; similarly, the light flow graph and the affine color change amount can be obtained based on the first sample image and the second sample image and finally the third target relationship can be obtained, and similarly, the light flow graph and the affine color change amount can be obtained based on the first sample target image and the second sample target image and finally the third target relationship can be obtained. It should be noted that the same manner of determining the target relationship does not mean that the processing objects of the determination manner are identical, for example, the object processed by the determination manner of the first target relationship is the image to be processed and the first image, the object processed by the determination manner of the third target relationship is the first sample image and the second sample image, and the object processed by the determination manner of the fourth target relationship is the first target sample image and the second target sample image.
Accordingly, referring to fig. 6, there is provided an image processing apparatus 600 according to an embodiment of the present disclosure, including:
an image acquisition unit 601 for acquiring an image to be processed;
a first relationship determining unit 602, configured to determine a first target conversion relationship between the image to be processed and a preset first image;
a second relationship determining unit 603, configured to input the first target conversion relationship to an image processing model, to obtain a second target conversion relationship;
an image generating unit 604, configured to generate an image based on a preset second image and the second target conversion relationship, so as to obtain the target image;
wherein the second image is an image having a target style paired with the first image, and the target image is an image having the target style paired with the image to be processed.
In some embodiments, the image processing model is obtained by: acquiring a plurality of pairs of sample image pairs, the pairs comprising a sample image and a target sample image paired with the sample image having the target style; acquiring a third target conversion relation between the first sample image and the second sample image; acquiring a fourth target conversion relation between the first target sample image and the second target sample image; and training the image processing model based on the third target conversion relation and the fourth target conversion relation.
In some embodiments, the target conversion relationship includes a spatial conversion relationship and a color conversion relationship.
In some embodiments, the first relation determining unit is specifically configured to: acquiring a light flow graph based on the image to be processed and the preset first image; obtaining color affine variation based on the image to be processed and the preset first image; processing the optical flow graph based on the affine change amount of the color to obtain a target conversion relation graph; and taking the target conversion relation graph as the first target conversion relation.
In some embodiments, the obtaining a light flow map based on the image to be processed and the preset first image includes:
acquiring a spatial scale variation based on the image to be processed and the preset first image;
the optical flow map is obtained based on the spatial scale variation.
In some embodiments, the first relation determining unit is specifically configured to:
obtaining a conversion relation determining model;
inputting the image to be processed and the preset first image into a conversion relation determining model to obtain the first target conversion relation;
the conversion relation determination model is obtained through the following steps:
Inputting the sample image to be converted into a conversion relation determining model to obtain an initial target conversion relation;
obtaining an initial conversion image based on the sample image to be converted and the initial target conversion relation;
and adjusting parameters of the conversion relation determination model based on the loss between the initial conversion image and the target conversion sample image to obtain a trained conversion relation determination model.
In some embodiments, the third target conversion relationship or the fourth target conversion relationship is determined in the same manner as the first target conversion relationship.
In some embodiments, the image generation unit is specifically configured to:
obtaining an intermediate image based on the preset second image and the space conversion relation;
and obtaining the target image based on the intermediate image and the color conversion relation.
For embodiments of the device, reference is made to the description of method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the modules illustrated as separate modules may or may not be separate. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
Accordingly, in accordance with one or more embodiments of the present disclosure, there is provided an electronic device comprising:
at least one memory and at least one processor;
wherein the memory is for storing program code, and the processor is for invoking the program code stored by the memory to cause the electronic device to perform an image processing method provided in accordance with one or more embodiments of the present disclosure.
Accordingly, in accordance with one or more embodiments of the present disclosure, there is provided a non-transitory computer storage medium storing program code executable by a computer device to cause the computer device to perform an image processing method provided in accordance with one or more embodiments of the present disclosure.
Referring now to fig. 7, a schematic diagram of an electronic device (e.g., a terminal device or server) 800 suitable for use in implementing embodiments of the present disclosure is shown. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and stationary terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 7 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.
As shown in fig. 7, the electronic device 800 may include a processing means (e.g., a central processor, a graphics processor, etc.) 801, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 802 or a program loaded from a storage means 808 into a Random Access Memory (RAM) 803. In the RAM803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.
In general, the following devices may be connected to the I/O interface 805: input devices 806 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 807 including, for example, a Liquid Crystal Display (LCD), speakers, vibrators, etc.; storage 808 including, for example, magnetic tape, hard disk, etc.; communication means 809. The communication means 809 may allow the electronic device 800 to communicate wirelessly or by wire with other devices to exchange data. While fig. 7 shows an electronic device 800 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication device 809, or installed from storage device 808, or installed from ROM 802. The above-described functions defined in the methods of the embodiments of the present disclosure are performed when the computer program is executed by the processing device 801.
It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform the methods of the present disclosure described above.
Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
According to one or more embodiments of the present disclosure, there is provided an image processing method including:
acquiring an image to be processed;
determining a first target conversion relation between the image to be processed and a preset first image;
inputting the first target conversion relation into an image processing model to obtain a second target conversion relation;
obtaining the target image based on a preset second image and the second target conversion relation;
wherein the second image is an image having a target style paired with the first image, and the target image is an image having the target style paired with the image to be processed.
According to one or more embodiments of the present disclosure, the image processing model is obtained by:
acquiring a plurality of pairs of sample image pairs, the pairs comprising a sample image and a target sample image paired with the sample image having the target style;
acquiring a third target conversion relation between the first sample image and the second sample image;
acquiring a fourth target conversion relation between the first target sample image and the second target sample image;
and training the image processing model based on the third target conversion relation and the fourth target conversion relation.
According to one or more embodiments of the present disclosure, the target conversion relationship includes a spatial conversion relationship and a color conversion relationship.
According to one or more embodiments of the present disclosure, the first relationship determination unit is specifically configured to: acquiring a light flow graph based on the image to be processed and the preset first image; obtaining color affine variation based on the image to be processed and the preset first image; processing the optical flow graph based on the affine change amount of the color to obtain a target conversion relation graph; and taking the target conversion relation graph as the first target conversion relation.
According to one or more embodiments of the present disclosure, the obtaining a light flow map based on the image to be processed and the preset first image includes:
acquiring a spatial scale variation based on the image to be processed and the preset first image;
the optical flow map is obtained based on the spatial scale variation.
According to one or more embodiments of the present disclosure, the first relationship determination unit is specifically configured to:
obtaining a conversion relation determining model;
inputting the image to be processed and the preset first image into a conversion relation determining model to obtain the first target conversion relation;
The conversion relation determination model is obtained through the following steps:
inputting the sample image to be converted into a conversion relation determining model to obtain an initial target conversion relation;
obtaining an initial conversion image based on the sample image to be converted and the initial target conversion relation;
and adjusting parameters of the conversion relation determination model based on the loss between the initial conversion image and the target conversion sample image to obtain a trained conversion relation determination model.
According to one or more embodiments of the present disclosure, the third target conversion relationship or the fourth target conversion relationship is determined in the same manner as the first target conversion relationship.
According to one or more embodiments of the present disclosure, the image generation unit is specifically configured to:
obtaining an intermediate image based on the preset second image and the space conversion relation;
and obtaining the target image based on the intermediate image and the color conversion relation.
According to one or more embodiments of the present disclosure, there is provided an image processing apparatus including:
an image acquisition unit for acquiring an image to be processed;
a first relation determining unit, configured to determine a first target conversion relation between the image to be processed and a preset first image;
The second relation determining unit is used for inputting the first target conversion relation into the image processing model to obtain a second target conversion relation;
an image generating unit, configured to generate an image based on a preset second image and the second target conversion relationship, so as to obtain the target image;
wherein the second image is an image having a target style paired with the first image, and the target image is an image having the target style paired with the image to be processed.
According to one or more embodiments of the present disclosure, there is provided an electronic device including: at least one memory and at least one processor; wherein the memory is for storing program code, and the processor is for invoking the program code stored by the memory to cause the electronic device to perform an image processing method provided in accordance with one or more embodiments of the present disclosure.
According to one or more embodiments of the present disclosure, there is provided a non-transitory computer storage medium storing program code which, when executed by a computer device, causes the computer device to perform an image processing method provided according to one or more embodiments of the present disclosure.
The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by persons skilled in the art that the scope of the disclosure referred to in this disclosure is not limited to the specific combinations of features described above, but also covers other embodiments which may be formed by any combination of features described above or equivalents thereof without departing from the spirit of the disclosure. Such as those described above, are mutually substituted with the technical features having similar functions disclosed in the present disclosure (but not limited thereto).
Moreover, although operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are example forms of implementing the claims.
Claims (11)
1. An image processing method, comprising:
acquiring an image to be processed;
determining a first target conversion relation between the image to be processed and a preset first image;
inputting the first target conversion relation into an image processing model to obtain a second target conversion relation;
obtaining the target image based on a preset second image and the second target conversion relation;
wherein the second image is an image having a target style paired with the first image, and the target image is an image having the target style paired with the image to be processed.
2. The method according to claim 1, wherein the image processing model is obtained by:
acquiring a plurality of pairs of sample image pairs, the pairs comprising a sample image and a target sample image paired with the sample image having the target style;
Acquiring a third target conversion relation between the first sample image and the second sample image;
acquiring a fourth target conversion relation between the first target sample image and the second target sample image;
and training the image processing model based on the third target conversion relation and the fourth target conversion relation.
3. The method according to claim 1 or 2, wherein the target conversion relationship includes a spatial conversion relationship and a color conversion relationship.
4. The method according to claim 1, wherein determining a first target conversion relationship between the image to be processed and a preset first image comprises:
acquiring a light flow graph based on the image to be processed and the preset first image;
obtaining color affine variation based on the image to be processed and the preset first image;
processing the optical flow graph based on the affine change amount of the color to obtain a target conversion relation graph;
and taking the target conversion relation graph as the first target conversion relation.
5. The method of claim 4, wherein the obtaining a dataflow graph based on the image to be processed and the preset first image includes:
Acquiring a spatial scale variation based on the image to be processed and the preset first image;
the optical flow map is obtained based on the spatial scale variation.
6. The method according to claim 1, wherein determining a first target conversion relationship between the image to be processed and a preset first image comprises:
obtaining a conversion relation determining model;
inputting the image to be processed and the preset first image into the conversion relation determining model to obtain the first target conversion relation;
the conversion relation determination model is obtained through the following steps:
inputting the sample image to be converted into a conversion relation determining model to obtain an initial target conversion relation;
obtaining an initial conversion image based on the sample image to be converted and the initial target conversion relation;
and adjusting parameters of the conversion relation determination model based on the loss between the initial conversion image and the target conversion sample image to obtain a trained conversion relation determination model.
7. The method according to claim 4 or 6, wherein,
the third target conversion relation or the fourth target conversion relation is determined in the same manner as the first target conversion relation.
8. A method according to claim 3, wherein the obtaining the target image based on the preset second image and the second target conversion relation includes:
obtaining an intermediate image based on the preset second image and the space conversion relation;
and obtaining the target image based on the intermediate image and the color conversion relation.
9. An image processing apparatus, comprising:
an image acquisition unit for acquiring an image to be processed;
a first relation determining unit, configured to determine a first target conversion relation between the image to be processed and a preset first image;
the second relation determining unit is used for inputting the first target conversion relation into the image processing model to obtain a second target conversion relation;
an image generating unit, configured to generate an image based on a preset second image and the second target conversion relationship, so as to obtain the target image;
wherein the second image is an image having a target style paired with the first image, and the target image is an image having the target style paired with the image to be processed.
10. An electronic device, comprising:
At least one memory and at least one processor;
wherein the memory is for storing program code and the processor is for invoking the program code stored in the memory to cause the electronic device to perform the method of any of claims 1-8.
11. A non-transitory computer storage medium comprising,
the non-transitory computer storage medium stores program code that, when executed by a computer device, causes the computer device to perform the method of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310644495.0A CN116664391A (en) | 2023-06-01 | 2023-06-01 | Image processing method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310644495.0A CN116664391A (en) | 2023-06-01 | 2023-06-01 | Image processing method, device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116664391A true CN116664391A (en) | 2023-08-29 |
Family
ID=87716774
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310644495.0A Pending CN116664391A (en) | 2023-06-01 | 2023-06-01 | Image processing method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116664391A (en) |
-
2023
- 2023-06-01 CN CN202310644495.0A patent/CN116664391A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111459364B (en) | Icon updating method and device and electronic equipment | |
| CN113449070A (en) | Multimodal data retrieval method, device, medium and electronic equipment | |
| CN116310582A (en) | Classification model training method, image classification method, device, medium and equipment | |
| CN111311358B (en) | Information processing method and device and electronic equipment | |
| CN117241092A (en) | Video processing method and device, storage medium and electronic equipment | |
| CN116596748A (en) | Image stylization processing method, apparatus, device, storage medium, and program product | |
| CN116186545A (en) | Training and application methods and devices of pre-training model, electronic equipment and medium | |
| CN116092092A (en) | Matching method, device, medium and electronic equipment | |
| CN111680754B (en) | Image classification method, device, electronic equipment and computer readable storage medium | |
| CN116072108A (en) | Model generation method, voice recognition method, device, medium and equipment | |
| CN111737575B (en) | Content distribution method, content distribution device, readable medium and electronic equipment | |
| CN116664391A (en) | Image processing method, device, electronic equipment and storage medium | |
| CN115269978A (en) | Video tag generation method, device, equipment and medium | |
| CN111258582B (en) | Window rendering method and device, computer equipment and storage medium | |
| CN111399730A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN111368204A (en) | Content pushing method and device, electronic equipment and computer readable medium | |
| CN111258670B (en) | Method and device for managing component data, electronic equipment and storage medium | |
| CN116030321A (en) | Method, device, electronic equipment and storage medium for generating image | |
| CN116340632A (en) | Object recommendation method, device, medium and electronic equipment | |
| CN116886531A (en) | Service processing method, device, medium and electronic equipment | |
| CN116319932A (en) | Training method, device, equipment and storage medium of content push model | |
| CN117687551A (en) | Display object determining method and device, storage medium and electronic equipment | |
| CN116342721A (en) | Image compression method, device, equipment and storage medium | |
| CN116071224A (en) | Image conversion model training method, image conversion method and device | |
| CN117252954A (en) | Image processing method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |