CN115578745A - Method and apparatus for generating image - Google Patents

Method and apparatus for generating image Download PDF

Info

Publication number
CN115578745A
CN115578745A CN202110758294.4A CN202110758294A CN115578745A CN 115578745 A CN115578745 A CN 115578745A CN 202110758294 A CN202110758294 A CN 202110758294A CN 115578745 A CN115578745 A CN 115578745A
Authority
CN
China
Prior art keywords
image
garment
information
person
clothing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110758294.4A
Other languages
Chinese (zh)
Inventor
陶大程
王宁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jingdong Technology Information Technology Co Ltd
Original Assignee
Jingdong Technology Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jingdong Technology Information Technology Co Ltd filed Critical Jingdong Technology Information Technology Co Ltd
Priority to CN202110758294.4A priority Critical patent/CN115578745A/en
Priority to PCT/CN2022/098625 priority patent/WO2023279922A1/en
Publication of CN115578745A publication Critical patent/CN115578745A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G06Q30/0601—Electronic shopping [e-shopping]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G06Q30/0601—Electronic shopping [e-shopping]
    • G06Q30/0641—Shopping interfaces
    • G06Q30/0643—Graphical representation of items or shoppers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Business, Economics & Management (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Accounting & Taxation (AREA)
  • Mathematical Physics (AREA)
  • Finance (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)

Abstract

The application discloses a method and a device for generating an image, and relates to the technical field of computers. The method comprises the following steps: acquiring a character image, and identifying reference deformation information of a reference garment in the character image, wherein the reference deformation information is used for representing deformation information of the garment in the character image; acquiring a target clothing image, and generating a deformation clothing image corresponding to the target clothing image according to the target clothing image and the reference deformation information; and generating a target image according to the person image and the deformed clothing image. By adopting the method, the accuracy of generating the target image can be improved.

Description

Method and apparatus for generating image
Technical Field
The present disclosure relates to the field of computer technology, and in particular, to a method and apparatus for generating an image.
Background
With the development of e-commerce, online shopping is gradually replacing offline shopping. When clothes are selected in an online shopping mode, how to provide try-on service for online users becomes a problem to be solved primarily by an online platform. The existing method for providing try-on service for online users is as follows: fitting a virtual fitting image of the user based on a large amount of three-dimensional user data and three-dimensional garment data, or fitting the garment according to the planar data of the user to generate the virtual fitting image after the garment is dressed by the user.
However, the method of fitting the virtual fitting image of the user based on a large amount of three-dimensional user data and three-dimensional garment data has the problems of low efficiency and waste of computing resources; the method for generating the virtual fitting image after the clothing is dressed by the user has the problem of inaccurate fitting after the clothing is fitted according to the plane data of the user.
Disclosure of Invention
The present disclosure provides a method, an apparatus, an electronic device, and a computer-readable storage medium for generating an image.
According to a first aspect of the present disclosure, there is provided a method for generating an image, comprising: acquiring a character image, and identifying reference deformation information of a reference garment in the character image, wherein the reference deformation information is used for representing the deformation information of the reference garment in the character image; acquiring a target clothing image, and generating a deformed clothing image corresponding to the target clothing image according to the target clothing image and the reference deformation information; and generating a target image according to the person image and the deformed clothing image.
In some embodiments, identifying reference deformation information for a reference garment in the image of the person includes: acquiring human body analysis data, and determining reference clothing region information in the person image according to the human body analysis data, wherein the reference clothing region information is used for representing the region of the reference clothing in the person image; acquiring an image of a reference garment in the person image according to the person image and the reference garment region information; and identifying the image of the reference garment to obtain the deformation information of the reference garment, and determining the deformation information of the reference garment as the reference deformation information.
In some embodiments, identifying reference deformation information for a reference garment in the image of the person includes: inputting the figure image into a trained model network, and obtaining reference deformation information; wherein the model network is based on loss training between the flat clothing image corresponding to the reference clothing image and the clothing image of the reference clothing in an undeformed state.
In some embodiments, generating a deformed garment image corresponding to the target garment image according to the target garment image and the reference deformation information includes: acquiring characteristic information of the target garment in the target garment image; acquiring reference clothing region information, wherein the reference clothing region information is used for representing the region of the reference clothing in the person image; determining initial deformation information according to the characteristic information of the target garment and the reference garment region information; and generating a deformed clothing image according to the posture information of the person in the person image, the target clothing image, the reference deformation information and the initial deformation information.
In some embodiments, a method for generating an image comprises: acquiring attribute information of the person image, wherein the attribute information of the person image comprises at least one of the following items: characteristic information of a person in the person image, a human body area of the person, and posture information of the person; generating a target image according to the person image and the deformed clothing image, wherein the target image comprises: and generating a target image according to the person image, the deformed clothing image and the attribute information.
According to a second aspect of the present disclosure, there is provided an apparatus for generating an image, comprising: the identification unit is configured to acquire a person image and identify reference deformation information of a reference garment in the person image, wherein the reference deformation information is used for representing the deformation information of the reference garment in the person image; the acquiring unit is configured to acquire a target garment image and generate a deformed garment image corresponding to the target garment image according to the target garment image and the reference deformation information; and a generating unit configured to generate a target image from the person image and the deformed clothing image.
In some embodiments, the identification unit comprises: the first acquisition module is configured to acquire human body analysis data and determine reference clothing region information in the character image according to the human body analysis data, wherein the reference clothing region information is used for representing the region of the reference clothing in the character image; the second acquisition module is configured to acquire an image of a reference garment in the person image according to the person image and the reference garment region information; the first determining module is configured to identify the image of the reference garment to obtain deformation information of the reference garment, and determine the deformation information of the reference garment as the reference deformation information.
In some embodiments, the identification unit comprises: the recognition module is configured to input the figure image into the trained model network and obtain reference deformation information; the model network is based on loss training between the flat clothing image corresponding to the reference clothing image and the clothing image of the reference clothing in the undeformed state.
In some embodiments, the obtaining unit comprises: the third acquisition module is configured to acquire characteristic information of the target garment in the target garment image; the fourth acquisition module is configured to acquire reference clothing region information, wherein the reference clothing region information is used for representing the region of the reference clothing in the person image; a second determining module configured to determine initial deformation information according to the feature information of the target garment and the reference garment region information; and the first generation module is configured to generate a deformed clothing image according to the posture information of the person in the person image, the target clothing image, the reference deformation information and the initial deformation information.
In some embodiments, an apparatus comprises: a fifth acquiring module configured to acquire attribute information of the person image, wherein the attribute information of the person image includes at least one of: characteristic information of a person in the person image, a human body area of the person, and posture information of the person; a generation unit comprising: and the second generation module is configured to generate a target image according to the person image, the deformed clothing image and the attribute information.
According to a third aspect of the present disclosure, an embodiment of the present disclosure provides an electronic device, including: one or more processors: a storage device for storing one or more programs which, when executed by one or more processors, cause the one or more processors to implement a method for generating an image as provided by the first aspect.
According to a fourth aspect of the present disclosure, embodiments of the present disclosure provide a computer readable storage medium having stored thereon a computer program, wherein the program, when executed by a processor, implements the method for generating an image as provided by the first aspect.
The method and the device for generating the image provided by the disclosure comprise the following steps: acquiring a character image, and identifying reference deformation information of a reference garment in the character image, wherein the reference deformation information is used for representing the deformation information of the reference garment in the character image; acquiring a target clothing image, and generating a deformation clothing image corresponding to the target clothing image according to the target clothing image and the reference deformation information; and generating the target image according to the person image and the deformed clothing image, so that the accuracy of generating the target image can be improved, and a large amount of computing resources can be avoided.
It should be understood that the statements in this section are not intended to identify key or critical features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
FIG. 1 is an exemplary system architecture diagram in which embodiments of the present application may be applied;
FIG. 2 is a flow diagram of one embodiment of a method for generating an image according to the present application;
FIG. 3 is a flow diagram of another embodiment of a method for generating an image according to the present application;
FIG. 4 is a flow chart of one application scenario of a method for generating an image according to the present application;
FIG. 5 is a schematic diagram of an embodiment of an apparatus for generating an image according to the present application;
fig. 6 is a block diagram of an electronic device for implementing a method for generating an image according to an embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 shows an exemplary system architecture 100 to which embodiments of the method for generating an image or the apparatus for generating an image of the present application may be applied.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may be user terminal devices on which various client applications may be installed, such as shopping-like applications, image-like applications, play-like applications, search-like applications, financial-like applications, etc.
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting receiving server messages, including but not limited to smartphones, tablets, e-book readers, electronic players, laptop portable computers, desktop computers, and the like.
The terminal apparatuses 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, and 103 are hardware, various electronic devices may be used, and when the terminal devices 101, 102, and 103 are software, the electronic devices may be installed in the above-listed electronic devices. It may be implemented as multiple pieces of software or software modules (e.g., multiple software modules used to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.
The server 105 may obtain the person image and identify reference deformation information of the reference garment in the person image, where the reference deformation information is used to represent deformation information of the reference garment in the person image. And acquiring a target clothing image, and generating a deformed clothing image corresponding to the target clothing image according to the target clothing image and the reference deformation information. And then, generating a target image according to the person image and the deformation garment image.
It should be noted that the method for generating an image provided by the embodiment of the present disclosure may be executed by the server 105, and accordingly, the apparatus for generating an image may be disposed in the server 105.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for an implementation.
With continued reference to FIG. 2, a flow 200 of one embodiment of a method for generating an image according to the present disclosure is shown, comprising the steps of:
step 201, obtaining a person image, and identifying reference deformation information of a reference garment in the person image, wherein the reference deformation information is used for representing deformation information of the reference garment in the person image.
In the present embodiment, the execution subject (e.g., the server 105 shown in fig. 1) of the method for generating an image may acquire a person image from the internet or a terminal device in a wired or wireless manner, and identify a reference garment in the person image and identify deformation information of the reference garment. The character image may be a character image of a person who needs to try on the clothing, which is uploaded by a user who needs to try on the clothing on a wire (the person who needs to try on the clothing may be the uploading user, or may be a target wearer of the clothing selected by the user, such as friends or relatives of the user), and the character image includes a clothing region of a character, such as an image of an upper half of a human body, an image of a lower half of a human body, or an image of a whole body. The reference garment refers to a garment currently worn by a person in the person image. The reference deformation information of the reference garment refers to deformation information presented by a garment currently worn by a person in a person image, and it can be understood that the garment is planar in a flat/square state, has no deformation, and only has the shape of the garment, and when the person wears the garment, the shape of the garment deforms based on the contour of the human body and the posture of the person.
Specifically, the reference clothing in the character image can be identified through an image identification method, and the reference deformation information of the reference clothing can be obtained, and the reference clothing in the character image and the reference deformation information of the reference clothing can also be identified based on a pre-trained deep learning model.
Step 202, obtaining a target clothing image, and generating a deformed clothing image corresponding to the target clothing image according to the target clothing image and the reference deformation information.
In this embodiment, the target garment image may be obtained, and a deformed garment image corresponding to the target garment image may be generated according to the target garment image and the reference deformation information of the reference garment. The target clothing image can be an image of a clothing needing to be tried on, which is input by a user, the target clothing image can be an image of a clothing uploaded by the user, and can be an image of a clothing selected/intended to be purchased by an online shopping/trying-on platform of the user.
Specifically, the target clothing image and the reference deformation information may be input into a pre-trained deep learning model, and the deformed target clothing image may be output based on the deep learning model. The shape and the contour of the target garment image can also be adjusted based on the deformation parameters of the reference deformation information, so that the deformed garment image corresponding to the target garment image is generated after the target garment image is distorted/deformed.
And step 203, generating a target image according to the person image and the deformed clothing image.
In this embodiment, the target image may be generated from the person image and the morphed clothing image. Specifically, an image processing method may be adopted to delete/erase an existing reference garment in the person image, and paste the deformed garment image to the body part in the person image, so as to replace the original reference garment on the body in the person image with the deformed garment image. The method can also be used for directly pasting the deformed clothing image to the human body part in the character image so as to realize covering of the original reference clothing on the human body in the character image by adopting the deformed clothing image, and can also be used for generating a target image by utilizing a deep learning model and based on the character image and the deformed clothing image.
The method for generating the image, provided by the embodiment, includes the steps of obtaining a person image, and identifying reference deformation information of a reference garment in the person image, wherein the reference deformation information is used for representing deformation information of the garment in the person image; acquiring a target clothing image, and generating a deformation clothing image corresponding to the target clothing image according to the target clothing image and the reference deformation information; the target image is generated according to the character image and the deformed clothing image, the deformation characteristic of the existing reference image in the character image can be obtained firstly, the deformed clothing image after the deformation of the target image is generated based on the deformation characteristic, the deformed clothing image is attached to the human body image in the character image under the condition that the original clothing characteristic of the target image is stored, and therefore the finally generated target image can accurately show the result of trying on the target clothing by a user.
With continued reference to FIG. 3, a flow 300 of another embodiment of a method for generating an image according to the present disclosure is shown, comprising the steps of:
step 301, obtaining a person image, obtaining human body analysis data, and determining reference clothing region information in the person image according to the human body analysis data.
In the present embodiment, an execution subject (for example, the server 105 shown in fig. 1) of the method for generating an image may acquire a person image from the internet or a terminal device in a wired or wireless manner, and acquire body analysis data, wherein the body analysis data may include data of the structure of each part of the human body, such as the positional relationship between the arm and the body part, the average proportional relationship, and the like. Then, the reference clothing region information in the person image is determined according to the human body analysis data, wherein the reference clothing region information refers to the region of the reference clothing in the person image or the region of the human body part of the reference clothing in the person image.
Step 302, according to the person image and the reference clothing region information, an image of the reference clothing in the person image is obtained.
In the present embodiment, an image of a region indicated by the reference garment region information in the person image may be extracted using an image recognition algorithm or a deep learning model, and the extracted image may be used as an image of a reference garment in the person image, that is, an image of a garment currently/worn before fitting by a person in the person image.
And 303, identifying the image of the reference garment to obtain deformation information of the reference garment, and determining the deformation information of the reference garment as the reference deformation information.
In this embodiment, the reference deformation information of the reference garment image, which is deformation information of the reference garment image (an image of a garment on a person already worn in the person image) compared to the original shape of the garment in a flat/unworn state of the reference garment, may be determined using an image recognition algorithm or a deep learning model.
And 304, acquiring a target clothing image, and generating a deformed clothing image corresponding to the target clothing image according to the target clothing image and the reference deformation information.
Step 305, generating a target image according to the person image and the deformed clothing image.
In this embodiment, the description of step 304 and step 305 is the same as the description of step 202 and step 203, and is not repeated here.
Compared with the embodiment described in fig. 2, the method for generating an image according to the present embodiment adds the steps of determining the reference clothing region information in the reference image according to the human body analysis data, and determining the reference clothing image in the person image according to the person image and the reference clothing region information, so as to improve the accuracy of determining the reference clothing image in the person image.
Optionally, identifying reference deformation information of a reference garment in the person image includes: inputting the figure image into a trained model network, and obtaining reference deformation information; the model network is based on loss training between the flat clothing image corresponding to the reference clothing image and the clothing image of the reference clothing in the undeformed state.
In this embodiment, the character image may be input into a pre-trained model network to output reference deformation information of a reference garment in the character image based on the model network. In the training process of the model network, loss training between a flat clothing image corresponding to a reference clothing image and a clothing image of the reference clothing in an undeformed state can be adopted, wherein the flat clothing image corresponding to the reference clothing image is a clothing image obtained after the clothing in the reference clothing image is unfolded, for example, if a person in the person image is in a posture that two arms of the person encircle the chest, sleeves of the clothing in the reference clothing image are folded in the chest, and the flat clothing image is a clothing image obtained after the sleeves of the clothing are opened/flattened towards two sides after the image processing is performed on the reference clothing image. The garment image of the reference garment in the undeformed state refers to a garment image of the reference garment image when the garment is not worn on a human body and is not deformed, and the garment image can be acquired based on the internet, user input or a deep learning model.
In this embodiment, the reference deformation information for identifying the reference garment in the person image is obtained based on a trained model network, and the model network is based on loss training between the flat garment image corresponding to the reference garment image and the garment image of the reference garment in the non-deformed state, so that the deformation characteristics of the reference garment can be learned in the process that the flat garment image corresponding to the reference garment image continuously fits the garment image of the reference garment in the non-deformed state, and the accuracy and efficiency for determining the reference deformation information are improved.
In some optional implementations of the embodiments described above in connection with fig. 2 and 3, generating a deformed garment image corresponding to the target garment image according to the target garment image and the reference deformation information includes: acquiring characteristic information of the target garment in the target garment image; acquiring reference clothing region information, wherein the reference clothing region information is used for representing the region of the reference clothing in the person image; determining initial deformation information according to the characteristic information of the target garment and the reference garment region information; and generating a deformation clothing image according to the posture information of the person in the person image, the target clothing image, the reference deformation information and the initial deformation information.
In this embodiment, the feature information of the target garment in the target garment image and the reference garment region information in the person image may be obtained based on an image recognition method, a feature calculation method, a deep learning model, or the like. The characteristic information of the target garment includes characteristics of the target garment, such as the shape of the target garment, the folds of the target garment, the color of the target garment, and the like. The reference clothing region information is for characterizing a dressing region of a person in the person image. Such as the upper body of the human body, the lower body of the human body, specific coordinate information in the character image, and the like.
The characteristics of the target garment image are adjusted/morphed based on the reference garment region information to obtain initial deformation information of what kind of deformation the target garment image should be deformed when fitting the target garment image to the reference garment region.
And recognizing the posture of the person in the person image to obtain posture information of the person, and performing deformation processing on the target garment in the target garment image according to the posture information of the person, the reference deformation information and the initial deformation information to generate a deformed garment image.
In this embodiment, initial deformation information is generated for the target garment according to the reference garment region information, and then a deformed garment image in the target garment image after deformation of the target garment is generated according to the person posture, the reference deformation information, and the initial deformation information, so that efficiency and accuracy of generating the deformed garment image can be improved.
In some alternative implementations of the embodiments described above in connection with fig. 2 and 3, the method for generating an image includes: acquiring attribute information of the person image, wherein the attribute information of the person image comprises at least one of the following items: characteristic information of a person in the person image, a human body area of the person, and posture information of the person; generating a target image according to the person image and the deformed clothing image, wherein the target image comprises: and generating a target image according to the person image, the deformed clothing image and the attribute information.
In this embodiment, attribute information of the person image may be obtained, and the person image may be fitted according to the deformed garment image and the attribute information of the person to generate a target image, where the attribute information of the person image may be feature information of the person in the person image, such as a color of an arm of the person, a facial expression feature of the person, a height and thinness of the person, and the like; the attribute information of the person image may be human body region information of a person in the person image, such as position information of an arm, position information of a leg, and the like of the person; the attribute information of the personal image may also be pose information, or the like of a person in the personal image.
In this embodiment, the accuracy and reality of generating the target image can be improved by referring to the attribute information of the person image when the target image is generated.
In some application scenarios, as shown in fig. 4, the method for generating an image includes:
step 401, obtaining a reference person image I inputted by a user, and analyzing to obtain a reference clothing region S according to human body analysis data (the human body analysis data can be obtained from a known data set) c Then, the tensor of the reference person image I and the tensor of the reference clothing region Sc are subjected to dot product operation to obtain the reference person image IClothing image I c . The reference person image includes a reference person, that is, a person having a fitting requirement.
It should be noted that in the application scenario, each parameter adopts a tensor expression form during calculation, and when the method described in the present application is specifically applied to generate a target image, each calculation parameter may also be in any parameter form that can be used for calculation, such as a matrix and a vector.
Step 402, referring to the clothing image I c And a reference garment region S c Splicing to obtain matrix (I) c ，S c ) The resulting matrix is then input into a first model network. Wherein the first model network may be a spatial transformation neural network for outputting a displacement transformation vector, i.e. reference deformation information F of the reference garment, based on the input image 1 . It should be noted that the reference garment exhibits a deformed/distorted garment state that fits the human figure posture, based on the reference deformation information F 1 The reference garment may be inverted to generate a garment image of the reference garment in a flat state.
During the training phase of the first model network, the loss function of the first model network may adopt the output (I' c ，S' c ) And (I) c ，S c ) Similarity of corresponding flattened states, including l' c ，S' c Corresponding regularization L1 loss L 1 And l' c Corresponding perceptual loss L perc And style loss L sty And a second order difference constraint L const . Wherein, I' c Characterization according to reference deformation information F 1 A flat reference garment image, S ', obtained by inverting the reference garment' c Characterization according to reference deformation information F 1 The reference garment may be inverted to obtain a flat reference garment region.
Step 403, obtaining clothing information R c ＝(C，M c ，S c ，M p ) Wherein C represents a target garment image; m c Characterizing the shape of the target garment, which may be obtained based on image recognition of the target garment image, or from a combination thereofKnown data sets are obtained; s c Characterizing the reference garment region (corresponding to the reference garment, i.e. the garment currently worn by the reference person in the reference person image), M p Pose key points that characterize the pose of the reference character as presented in the reference character image.
Step 404, the clothing information R is processed c Inputting the initial deformation information into a spatial transformation neural network structure in a second model network, and obtaining the initial deformation information F of the target clothing image C output by the second model network 2 。
Step 405, the target clothing image C is based on the initial deformation information F 2 After the deformation, a preliminarily deformed garment C 'can be obtained' g (preliminary deformation state of the target garment).
Step 406, referring the target image C to the clothing region S c The pose key point M of the reference character p And preliminarily deformed garment C' g And reference deformation information F of reference garment 1 And after matrix splicing, inputting the obtained data into a U-Net network structure in a second model network, and obtaining a deformed clothing image C' of the target clothing output by the U-Net network structure. The U-Net network structure comprises a down-sampling layer and an up-sampling layer, wherein the down-sampling layer is used for gradually showing environment information, and the up-sampling layer is used for restoring detail information of an image by combining information of each down-sampling layer and input information of the up-sampling process and gradually restoring image precision.
It should be noted that, in the process of generating the deformed garment image of the target garment, the reference deformation information F of the reference garment is referred to 1 The deformation state similar to that of the reference garment can be simulated on any target garment, so that the generated deformation garment image of the target garment not only contains the garment characteristics (such as color, texture, pattern and the like) of the target garment, but also contains the deformation characteristics of the reference garment after the person wears the target garment.
In the training stage of the second model network, the loss function of the second model network can adopt the perceptual loss L perc Loss of style L sty Regularization L1 loss L 1 To counter the loss L adv And second order differential constraint L onst Loss functionIs a deformed garment image C 'or an intermediate result of the model (e.g. a preliminarily deformed garment C' g ) And a reference garment image.
Step 407, obtaining the information R p Wherein Rp = (I) r ，S r ，C'，C，M p ，A c ) Wherein, I r The information of the clothing and arm area of the reference person image I which is needed to be reserved by the reference person is removed; s. the r Is a human body area except clothes in the human body analysis label; a. The c Is a simulated arm color, and can be determined based on the average value of the skin colors of the arm area in the reference character image I.
Step 408, the information R is processed p Inputting the target image I 'into the third model network to obtain the target image I' output by the third model network, namely, the dressing result of the person in the reference person image wearing the target garment/the fitting result of the target garment. Wherein the third model network can be a U-Net network structure. It is understood that the target image I' includes both the clothing characteristics (such as color, texture, pattern, etc.) of the target clothing C and the deformation characteristics (such as wrinkle area, deformation degree) of the reference clothing, and the identity characteristics (such as face, stature) of the reference person I.
In the training stage of the third model network, the loss function of the third model network may include a perceptual loss L perc Style loss L sty Regularization L1 loss L 1 And to combat the loss L adv The constraint object of the loss function is the output I' of the third model network and the reference character image I.
With further reference to fig. 5, as an implementation of the methods illustrated in the above figures, the present disclosure provides an embodiment of an apparatus for generating an image, which corresponds to the method embodiments illustrated in fig. 2 and 3, and which is particularly applicable in various electronic devices.
As shown in fig. 5, the apparatus for generating an image of the present embodiment includes: identification unit 501, acquisition unit 502, and generation unit 503. The identification unit is configured to acquire a person image and identify reference deformation information of a reference garment in the person image, wherein the reference deformation information is used for representing deformation information of the reference garment in the person image; the acquiring unit is configured to acquire a target garment image and generate a deformed garment image corresponding to the target garment image according to the target garment image and the reference deformation information; and the generating unit is configured to generate a target image according to the person image and the deformation clothing image.
In some embodiments, the identification unit comprises: the first acquisition module is configured to acquire human body analysis data and determine reference clothing region information in the character image according to the human body analysis data, wherein the reference clothing region information is used for representing the region of the reference clothing in the character image; the second acquisition module is configured to acquire an image of a reference garment in the person image according to the person image and the reference garment region information; the first determining module is configured to identify the image of the reference garment to obtain deformation information of the reference garment, and determine the deformation information of the reference garment as the reference deformation information.
In some embodiments, the identification unit comprises: the identification module is configured to input the figure image into the trained model network and obtain reference deformation information; the model network is based on loss training between the flat clothing image corresponding to the reference clothing image and the clothing image of the reference clothing in the undeformed state.
In some embodiments, the obtaining unit comprises: the third acquisition module is configured to acquire characteristic information of the target garment in the target garment image; the fourth acquisition module is configured to acquire reference clothing region information, wherein the reference clothing region information is used for representing the region of the reference clothing in the person image; a second determining module configured to determine initial deformation information according to the feature information of the target garment and the reference garment region information; and the first generation module is configured to generate a deformed clothing image according to the posture information of the person in the person image, the target clothing image, the reference deformation information and the initial deformation information.
In some embodiments, an apparatus comprises: a fifth acquiring module configured to acquire attribute information of the person image, wherein the attribute information of the person image includes at least one of: characteristic information of a person in the person image, a human body area of the person, and posture information of the person; a generation unit comprising: and the second generation module is configured to generate a target image according to the person image, the deformed clothing image and the attribute information.
The units in the apparatus 500 described above correspond to the steps in the method described with reference to fig. 2 and 3. Thus, the operations, features and technical effects that can be achieved by the above-described method for generating information are also applicable to the apparatus 500 and the units included therein, and are not described herein again.
According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.
As shown in fig. 6, is a block diagram of an electronic device 600 for a method of generating an image according to an embodiment of the application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 6, the electronic apparatus includes: one or more processors 601, memory 602, and interfaces for connecting the various components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). One processor 601 is illustrated in fig. 6.
The memory 602 is a non-transitory computer readable storage medium as provided herein. Wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method for generating an image provided herein. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the method for generating an image provided by the present application.
The memory 602, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the method for generating an image in the embodiment of the present application (e.g., the identifying unit 501, the obtaining unit 502, and the generating unit 503 shown in fig. 5). The processor 601 executes various functional applications of the server and data processing, i.e., implements the method for generating an image in the above-described method embodiments, by running non-transitory software programs, instructions, and modules stored in the memory 602.
The memory 602 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the electronic device for extracting the video clip, and the like. Further, the memory 602 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 602 optionally includes memory remotely located from the processor 601, and these remote memories may be connected over a network to an electronic device for retrieving video clips. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the method for generating an image may further include: an input device 603, an output device 604, and a bus 605. The processor 601, the memory 602, the input device 603, and the output device 604 may be connected by a bus 605 or other means, and are exemplified by the bus 605 in fig. 6.
The input device 603 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic apparatus for extracting the video clip, such as an input device of a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a track ball, a joystick, or the like. The output devices 604 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present application can be achieved.
The above-described embodiments are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (12)

1. A method for generating an image, comprising:
acquiring a person image, and identifying reference deformation information of a reference garment in the person image, wherein the reference deformation information is used for representing the deformation information of the reference garment in the person image;
acquiring a target clothing image, and generating a deformed clothing image corresponding to the target clothing image according to the target clothing image and the reference deformation information;
and generating a target image according to the figure image and the deformed clothing image.
2. The method of claim 1, wherein the identifying reference deformation information for a reference garment in the person image comprises:
acquiring human body analysis data, and determining reference clothing region information in the person image according to the human body analysis data, wherein the reference clothing region information is used for representing the region of the reference clothing in the person image;
acquiring an image of a reference garment in the person image according to the person image and the reference garment region information;
and identifying the image of the reference garment to obtain the deformation information of the reference garment, and determining the deformation information of the reference garment as the reference deformation information.
3. The method of claim 2, wherein the identifying reference deformation information for a reference garment in the person image comprises:
inputting the figure image into a trained model network, and obtaining the reference deformation information;
wherein the model network is based on loss training between the flat garment image corresponding to the reference garment image and the garment image of the reference garment in an undeformed state.
4. The method of claim 1, wherein the generating a deformed garment image corresponding to the target garment image from the target garment image and the reference deformation information comprises:
acquiring characteristic information of the target garment in the target garment image;
acquiring reference clothing region information, wherein the reference clothing region information is used for representing the region of the reference clothing in the person image;
determining initial deformation information according to the characteristic information of the target garment and the reference garment region information;
and generating the deformation clothing image according to the posture information of the person in the person image, the target clothing image, the reference deformation information and the initial deformation information.
5. The method of claim 1, wherein the method comprises:
acquiring attribute information of the person image, wherein the attribute information of the person image comprises at least one of the following items: feature information of a person in the person image, a human body area of the person, and posture information of the person;
generating a target image according to the person image and the deformed clothing image, wherein the generating comprises:
and generating the target image according to the person image, the deformed clothing image and the attribute information.
6. An apparatus for generating an image, comprising:
the identification unit is configured to acquire a person image and identify reference deformation information of a reference garment in the person image, wherein the reference deformation information is used for representing deformation information of the reference garment in the person image;
the acquisition unit is configured to acquire a target garment image and generate a deformed garment image corresponding to the target garment image according to the target garment image and the reference deformation information;
a generating unit configured to generate a target image from the person image and the morphed clothing image.
7. The apparatus of claim 6, wherein the identifying unit comprises:
the first acquisition module is configured to acquire human body analysis data and determine reference clothing region information in the person image according to the human body analysis data, wherein the reference clothing region information is used for representing the region of the reference clothing in the person image;
a second obtaining module configured to obtain an image of a reference garment in the person image according to the person image and the reference garment region information;
the first determining module is configured to identify the image of the reference garment to obtain deformation information of the reference garment, and determine the deformation information of the reference garment as the reference deformation information.
8. The apparatus of claim 7, wherein the identifying unit comprises:
the identification module is configured to input the figure image into a trained model network and obtain the reference deformation information;
wherein the model network is based on loss training between the flat garment image corresponding to the reference garment image and the garment image of the reference garment in an undeformed state.
9. The apparatus of claim 6, wherein the obtaining unit comprises:
a third obtaining module configured to obtain feature information of a target garment in the target garment image;
a fourth obtaining module configured to obtain reference clothing region information, wherein the reference clothing region information is used for representing a region of the reference clothing in the person image;
a second determining module configured to determine initial deformation information according to the characteristic information of the target garment and the reference garment region information;
a first generating module configured to generate the deformed clothing image according to the posture information of the person in the person image, the target clothing image, the reference deformation information, and the initial deformation information.
10. The apparatus of claim 6, wherein the apparatus comprises:
a fifth obtaining module configured to obtain attribute information of the personal image, wherein the attribute information of the personal image includes at least one of: feature information of a person in the person image, a human body area of the person, and posture information of the person;
the generation unit includes:
a second generating module configured to generate the target image according to the person image, the deformed garment image, and the attribute information.
11. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.
12. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-5.
CN202110758294.4A 2021-07-05 2021-07-05 Method and apparatus for generating image Pending CN115578745A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202110758294.4A CN115578745A (en) 2021-07-05 2021-07-05 Method and apparatus for generating image
PCT/CN2022/098625 WO2023279922A1 (en) 2021-07-05 2022-06-14 Method and apparatus for generating image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110758294.4A CN115578745A (en) 2021-07-05 2021-07-05 Method and apparatus for generating image

Publications (1)

Publication Number Publication Date
CN115578745A true CN115578745A (en) 2023-01-06

Family

ID=84578772

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110758294.4A Pending CN115578745A (en) 2021-07-05 2021-07-05 Method and apparatus for generating image

Country Status (2)

Country Link
CN (1) CN115578745A (en)
WO (1) WO2023279922A1 (en)

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11439194B2 (en) * 2017-08-11 2022-09-13 North Carolina State University Devices and methods for extracting body measurements from 2D images
US11080817B2 (en) * 2019-11-04 2021-08-03 Adobe Inc. Cloth warping using multi-scale patch adversarial loss
CN111709874B (en) * 2020-06-16 2023-09-08 北京百度网讯科技有限公司 Image adjustment method, device, electronic equipment and storage medium
CN112258269B (en) * 2020-10-19 2024-05-28 武汉纺织大学 Virtual fitting method and device based on 2D image
CN113052980B (en) * 2021-04-27 2022-10-14 云南大学 Virtual fitting method and system

Also Published As

Publication number Publication date
WO2023279922A1 (en) 2023-01-12

Similar Documents

Publication Publication Date Title
CN112541963B (en) Three-dimensional avatar generation method, three-dimensional avatar generation device, electronic equipment and storage medium
CN113643412B (en) Virtual image generation method and device, electronic equipment and storage medium
JP2020522285A (en) System and method for whole body measurement extraction
CN111860167B (en) Face fusion model acquisition method, face fusion model acquisition device and storage medium
CN113658309B (en) Three-dimensional reconstruction method, device, equipment and storage medium
JP2021108206A (en) Image adjustment method, apparatus, electronic device, storage medium and program
US9058056B2 (en) System and method of dynamically generating a frequency pattern to realize the sense of touch in a computing device
CN111968203B (en) Animation driving method, device, electronic equipment and storage medium
CN112270711B (en) Model training and posture prediction method, device, equipment and storage medium
CN113129450A (en) Virtual fitting method, device, electronic equipment and medium
CN113240778A (en) Virtual image generation method and device, electronic equipment and storage medium
CN111709875A (en) Image processing method, image processing device, electronic equipment and storage medium
CN111754303A (en) Method and apparatus for virtual changing of clothing, device and medium
CN114549710A (en) Virtual image generation method and device, electronic equipment and storage medium
CN116228867B (en) Pose determination method, pose determination device, electronic equipment and medium
CN112634282A (en) Image processing method and device and electronic equipment
CN111523467A (en) Face tracking method and device
CN114386503A (en) Method and apparatus for training a model
CN116311519B (en) Action recognition method, model training method and device
CN112116548A (en) Method and device for synthesizing face image
CN115761855B (en) Face key point information generation, neural network training and three-dimensional face reconstruction method
CN115578745A (en) Method and apparatus for generating image
CN112200169B (en) Method, apparatus, device and storage medium for training a model
CN114119154A (en) Virtual makeup method and device
CN111738293A (en) Point cloud data processing method and device, electronic equipment and readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination