WO2021232708A1 - 一种图像处理方法及电子设备 - Google Patents

一种图像处理方法及电子设备 Download PDF

Info

Publication number
WO2021232708A1
WO2021232708A1 PCT/CN2020/128812 CN2020128812W WO2021232708A1 WO 2021232708 A1 WO2021232708 A1 WO 2021232708A1 CN 2020128812 W CN2020128812 W CN 2020128812W WO 2021232708 A1 WO2021232708 A1 WO 2021232708A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
processed
texture
shape
reference object
Prior art date
Application number
PCT/CN2020/128812
Other languages
English (en)
French (fr)
Inventor
郭益林
张知行
黄星
方轲
宋丛礼
Original Assignee
北京达佳互联信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京达佳互联信息技术有限公司 filed Critical 北京达佳互联信息技术有限公司
Publication of WO2021232708A1 publication Critical patent/WO2021232708A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/18—Image warping, e.g. rearranging pixels individually
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image

Definitions

  • This application relates to the field of image processing technology, and in particular to an image processing method and electronic equipment.
  • Image processing technology is a technology that converts one type of image into another type of image. For example: convert semantic segmentation images into real street scene images, gray images into color images, day images into dark night images, low-pixel images into high-pixel images, etc.
  • the generative adversarial network is a deep learning model.
  • the generative confrontation network GAN has been widely used in the field of image processing technology.
  • the age change function, gender change function, and comic face function in applications have all applied the generative confrontation network GAN to process the original image to obtain the target image.
  • the accuracy of the target image obtained by using the generative adversarial network GAN to process the object to be processed with a large difference between the shape of the reference object and the shape of the reference object is poor.
  • the present application provides an image processing method and electronic equipment to solve the problem of poor accuracy when processing an object with a large difference in shape from the shape of a reference object to be processed as a picture of the reference object type by using a generative countermeasure network GAN.
  • the embodiments of the present application provide an image processing method.
  • the method includes: obtaining an original image; the original image includes an object to be processed; The shape of the object is transformed into the shape of the reference object to obtain an intermediate image; the object to be processed after the image is deformed in the intermediate image is textured, and the texture of the object to be processed after the texture processing is processed into the texture of the reference object to obtain the target image.
  • an image processing device including: an acquisition module configured to acquire an original image; the original image includes an object to be processed; an image deformation processing module configured to perform image deformation on the object to be processed , Transform the shape of the deformed object to be processed into the shape of the reference object to obtain an intermediate image; the image texture processing module is configured to perform texture processing on the deformed object to be processed in the intermediate image, and the texture processed The texture of the object to be processed is processed as the texture of the reference object, and the target image is obtained.
  • an electronic device including: a processor; and a memory for storing instructions executable by the processor.
  • the processor is configured to execute the instructions to implement the above-mentioned first aspect and the image processing method shown in any one of the possible implementation manners of the first aspect.
  • a computer-readable storage medium When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute any of the first aspect and the first aspect.
  • One possible implementation is shown in the image processing method.
  • a computer program product which can be directly loaded into the internal memory of a computer and contains software code.
  • the computer program can realize the first aspect and the first aspect after being loaded and executed by the computer.
  • Fig. 1 is a network architecture of the present application according to an exemplary embodiment
  • Fig. 2 is a schematic flowchart showing an image processing method according to an exemplary embodiment
  • Fig. 3 is a schematic diagram showing key points of a human face according to an exemplary embodiment
  • Fig. 4 is a schematic diagram showing an original image and a target image according to an exemplary embodiment
  • Fig. 5 is a schematic flowchart showing an image processing method according to another exemplary embodiment
  • Fig. 6 is a block diagram showing an image processing device according to an exemplary embodiment
  • Fig. 7 is a schematic diagram showing the structure of an electronic device according to an exemplary embodiment.
  • words such as “exemplary” or “for example” are used as examples, illustrations, or illustrations. Any embodiment or design solution described as “exemplary” or “for example” in the embodiments of the present application should not be construed as being more preferable or advantageous than other embodiments or design solutions. To be precise, words such as “exemplary” or “for example” are used to present related concepts in a specific manner.
  • At least one refers to one or more.
  • Multiple means two or more.
  • "and/or” is merely an association relationship describing associated objects, indicating that there can be three types of relationships, for example, A and/or B, which can mean that A exists alone, and A and A exist at the same time. B, there are three cases of B alone.
  • the character "/" in this text generally indicates that the associated objects before and after are in an "or" relationship.
  • FIG. 1 shows the network architecture of the present application according to an exemplary embodiment.
  • the network architecture shown in FIG. 1 may include a server 101 and multiple terminal devices 102.
  • two terminal devices 102 are taken as an example for description.
  • the server 101 and each terminal device 102 are connected through a network.
  • the terminal device 102 may be used to receive the original image and process the original image.
  • the terminal device 102 may send the original image to the server 101, and receive the processing result of the original image sent by the server 101.
  • the terminal device 102 may be any one of computing devices such as a palmtop computer, a notebook computer, a smart phone, a tablet computer, or a desktop computer.
  • the server 101 may be configured to receive the original image sent by the terminal device 102, and after processing the received original image, send the processing result of the original image to the terminal device 102.
  • the server 101 may be a server, a server cluster composed of multiple servers, or a cloud computing service center.
  • Fig. 2 is a schematic flowchart showing an image processing method according to an exemplary embodiment. Applied to the terminal device 102, the method shown in FIG. 2 may include the following steps:
  • the terminal device 102 obtains the original image in real time through a camera. In some embodiments, the terminal device 102 selects the original image indicated by the user selection instruction from the stored images according to the received user selection instruction.
  • the user selection instruction may be that the terminal device 102 receives a user selection instruction sent by the user through the input unit.
  • the object to be processed included in the original image may be at least one of people, animals, plants, or other objects.
  • the terminal device receives an image processing instruction of "processing the object to be processed in the original image into an object of the same type as the reference object" in the application selected by the user.
  • the original image is an image obtained in real time by the camera of the terminal device.
  • the object to be processed is a person in the original image who is at the age of eighteen years old, and the object of the same type as the reference object is a person who is just past infancy.
  • the image processing instruction is used to process a person in the original image who is at the age of eighteen years old as a person who is just past infancy corresponding to the object.
  • S101 Perform image deformation on the object to be processed, and transform the shape of the object to be processed after the image deformation is transformed into the shape of the reference object to obtain an intermediate image.
  • the terminal device 102 may perform image deformation on the object to be processed in the original image through the following steps, and transform the shape of the object to be processed after the image deformation into the shape of the reference object to obtain an intermediate image.
  • Step 1 The terminal device 102 acquires multiple first key points used to characterize the object to be processed.
  • One of the plurality of first key points is used to characterize a part of the object to be processed.
  • the key point is a stable pixel point in the image that does not rotate and can overcome gray-scale inversion.
  • Key points represent the pixel points of the image or target in a similar or at least very similar invariant form in similar images containing the same scene or the same target. For example, the middle point of the eyeball in a human face, multiple images are collected from different angles, and the middle point of the eyeball of the human face in different images can be identified.
  • the terminal device 102 may use a key point extraction algorithm to obtain the first key point in the object to be processed.
  • the key point is a stable pixel point that does not rotate and can overcome gray-scale inversion.
  • the key point extraction algorithm can be Moravec operator, Forstner operator or Harris operator.
  • FIG. 3 is a schematic diagram showing the key points of the human face according to an exemplary embodiment.
  • Step 2 The terminal device 102 acquires multiple second key points of the reference object.
  • the multiple second key points are used to characterize the shape of the target reference object.
  • the reference object may be the average face of a person in the infancy stage obtained from a plurality of images containing the faces of the person in the infancy stage.
  • the average face refers to the synthetic face of a certain group obtained through computer technology processing.
  • Step 3 The terminal device 102 deforms the image of the object to be processed according to the plurality of first key points and the plurality of second key points, and transforms the shape of the object to be processed after the image deformation into the shape of the reference object to obtain an intermediate image.
  • the terminal device 102 uses a liquefaction model or a liquefaction algorithm to perform image deformation on the object to be processed in the original image, and transforms the shape of the object to be processed after the image deformation into the shape of the reference object to obtain an intermediate image.
  • the liquefaction model or the liquefaction algorithm is used to: firstly, the object to be processed in the original image is triangulated according to the multiple first key points of the object to be processed in the original image to obtain multiple first triangular patches. According to a plurality of second key points of the reference object, the reference object is divided into triangular patches to obtain a plurality of second triangular patches.
  • the terminal device 102 matches the plurality of first key points and the plurality of second key points according to the plurality of first key points and the plurality of second key points, and obtains a plurality of first key points with position correspondences.
  • the terminal device 102 may use a key point matching algorithm to establish the position correspondence between the first key point and the second key point, where the key point matching algorithm may be a correlation coefficient method, a relaxation method, or a least square method.
  • the terminal device 102 obtains the pixel point transformation matrix of the object to be processed and the reference object based on the first key point and the second key point having a position correspondence.
  • the terminal device 102 performs image deformation on the object to be processed based on the acquired pixel point transformation matrix to obtain the object to be processed after the image is deformed.
  • the second key point of the reference object represents the shape of the reference object
  • the first key point of the object to be processed represents the shape of the object to be processed.
  • the terminal device 102 processes the first key point of the object to be processed according to the second key point of the reference object that matches it.
  • the object to be processed after the image is deformed is closer to the shape of the reference object.
  • the terminal device 102 may also use any one of similarity transformation, affine transformation, or projection transformation to perform shape mapping processing on the original image to obtain an intermediate image.
  • S102 Perform texture processing on the object to be processed after the image is deformed in the intermediate image, and process the texture of the object to be processed after the texture processing as the texture of the reference object to obtain the target image.
  • the terminal device 102 inputs the intermediate image into the generative countermeasure network GAN, and the generative countermeasure network GAN transforms the texture of the object to be processed after the image deformation process in the intermediate image into the texture of the reference object according to the texture characteristics of the reference object, and obtains the target image And output.
  • the generative network in the generative confrontation network GAN generates an output image based on the intermediate image and inputs the output image into the discriminant network in the generative confrontation network GAN; the discriminant network is constructed based on the reference object; the discriminant network judges whether the output image is true or false, And the judgment result is fed back to the generation network; the generation network adjusts the parameters of the generated output image according to the judgment result, and generates a new output image, until the discrimination network determines that the new output image generated by the generation network is true, the generative confrontation network GAN will determine The new output image that is true is output as the target image.
  • the terminal device 102 displays the target image to the user through an output device. Based on the example of S101, the terminal device 102 displays the target image to the user through the display (for example: screen) of the terminal device 102.
  • Fig. 4 is a schematic diagram showing an original image and a target image according to an exemplary embodiment.
  • the first column of images is the original image
  • the second column of images is the target image after the original image is processed.
  • the embodiment of this application uses an image processing solution that separates the deformation in image processing from the texture mapping.
  • the original image is deformed to obtain an intermediate image, so that the shape of the object to be processed in the intermediate image is close to the shape of the reference object, and then
  • the texture of the object to be processed after the image is deformed in the intermediate image is processed into the texture of the reference object, and the target image is obtained.
  • the terminal device 102 Due to the separation of image deformation and texture processing, the terminal device 102 only focuses on the shape mapping of the object to be processed in the original image and the shape of the reference object when performing image deformation, without considering other factors. At that time, only focus on the texture mapping between the texture of the object to be processed and the texture of the reference object.
  • Fig. 5 is a schematic flowchart showing an image processing method according to another exemplary embodiment. The method shown in FIG. 5 may include the following steps:
  • S200 The terminal device 102 obtains an original image, and the original image includes an object to be processed.
  • the method of obtaining the original image refer to the description of S100.
  • the terminal device 102 sends an original image and an image processing instruction to the server 101.
  • the terminal device 102 may send the original image and the image processing instruction to the server 101 in two times, or the terminal device 102 may send the original image and the image processing instruction to the server 101 once.
  • S202 The server 101 performs image deformation on the object to be processed in the original image according to the original image and the image processing instruction, and transforms the shape of the object to be processed after the image deformation into the shape of the reference object to obtain an intermediate image.
  • the method of obtaining the intermediate image refer to the description of S102.
  • S203 The server 101 performs texture processing on the object to be processed after the image is deformed in the intermediate image, and processes the texture of the object to be processed after the texture processing as the texture of the reference object to obtain the target image.
  • the terminal device 102 performs texture processing on the object to be processed after the image is deformed in the intermediate image in S102, and processes the texture of the object to be processed after the texture processing into the texture of the reference object.
  • S204 The server 101 sends the target image to the terminal device 102.
  • the terminal device 102 presents the target image to the user through an output device.
  • the embodiment of the application adopts an image processing scheme that separates the deformation in image processing from texture mapping.
  • an intermediate image is obtained through image deformation, so that the shape of the object to be processed in the intermediate image is close to the shape of the reference object, and then the texture mapping method is used
  • the texture of the object to be processed in the intermediate image is processed as the texture of the reference object to obtain the target image. Due to the separation of image deformation and texture mapping, the server 101 only focuses on the shape of the object to be processed in the original image and the shape mapping of the reference object when performing image deformation, without considering other factors, when performing texture processing on the intermediate image , Only focus on the texture mapping of the object to be processed in the intermediate image and the texture of the reference object.
  • the embodiment of the present application may divide the image processing apparatus into functional modules according to the foregoing method examples.
  • each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or software functional modules. It should be noted that the division of modules in the embodiments of the present application is illustrative, and is only a logical function division, and there may be other division methods in actual implementation.
  • Fig. 6 is a block diagram showing an image processing device according to an exemplary embodiment.
  • the image processing device 90 includes an acquisition module 901 configured to acquire an original image; the original image includes an object to be processed; an image deformation processing module 902 is configured to deform the image of the object to be processed, and transform the image to be processed.
  • the shape of the processing object is transformed into the shape of the reference object to obtain an intermediate image;
  • the image texture processing module 903 is configured to perform texture processing on the object to be processed after the image deformation processing in the intermediate image, and the texture of the object to be processed after the texture processing Process the texture of the reference object to obtain the target image.
  • the image processing apparatus 90 is a terminal device, in conjunction with FIG.
  • the acquisition module 901 is configured to perform S100
  • the image deformation processing module 902 is configured to perform S101
  • the image texture processing module 903 is configured to S102 is executed.
  • the image processing device 90 is a server, with reference to FIG. 5, the acquiring module 901 is configured to perform the receiving step in S201, the image deformation processing module 902 is configured to be used in S202, and the image texture processing module 903 is configured to Go to S203.
  • the acquiring module 901 is further configured to: acquire multiple first key points of the object to be processed; acquire multiple second key points of the reference object; the image deformation processing module 902 is specifically configured to: The first key point and the multiple second key points are used to deform the image of the object to be processed.
  • the obtaining module 901 is further configured to: obtain the first key point and the second key point having a position correspondence relationship based on the multiple first key points and the multiple second key points; The first key point and the second key point of, obtain the pixel point transformation matrix of the object to be processed and the reference object; the image deformation processing module 902 is specifically configured to deform the image of the object to be processed based on the pixel point transformation matrix, and the transformed image The shape of the object to be processed is transformed into the shape of the reference object to obtain an intermediate image.
  • the image deformation processing module 902 may be configured to: divide the object to be processed into triangular patches according to a plurality of first key points; divide the reference object into triangular patches according to a plurality of second key points ; Obtain the triangle face of the object to be processed and the triangle face of the reference object with position correspondence; according to the triangle face of the object to be processed and the triangle face of the reference object with position correspondence, the image of the object to be processed is deformed, The shape of the object to be processed after the image is deformed is transformed into the shape of the reference object to obtain an intermediate image.
  • the image texture processing module 903 may be configured to: input the intermediate image into the generative countermeasure network GAN; the generative countermeasure network GAN transforms the image to be processed in the intermediate image according to the texture characteristics of the reference object The texture of the object is processed as the texture of the reference object, and the target image is obtained and output.
  • the image texture processing module 903 may be configured to: the generation network in the generative countermeasure network GAN generates an output image according to the intermediate image and inputs the output image into the discriminant network in the generative countermeasure network GAN; the discriminant network is based on Constructed with reference to the object; the judgment network judges the true or false of the output image, and the judgment result is fed back to the generation network; the generation network adjusts the parameters of the generated output image according to the judgment result, and generates a new output image, until the judgment network determines the generation network generated The new output image is true, and the generative adversarial network GAN will output the new output image determined to be true as the target image.
  • the image processing device provided by the embodiment of the present application adopts an image processing scheme that separates the deformation in image processing from texture mapping.
  • the shape mapping process is performed to obtain an intermediate image, so that the shape of the object to be processed in the intermediate image is the same as the object type of the object.
  • the shape of is close, and then the original image is mapped to the intermediate image through the texture mapping method to obtain the target image.
  • the image processing device only focuses on the shape of the object to be processed in the original image and the shape mapping of the target type of object when performing image deformation.
  • the intermediate image is subjected to texture mapping, only focus on the texture mapping between the original image and the intermediate image.
  • Fig. 7 is a block diagram showing an electronic device 1000 according to an exemplary embodiment.
  • the electronic device 100 includes a processor 1010 and a memory 1020 for storing executable instructions of the processor;
  • processor 1010 is configured to execute:
  • the original image includes an object to be processed
  • Texture processing is performed on the object to be processed after the image is deformed in the intermediate image, and the texture of the object to be processed after the texture processing is processed into the texture of the reference object to obtain a target image.
  • processor 1010 may be configured to execute:
  • the processing object performs image deformation, and the shape of the object to be processed after the image deformation is transformed into the shape of the reference object to obtain an intermediate image.
  • processor 1010 may be configured to execute:
  • Image deformation is performed on the object to be processed based on the pixel point transformation matrix, and the shape of the object to be processed after the image deformation is transformed into the shape of a reference object to obtain an intermediate image.
  • processor 1010 may be configured to execute:
  • the image of the object to be processed is deformed, and the shape of the object to be processed after the image deformation is transformed into a reference The shape of the object to get the intermediate image.
  • processor 1010 may be configured to execute:
  • the generative adversarial network GAN processes the texture of the object to be processed after image deformation processing in the intermediate image into the texture of the reference object according to the texture feature of the reference object, and obtains and outputs a target image.
  • processor 1010 may be configured to execute:
  • the generation network in the generative confrontation network GAN generates an output image according to the intermediate image and inputs the output image to the discriminant network in the generative confrontation network GAN; the discriminant network is constructed based on the reference object;
  • the discrimination network judges the authenticity of the output image, and feeds back the judgment result to the generation network
  • the generation network adjusts the parameters of the generated output image according to the judgment result, and generates a new output image, until the discriminant network determines that the new output image generated by the generation network is true, the generative confrontation network GAN will The new output image determined to be true is output as the target image.
  • the embodiments of the present application also provide a computer-readable storage medium with a computer program stored on the computer-readable storage medium.
  • the computer program runs on a computer, the computer executes any of the above-provided embodiments. The action or step mentioned.
  • the embodiment of the application also provides a chip.
  • the chip integrates a circuit and one or more interfaces for realizing the functions of the above-mentioned image processing device.
  • the functions supported by the chip may include processing actions based on the embodiment described in FIG. 2 or FIG. 5, which will not be repeated here.
  • a person of ordinary skill in the art can understand that all or part of the steps for implementing the above-mentioned embodiments can be completed by a program instructing related hardware.
  • the program can be stored in a computer-readable storage medium.
  • the aforementioned storage medium may be a read-only memory, a random access memory, and the like.
  • the aforementioned processing unit or processor may be a central processing unit, a general-purpose processor, an application specific integrated circuit (ASIC), a microprocessor (digital signal processor, DSP), and a field programmable gate array (field programmable gate array).
  • ASIC application specific integrated circuit
  • DSP digital signal processor
  • field programmable gate array field programmable gate array
  • FPGA field programmable gate array
  • the embodiments of the present application also provide a computer program product containing instructions, which when the instructions run on a computer, cause the computer to execute any one of the methods in the foregoing embodiments.
  • the computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present application are generated in whole or in part.
  • the computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • computer instructions can be transmitted from a website, computer, server, or data center through a cable (such as Coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL) or wireless (such as infrared, wireless, microwave, etc.) transmission to another website site, computer, server or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer, or may include one or more data storage devices such as a server or a data center that can be integrated with the medium.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).
  • the foregoing devices for storing computer instructions or computer programs provided in the embodiments of the present application are non-transitory. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Generation (AREA)
  • Image Processing (AREA)

Abstract

本申请关于一种图像处理方法及电子设备，以至少解决利用生成式对抗网络GAN，将形状与参考对象的形状差异大的待处理对象处理为参考对象的类型的图片时，准确性差的问题。该方法包括：获取原始图像；原始图像包括待处理对象；对待处理对象进行图像变形，将图像变形后的待处理对象的形状变换为参考对象的形状，得到中间图像；对中间图像中图像变形后的待处理对象进行纹理处理，将纹理处理后的待处理对象的纹理处理为参考对象的纹理，得到目标图像。

Description

一种图像处理方法及电子设备
相关申请的交叉引用
本申请要求于2020年05月21日提交中国专利局、申请号为202010437578.9、发明名称为“一种图像处理方法、装置、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，尤其涉及一种图像处理方法及电子设备。
背景技术
图像处理技术是将一种类型的图像转换为另一种类型的图像的技术。例如：将语义分割图像转换为真实街景图像，灰色图像转换为彩色图像，白天的图像转换为黑夜的图像，低像素图像转换为高像素的图像等。
生成式对抗网络(generative adversarial network，GAN)是一种深度学习模型。近年，生成式对抗网络GAN广泛应用于图像处理技术领域，例如：应用程序中的年龄变化功能、性别变化功能、漫画脸功能等都应用了生成式对抗网络GAN对原始图像进行处理得到目标图像。然而，使用生成式对抗网络GAN对形状与参考对象的形状差异大的待处理对象进行处理得到的目标图像的准确性差。
发明内容
本申请提供一种图像处理方法及电子设备，以解决利用生成式对抗网络GAN，将形状与参考对象的形状差异大的待处理对象，处理为参考对象的类型的图片时，准确性差的问题。
根据本申请实施例的第一方面，本申请实施例提供一种图像处理方法，该方法包括：获取原始图像；原始图像包括待处理对象；对待处理对象进行图像变形，将图像变形后的待处理对象的形状变换为参考对象的形状，得到中间图像；对中间图像中图像变形后的待处理对象进行纹理处理，将纹理处理后的待处理对象的纹理处理为参考对象的纹理，得到目标图像。
根据本申请实施例的第二方面，提供一种图像处理装置，包括：获取模块，被配置为获取原始图像；原始图像包括待处理对象；图像变形处理模块，被配置为对待处理对象进行图像变形，将图像变形后的待处理对象的形状变换为参考对象的形状，得到中间图像；图像纹理处理模块，被配置为对中间图像中图像变形后的待处理对象进行纹理处理，将纹理处理后的待处理对象的纹理处理为参考对象的纹理，得到目标图像。
根据本申请实施例的第三方面，提供一种电子设备，包括：处理器；用于存储处理器可执行指令的存储器。其中，处理器被配置为执行所述指令，以实现上述第一方面以及第一方面的任一种可能的实现方式所示的图像处理方法。
根据本申请实施例的第四方面，提供一种计算机可读存储介质，当该存储介质中的指令由电子设备的处理器执行时，使得电子设备能够执行如第一方面以及第一方面的任一种可能的实现方式所示的图像处理方法。
根据本申请实施例的第五方面，提供一种计算机程序产品，可直接加载到计算机的内部存储器中，并含有软件代码，该计算机程序经由计算机载入并执行后能够实现第一方面以及第一方面的任一种可能的实现方式所示的图像处理方法。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本申请。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本申请实施例，并与说明书一起用于解释本申请的原理，并不构成对本申请的不当限定。
图1是根据一示例性实施例示出的本申请的网络架构；
图2是根据一示例性实施例示出的一种图像处理方法的流程示意图；
图3是根据一示例性实施例示出的人脸关键点的示意图；
图4是根据一示例性实施例示出的原始图像与目标图像的示意图；
图5是根据另一示例性实施例示出的一种图像处理方法的流程示意图；
图6是根据一示例性实施例示出的一种图像处理装置框图；
图7是根据一示例性实施例示出的一种电子设备的结构示意图。
具体实施方式
在本申请的实施例中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请实施例中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
需要说明的是，本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施。以下示例性实施例中所描述的实施方式并不代表与本申请相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本申请的一些方面相一致的装置和方法的例子。
在本申请实施例中，“至少一个”是指一个或多个。“多个”是指两个或两个以上。
在本申请实施例中，“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。
图1是根据一示例性实施例示出的本申请的网络架构。图1所示的网络架构可以包括服务器101和多个终端设备102。图1中以两个终端设备102为例进行说明。其中，服务器101与每个终端设备102均通过网络连接。
终端设备102可以用于接收原始图像，并对原始图像进行处理。
在一些实施例中，终端设备102可以向服务器101发送原始图像，并接收服务器101发送的原始图像的处理结果。
终端设备102可以是掌上电脑、笔记本电脑、智能手机、平板电脑或台式电脑等计算设备中的任意一种。
服务器101可以用于接收终端设备102发送的原始图像，并对接收的原始图像进行处理后，将原始图像的处理结果发送给终端设备102。
服务器101可以是一台服务器，也可以是由多台服务器组成的服务器集群，或者是一个云计算服务中心。
图2是根据一示例性实施例示出的一种图像处理方法的流程示意图。应用于终端设备102，图2所示的方法可以包括以下步骤：
S100：获取原始图像，原始图像包括待处理对象。
本申请实施例对原始图像的获取方式不进行限定。在一些实施例中，终端设备102通过摄像头实时获取原始图像。在一些实施例中，终端设备102根据接收到的用户选择指令，从存储的图像中选择该用户选择指令所指示的原始图像。其中，用户选择指令可以是终端设备102接收用户通过输入单元发出的用户选择指令。原始图像包括的待处理对象可以是人物、动物、植物或其他物品等中的至少一种。当用户在使用终端设备中的应用程序时，终端设备接收用户选择的应用程序中“将原始图像中的待处理对象处理为参考对象相同类型的对象”的图像处理指令。其中，原始图像为该终端设备的摄像头实时获取的图像。待处理对象为原始图像中的处于满十八周岁年龄阶段的人，参考对象相同类型的对象为处于刚度过婴儿期阶段的人。该图像处理指令用于将原始图像中的处于满十八周岁年龄阶段的人处理为该对象对应的处于刚度过婴儿期阶段的人。
S101：对待处理对象进行图像变形，将图像变形后的待处理对象的形状变换为参考对象的形状，得到中间图像。
在一些实施例中，终端设备102可以通过如下步骤将原始图像中的待处理对象进行图像变形，将图像变形后的待处理对象的形状变换为参考对象的形状，得到中间图像。
步骤一：终端设备102获取用于表征待处理对象的多个第一关键点。多个第一关键点中的一个第一关键点用于表征待处理对象的一个部分。其中，关键点，是图像中一种稳定的、旋转不变、能克服灰度反转的像素点。关键点在含有相同场景或相同目标的相似图像中以一种相同的或至少非常相似的不变形式表示图像或目标的像素点。例如，人脸中的眼球的中间点，从不同的角度采集多幅图像，不同图像中的人脸的眼球的中间点都能够被识别。
终端设备102可以采用关键点提取算法获取待处理对象中的第一关键点。其中，关键点是一种稳定的、旋转不变、能克服灰度反转的像素点。关键点提取算法可以是Moravec算子、Forstner算子或harris算子等。
在一些实施例中，当待处理对象为人脸时，如图3是根据一示例性实施例示出的人脸关键点的示意图。
步骤二：终端设备102获取参考对象的多个第二关键点。多个第二关键点用于表征目参考对象的形状。
基于S100中的示例，参考对象可以是根据多个包含处于刚度过婴儿期阶段的人的脸 部的图像得到的处于刚度过婴儿期阶段的人的平均脸。其中，平均脸指经过计算机技术处理得到的某一群体的合成性面部。
步骤三：终端设备102根据多个第一关键点和多个第二关键点，对待处理对象进行图像变形，将图像变形后的待处理对象的形状变换为参考对象的形状，得到中间图像。
在一些实施例中，终端设备102通过液化模型或液化算法对原始图像中的待处理对象进行图像变形，将图像变形后的待处理对象的形状变换为参考对象的形状，得到中间图像。其中，液化模型或液化算法用于：首先，根据原始图像中待处理对象的多个第一关键点将原始图像中的待处理对象进行三角面片切分，得到多个第一三角面片。根据参考对象的多个第二关键点，将参考对象进行三角面片切分，得到多个第二三角面片。然后，获取具有位置对应关系的待处理对象的三角面片和参考对象的三角面片。再然后，根据具有位置对应关系的待处理对象的三角面片和参考对象的三角面片，对待处理对象进行图像变形，得到图像变形后的待处理对象。
在一些实施例中，终端设备102根据多个第一关键点和多个第二关键点，匹配多个第一关键点和该多个第二关键点，得到多个具有位置对应关系的第一关键点和第二关键点。终端设备102可以采用关键点匹配算法建立第一关键点与第二关键点的位置对应关系，其中，关键点匹配算法可以是相关系数法、松弛法或最小二乘法等。然后，终端设备102基于具有位置对应关系的第一关键点和第二关键点获取待处理对象与参考对象的像素点变换矩阵。终端设备102基于获取的像素点变换矩阵对待处理对象进行图像变形得到图像变形后的待处理对象。
参考对象的第二关键点表征了参考对象的形状，而待处理对象的第一关键点表征了待处理对象的形状。终端设备102基于参考对象的第二关键点与待处理对象的第一关键点，将待处理对象的第一关键点，根据与其匹配的参考对象的第二关键点进行处理。从而使得图像变形后的待处理对象更接近于参考对象的形状。
在一些实施例中，终端设备102还可以使用相似变换、仿射变换或投影变换等中的任意一种算法将原始图像进行形状映射处理，得到中间图像。
S102：将中间图像中图像变形后的待处理对象进行纹理处理，将纹理处理后的待处理对象的纹理处理为参考对象的纹理，得到目标图像。
终端设备102将中间图像输入生成式对抗网络GAN，生成式对抗网络GAN根据参考对象的纹理特征，将中间图像中图像变形处理后的待处理对象的纹理，处理为参考对象的纹理，得到目标图像并输出。其中，生成式对抗网络GAN中的生成网络根据中间图像生成输出图像并将输出图像输入生成式对抗网络GAN中的判别网络；判别网络是基于参考对象构建的；判别网络判断输出图像的真假，并将判断结果反馈给生成网络；生成网络根据判断结果调整生成输出图像的参数，并生成新的输出图像，直至判别网络确定生成网络生成的新的输出图像为真，生成式对抗网络GAN将确定为真的新的输出图像作为目标图像输出。
终端设备102将目标图像通过输出设备展现给用户。基于S101的示例，终端设备102将目标图像通过终端设备102的显示器(例如：屏幕)展现给用户。
图4是根据一示例性实施例示出的原始图像与目标图像的示意图。图4中第一列图像为原始图像，第二列图像为将原始图像处理后的目标图像。
本申请实施例通过将图像处理中的形变与纹理映射分离的图像处理方案，首先通过对 原始图像进行图像变形，得到中间图像，使得中间图像中待处理对象的形状与参考对象的形状接近，然后通过生成式对抗网络GAN将中间图像中图像变形后的待处理对象的纹理处理为参考对象的纹理，得到目标图像。
由于图像变形与纹理处理分离，使得终端设备102在进行图像变形的时候只专注于原始图像中的待处理对象的形状与参考对象的形状映射，不用考虑其他因素，在对中间图像进行纹理处理的时候，只专注于待处理对象的纹理与参考对象的纹理映射。
图5是根据另一示例性实施例示出的一种图像处理方法的流程示意图。图5所示的方法可以包括以下步骤：
S200：终端设备102获取原始图像，原始图像包括待处理对象。所述获取原始图像的方法可参考S100的描述。
S201：终端设备102向服务器101发送原始图像以及图像处理指令。
本申请实施例中终端设备102可以分两次分别向服务器101发送原始图像以及图像处理指令，或者，终端设备102一次向服务器101发送原始图像以及图像处理指令。
S202：服务器101根据原始图像以及图像处理指令，对原始图像中的待处理对象进行图像变形，将图像变形后的待处理对象的形状变换为参考对象的形状，得到中间图像。所述得到中间图像的方法可参考S102的描述。
S203：服务器101对中间图像中图像变形后的待处理对象进行纹理处理，将纹理处理后的待处理对象的纹理处理为参考对象的纹理，得到目标图像。
参考上述S102中终端设备102将中间图像中图像变形后的待处理对象进行纹理处理，将纹理处理后的待处理对象的纹理处理为参考对象的纹理的方式。
S204：服务器101向终端设备102发送目标图像。
终端设备102通过输出设备将目标图像展现给用户。
本申请实施例通过将图像处理中的形变与纹理映射分离的图像处理方案，首先通过图像变形，得到中间图像，使得中间图像中待处理对象的形状与参考对象的形状接近，然后通过纹理映射方法将中间图像中的待处理对象的纹理处理为参考对象的纹理，得到目标图像。由于图像变形与纹理映射分离，使得服务器101在进行图像变形的时候只专注于原始图像中的待处理对象的形状与参考对象的形状映射，不用考虑其他因素，在对中间图像进行纹理处理的时候，只专注于中间图像中待处理对象的纹理与参考对象的纹理映射。
上述主要从方法的角度对本申请实施例提供的方案进行了介绍。为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的方法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对图像处理装置进行功能模块的划分，例如可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
图6是根据一示例性实施例示出的一种图像处理装置框图。参照图6，该图像处理装 置90包括获取模块901，被配置为获取原始图像；原始图像包括待处理对象；图像变形处理模块902，被配置为对待处理对象进行图像变形，将图像变形后的待处理对象的形状变换为参考对象的形状，得到中间图像；图像纹理处理模块903，被配置为对中间图像中图像变形处理后的待处理对象进行纹理处理，将纹理处理后的待处理对象的纹理处理为参考对象的纹理，得到目标图像。例如：当图像处理装置90为终端设备时，结合图2，获取模块901被配置为用于执行S100,图像变形处理模块902被配置为用于执行S101,图像纹理处理模块903被配置为用于执行S102。当图像处理装置90为服务器时，结合图5，获取模块901被配置为用于执行S201中的接收步骤，图像变形处理模块902被配置为用于S202，图像纹理处理模块903被配置为用于执行S203。
在一些实施例中，获取模块901还被配置为：获取待处理对象的多个第一关键点；获取参考对象的多个第二关键点；图像变形处理模块902，具体被配置为：根据多个第一关键点和多个第二关键点，对待处理对象进行图像变形。
在一些实施例中，获取模块901还被配置为：根据多个第一关键点和多个第二关键点，得到具有位置对应关系的第一关键点和第二关键点；基于具有位置对应关系的第一关键点和第二关键点获取待处理对象与参考对象的像素点变换矩阵；图像变形处理模块902，具体被配置为基于像素点变换矩阵对待处理对象进行图像变形，将图像变形后的待处理对象的形状变换为参考对象的形状，得到中间图像。
在一些实施例中，图像变形处理模块902可以被配置为：根据多个第一关键点将待处理对象进行三角面片切分；根据多个第二关键点将参考对象进行三角面片切分；获取具有位置对应关系的待处理对象的三角面片和参考对象的三角面片；根据具有位置对应关系的待处理对象的三角面片和参考对象的三角面片，对待处理对象进行图像变形，将图像变形后的待处理对象的形状变换为参考对象的形状，得到中间图像。
在一些实施例中，图像纹理处理模块903可以被配置为：将中间图像输入生成式对抗网络GAN；生成式对抗网络GAN根据参考对象的纹理特征，将中间图像中图像变形处理后的待处理对象的纹理，处理为参考对象的纹理，得到目标图像并输出。
在一些实施例中，图像纹理处理模块903可以被配置为：生成式对抗网络GAN中的生成网络根据中间图像生成输出图像并将输出图像输入生成式对抗网络GAN中的判别网络；判别网络是基于参考对象构建的；判别网络判断输出图像的真假，并将判断结果反馈给生成网络；生成网络根据判断结果调整生成输出图像的参数，并生成新的输出图像，直至判别网络确定生成网络生成的新的输出图像为真，生成式对抗网络GAN将确定为真的新的输出图像作为目标图像输出。
本申请实施例提供的图像处理装置，采用将图像处理中的形变与纹理映射分离的图像处理方案，首先通过形状映射处理，得到中间图像，使得中间图像中待处理对象的形状与目标类型的对象的形状接近，然后通过纹理映射方法进行原始图像向中间图像的纹理映射，得到目标图像。另外，由于形变与纹理映射分离，使得图像处理装置在进行图像变形的时候只专注于原始图像中的待处理对象的形状与目标类型的对象的形状映射，不用考虑其他因素，在对原始图像和中间图像进行纹理映射的时候，只专注于原始图像与中间图像的纹理映射。
关于上述实施例中的图像处理装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。
上述各个模块对应执行的动作仅是具体举例，各个单元实际执行的动作参照上述基于图2、图5所述的实施例的描述中提及的动作或步骤。
图7是根据一示例性实施例示出的电子设备1000的框图，参照图7，该电子设备100包括：处理器1010和用于存储所述处理器可执行指令的存储器1020；
其中，所述处理器1010被配置为执行：
获取原始图像；所述原始图像包括待处理对象；
对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像；
对所述中间图像中图像变形后的所述待处理对象进行纹理处理，将纹理处理后的所述待处理对象的纹理处理为所述参考对象的纹理，得到目标图像。
其中，所述处理器1010可以被配置为执行：
获取所述待处理对象的多个第一关键点；获取所述参考对象的多个第二关键点；根据所述多个第一关键点和所述多个第二关键点，对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像。
其中，所述处理器1010可以被配置为执行：
根据所述多个第一关键点和所述多个第二关键点，得到具有位置对应关系的第一关键点和第二关键点；
基于具有位置对应关系的第一关键点和第二关键点获取所述待处理对象与所述参考对象的像素点变换矩阵；
基于所述像素点变换矩阵对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像。
其中，所述处理器1010可以被配置为执行：
根据所述多个第一关键点将所述待处理对象进行三角面片切分；
根据所述多个第二关键点将所述参考对象进行三角面片切分；
获取具有位置对应关系的所述待处理对象的三角面片和所述参考对象的三角面片；
根据具有位置对应关系的所述待处理对象的三角面片和所述参考对象的三角面片，对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像。
其中，所述处理器1010可以被配置为执行：
将所述中间图像输入生成式对抗网络GAN；
所述生成式对抗网络GAN根据所述参考对象的纹理特征，将所述中间图像中图像变形处理后的所述待处理对象的纹理，处理为所述参考对象的纹理，得到目标图像并输出。
其中，所述处理器1010可以被配置为执行：
所述生成式对抗网络GAN中的生成网络根据中间图像生成输出图像并将所述输出图像输入所述生成式对抗网络GAN中的判别网络；所述判别网络是基于所述参考对象构建的；
所述判别网络判断所述输出图像的真假，并将判断结果反馈给所述生成网络；
所述生成网络根据所述判断结果调整生成输出图像的参数，并生成新的输出图像，直至所述判别网络确定所述生成网络生成的新的输出图像为真，所述生成式对抗网络GAN将确定为真的新的输出图像作为目标图像输出。
本申请实施例还提供了一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，当该计算机程序在计算机上运行时，使得该计算机执行上文提供的任一实施例中提及的动作或步骤。
本申请实施例还提供了一种芯片。该芯片中集成了用于实现上述处图像处理装置的功能的电路和一个或者多个接口。可选的，该芯片支持的功能可以包括基于图2或图5所述的实施例中的处理动作，此处不再赘述。本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可通过程序来指令相关的硬件完成。所述的程序可以存储于一种计算机可读存储介质中。上述提到的存储介质可以是只读存储器，随机接入存储器等。上述处理单元或处理器可以是中央处理器，通用处理器、特定集成电路(application specific integrated circuit，ASIC)、微处理器(digital signal processor，DSP)，现场可编程门阵列(field programmable gate array，FPGA)或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。
本申请实施例还提供了一种包含指令的计算机程序产品，当该指令在计算机上运行时，使得计算机执行上述实施例中的任意一种方法。该计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，计算机指令可以从一个网站站点、计算机、服务器或者数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可以用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质(例如，软盘、硬盘、磁带)，光介质(例如，DVD)、或者半导体介质(例如固态硬盘(solid state disk，SSD))等。
应注意，本申请实施例提供的上述用于存储计算机指令或者计算机程序的器件，例如但不限于，上述存储器、计算机可读存储介质和通信芯片等，均具有非易失性(non-transitory)。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本申请的其它实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本申请未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本申请的真正范围和精神由下面的权利要求指出。
应当理解的是，本申请并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本申请的范围仅由所附的权利要求来限制。

Claims (19)

  1. 一种图像处理方法，所述方法包括：
    获取原始图像；所述原始图像包括待处理对象；
    对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像；
    对所述中间图像中图像变形后的所述待处理对象进行纹理处理，将纹理处理后的所述待处理对象的纹理处理为所述参考对象的纹理，得到目标图像。
  2. 根据权利要求1所述的方法，所述对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像，包括：
    获取所述待处理对象的多个第一关键点；获取所述参考对象的多个第二关键点；根据所述多个第一关键点和所述多个第二关键点，对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状处理为参考对象的形状，得到中间图像。
  3. 根据权利要求2所述的方法，所述根据所述多个第一关键点和所述多个第二关键点，对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像，包括：
    根据所述多个第一关键点和所述多个第二关键点，得到具有位置对应关系的第一关键点和第二关键点；
    基于具有位置对应关系的第一关键点和第二关键点获取所述待处理对象与所述参考对象的像素点变换矩阵；
    基于所述像素点变换矩阵对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像。
  4. 根据权利要求2所述的方法，所述根据所述多个第一关键点和所述多个第二关键点，对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像，包括：
    根据所述多个第一关键点将所述待处理对象进行三角面片切分；
    根据所述多个第二关键点将所述参考对象进行三角面片切分；
    获取具有位置对应关系的所述待处理对象的三角面片和所述参考对象的三角面片；
    根据具有位置对应关系的所述待处理对象的三角面片和所述参考对象的三角面片，对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像。
  5. 根据权利要求1-4任一项所述的方法，所述对所述中间图像中图像变形后的所述待处理对象进行纹理处理，将纹理处理后的所述待处理对象的纹理处理为所述参考对象的纹理，得到目标图像，包括：
    将所述中间图像输入生成式对抗网络GAN；
    所述生成式对抗网络GAN根据所述参考对象的纹理特征，将所述中间图像中图像变形后的所述待处理对象的纹理，处理为所述参考对象的纹理，得到目标图像并输出。
  6. 根据权利要求5所述的方法，所述生成式对抗网络GAN根据所述参考对象的纹理特征，将所述中间图像中图像变形后的所述待处理对象的纹理，处理为所述参考对象的纹理，得到目标图像并输出，包括：
    所述生成式对抗网络GAN中的生成网络根据中间图像生成输出图像并将所述输出图像输入所述生成式对抗网络GAN中的判别网络；所述判别网络是基于所述参考对象构建的；
    所述判别网络判断所述输出图像的真假，并将判断结果反馈给所述生成网络；
    所述生成网络根据所述判断结果调整生成输出图像的参数，并生成新的输出图像，直至所述判别网络确定所述生成网络生成的新的输出图像为真，所述生成式对抗网络GAN将确定为真的新的输出图像作为目标图像输出。
  7. 一种图像处理装置，所述图像处理装置包括：
    获取模块，被配置为获取原始图像；所述原始图像包括待处理对象；
    图像变形处理模块，被配置为对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像；
    图像纹理处理模块，被配置为对所述中间图像中图像变形后的所述待处理对象进行纹理处理，将纹理处理后的所述待处理对象的纹理处理为所述参考对象的纹理，得到目标图像。
  8. 根据权利要求7所述的图像处理装置，所述获取模块还被配置为：
    获取所述待处理对象的多个第一关键点；获取所述参考对象的多个第二关键点；
    所述图像变形处理模块，具体被配置为：
    根据所述多个第一关键点和所述多个第二关键点，对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像。
  9. 根据权利要求8所述的图像处理装置，所述获取模块还被配置为：
    根据所述多个第一关键点和所述多个第二关键点，得到具有位置对应关系的第一关键点和第二关键点；基于具有位置对应关系的第一关键点和第二关键点获取所述待处理对象与所述参考对象的像素点变换矩阵；
    所述图像变形处理模块，具体被配置为基于所述像素点变换矩阵对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像。
  10. 根据权利要求9所述的图像处理装置，所述图像变形处理模块具体被配置为：
    根据所述多个第一关键点将所述待处理对象进行三角面片切分；
    根据所述多个第二关键点将所述参考对象进行三角面片切分；
    获取具有位置对应关系的所述待处理对象的三角面片和所述参考对象的三角面片；
    根据具有位置对应关系的所述待处理对象的三角面片和所述参考对象的三角面片，对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像。
  11. 根据权利要求7-10任一项所述的图像处理装置，所述图像纹理处理模块具体被配置为：
    将所述中间图像输入生成式对抗网络GAN；
    所述生成式对抗网络GAN根据所述参考对象的纹理特征，将所述中间图像中图像变形后的所述待处理对象的纹理，处理为所述参考对象的纹理，得到目标图像并输出。
  12. 根据权利要求11所述的图像处理装置，所述图像纹理处理模块具体被配置为：
    所述生成式对抗网络GAN中的生成网络根据中间图像生成输出图像并将所述输出图 像输入所述生成式对抗网络GAN中的判别网络；所述判别网络是基于所述参考对象构建的；
    所述判别网络判断所述输出图像的真假，并将判断结果反馈给所述生成网络；
    所述生成网络根据所述判断结果调整生成输出图像的参数，并生成新的输出图像，直至所述判别网络确定所述生成网络生成的新的输出图像为真，所述生成式对抗网络GAN将确定为真的新的输出图像作为目标图像输出。
  13. 一种电子设备，包括：
    处理器和用于存储所述处理器可执行指令的存储器；
    其中，所述处理器被配置为执行：
    获取原始图像；所述原始图像包括待处理对象；
    对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像；
    对所述中间图像中图像变形后的所述待处理对象进行纹理处理，将纹理处理后的所述待处理对象的纹理处理为所述参考对象的纹理，得到目标图像。
  14. 根据权利要求13所述的电子设备，所述处理器具体被配置为执行：
    获取所述待处理对象的多个第一关键点；获取所述参考对象的多个第二关键点；根据所述多个第一关键点和所述多个第二关键点，对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像。
  15. 根据权利要求14所述的电子设备，所述处理器具体被配置为执行：
    根据所述多个第一关键点和所述多个第二关键点，得到具有位置对应关系的第一关键点和第二关键点；
    基于具有位置对应关系的第一关键点和第二关键点获取所述待处理对象与所述参考对象的像素点变换矩阵；
    基于所述像素点变换矩阵对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像。
  16. 根据权利要求14所述的电子设备，所述处理器具体被配置为执行：
    根据所述多个第一关键点将所述待处理对象进行三角面片切分；
    根据所述多个第二关键点将所述参考对象进行三角面片切分；
    获取具有位置对应关系的所述待处理对象的三角面片和所述参考对象的三角面片；
    根据具有位置对应关系的所述待处理对象的三角面片和所述参考对象的三角面片，对所述待处理对象进行图像变形，将图像变形后的所述待处理对象的形状变换为参考对象的形状，得到中间图像。
  17. 根据权利要求13-16任一项所述的电子设备，所述处理器具体被配置为执行：
    将所述中间图像输入生成式对抗网络GAN；
    所述生成式对抗网络GAN根据所述参考对象的纹理特征，将所述中间图像中图像变形后的所述待处理对象的纹理，处理为所述参考对象的纹理，得到目标图像并输出。
  18. 根据权利要求17所述的电子设备，所述处理器具体被配置为执行：
    所述生成式对抗网络GAN中的生成网络根据中间图像生成输出图像并将所述输出图像输入所述生成式对抗网络GAN中的判别网络；所述判别网络是基于所述参考对象构建的；
    所述判别网络判断所述输出图像的真假，并将判断结果反馈给所述生成网络；
    所述生成网络根据所述判断结果调整生成输出图像的参数，并生成新的输出图像，直至所述判别网络确定所述生成网络生成的新的输出图像为真，所述生成式对抗网络GAN将确定为真的新的输出图像作为目标图像输出。
  19. 一种计算机可读存储介质，当所述计算机可读存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行如权利要求1至6中任一项所述的方法。
PCT/CN2020/128812 2020-05-21 2020-11-13 一种图像处理方法及电子设备 WO2021232708A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010437578.9A CN113706369A (zh) 2020-05-21 2020-05-21 图像处理方法、装置、电子设备及存储介质
CN202010437578.9 2020-05-21

Publications (1)

Publication Number Publication Date
WO2021232708A1 true WO2021232708A1 (zh) 2021-11-25

Family

ID=78645885

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/128812 WO2021232708A1 (zh) 2020-05-21 2020-11-13 一种图像处理方法及电子设备

Country Status (2)

Country Link
CN (1) CN113706369A (zh)
WO (1) WO2021232708A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1971615A (zh) * 2006-11-10 2007-05-30 中国科学院计算技术研究所 一种基于人脸照片的卡通肖像生成方法
CN105184249A (zh) * 2015-08-28 2015-12-23 百度在线网络技术（北京）有限公司 用于人脸图像处理的方法和装置
US20160293119A1 (en) * 2015-04-01 2016-10-06 Samsung Display Co, Ltd. Display apparatus
CN110322398A (zh) * 2019-07-09 2019-10-11 厦门美图之家科技有限公司 图像处理方法、装置、电子设备及计算机可读存储介质

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107154030B (zh) * 2017-05-17 2023-06-09 腾讯科技（上海）有限公司 图像处理方法及装置、电子设备及存储介质
CN107527034B (zh) * 2017-08-28 2019-07-26 维沃移动通信有限公司 一种面部轮廓调整方法及移动终端
CN109087240B (zh) * 2018-08-21 2023-06-06 成都旷视金智科技有限公司 图像处理方法、图像处理装置及存储介质
CN109712080A (zh) * 2018-10-12 2019-05-03 迈格威科技有限公司 图像处理方法、图像处理装置及存储介质
CN109584168B (zh) * 2018-10-25 2021-05-04 北京市商汤科技开发有限公司 图像处理方法和装置、电子设备和计算机存储介质
CN109672830B (zh) * 2018-12-24 2020-09-04 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN109859097B (zh) * 2019-01-08 2023-10-27 北京奇艺世纪科技有限公司 脸部图像处理方法、设备、图像处理设备、介质
CN110298785A (zh) * 2019-06-29 2019-10-01 北京字节跳动网络技术有限公司 图像美化方法、装置及电子设备
CN110728620A (zh) * 2019-09-30 2020-01-24 北京市商汤科技开发有限公司 一种图像处理方法、装置和电子设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1971615A (zh) * 2006-11-10 2007-05-30 中国科学院计算技术研究所 一种基于人脸照片的卡通肖像生成方法
US20160293119A1 (en) * 2015-04-01 2016-10-06 Samsung Display Co, Ltd. Display apparatus
CN105184249A (zh) * 2015-08-28 2015-12-23 百度在线网络技术（北京）有限公司 用于人脸图像处理的方法和装置
CN110322398A (zh) * 2019-07-09 2019-10-11 厦门美图之家科技有限公司 图像处理方法、装置、电子设备及计算机可读存储介质

Also Published As

Publication number Publication date
CN113706369A (zh) 2021-11-26

Similar Documents

Publication Publication Date Title
US11978245B2 (en) Method and apparatus for generating image
US10650492B2 (en) Method and apparatus for generating image
WO2020006961A1 (zh) 用于提取图像的方法和装置
WO2021012526A1 (zh) 人脸识别模型的训练方法、人脸识别方法、装置、设备及存储介质
WO2020024483A1 (zh) 用于处理图像的方法和装置
CN106682632B (zh) 用于处理人脸图像的方法和装置
WO2020024484A1 (zh) 用于输出数据的方法和装置
WO2019242222A1 (zh) 用于生成信息的方法和装置
US20180204562A1 (en) Method and device for image recognition
CN111275784B (zh) 生成图像的方法和装置
US20220222925A1 (en) Artificial intelligence-based image processing method and apparatus, device, and storage medium
WO2020253127A1 (zh) 脸部特征提取模型训练方法、脸部特征提取方法、装置、设备及存储介质
CN111985281B (zh) 图像生成模型的生成方法、装置及图像生成方法、装置
CN109189544B (zh) 用于生成表盘的方法和装置
WO2019200702A1 (zh) 去网纹系统训练方法、去网纹方法、装置、设备及介质
CN111814620A (zh) 人脸图像质量评价模型建立方法、优选方法、介质及装置
CN108388889B (zh) 用于分析人脸图像的方法和装置
WO2020125229A1 (zh) 特征融合方法、装置、电子设备及存储介质
US20220358675A1 (en) Method for training model, method for processing video, device and storage medium
CN113569740B (zh) 视频识别模型训练方法与装置、视频识别方法与装置
CN108399401B (zh) 用于检测人脸图像的方法和装置
JP2020013553A (ja) 端末装置に適用される情報生成方法および装置
WO2023124040A1 (zh) 一种人脸识别方法及装置
CN111539897A (zh) 用于生成图像转换模型的方法和装置
WO2022073516A1 (zh) 生成图像的方法、装置、电子设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20936689

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 20.03.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20936689

Country of ref document: EP

Kind code of ref document: A1