WO2021218040A1 - 图像处理的方法及装置 - Google Patents

图像处理的方法及装置 Download PDF

Info

Publication number
WO2021218040A1
WO2021218040A1 PCT/CN2020/120102 CN2020120102W WO2021218040A1 WO 2021218040 A1 WO2021218040 A1 WO 2021218040A1 CN 2020120102 W CN2020120102 W CN 2020120102W WO 2021218040 A1 WO2021218040 A1 WO 2021218040A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
target object
shape
face
target
Prior art date
Application number
PCT/CN2020/120102
Other languages
English (en)
French (fr)
Inventor
汪训昌
黄伟钗
黎欢
庞文杰
马玟博
Original Assignee
百度在线网络技术（北京）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 百度在线网络技术（北京）有限公司 filed Critical 百度在线网络技术（北京）有限公司
Priority to JP2021526360A priority Critical patent/JP7183414B2/ja
Priority to EP20886186.4A priority patent/EP3933751A4/en
Priority to KR1020217015566A priority patent/KR20210070377A/ko
Priority to US17/330,507 priority patent/US20210279928A1/en
Publication of WO2021218040A1 publication Critical patent/WO2021218040A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/60—Editing figures and text; Combining figures or text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • This application relates to the field of image processing and computer technology, in particular to the field of artificial intelligence and computer vision technology, and in particular to an image processing method and device.
  • terminal devices With the development of terminal devices, more terminal devices include camera devices, and more suitable beauty applications have also been developed.
  • some beauty materials are usually provided, and the user can select the beauty material according to the needs, and the application can add the beauty material to the user's face.
  • the user can select the eyebrow shape provided in the application material, and then the application can cover the eyebrow shape in the application material on the eyebrows of the user's face to achieve eyebrow beautification.
  • the embodiments of the present application provide an image processing method and device to solve the technical problem of poor beautification effect in beautification processing in the prior art.
  • the first aspect of the embodiments of the present application provides an image processing method, including:
  • a second aspect of the embodiments of the present application provides an image processing apparatus, including:
  • a receiving module configured to receive a selection operation of a beauty material of a target object in the human face, the target object being an organ in the human face;
  • the processing module is further configured to use a pre-trained target key point model to identify the key point position of the target object in the face;
  • the processing module is further configured to adjust the shape of the target object in the face by using the position of the key point and the shape of the beauty material;
  • the processing module is also used to segment the adjusted target object in the image area to obtain the segmented area;
  • the processing module is further configured to fill the divided area according to the beauty material to obtain a target image.
  • a third aspect of the embodiments of the present application provides an electronic device, including: at least one processor; Instructions, the instructions are executed by the at least one processor, so that the at least one processor can execute the method according to any one of the foregoing first aspects.
  • the fourth aspect of the embodiments of the present application provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the method according to any one of the foregoing first aspects.
  • the fifth aspect of the embodiments of the present application provides a computer program, including program code.
  • the program code executes the method described in any of the above embodiments.
  • the embodiment of the present application provides an image processing method and device. After identifying the key point position of the target object in the face, first adjust the shape of the target object in the face according to the shape of the beauty material, and then The adjusted area is divided and filled with materials. Compared with the solution that may not be completely occluded by directly covering the target object with the beauty material, the solution of adjusting the shape and then dividing and filling in the embodiment of the present application can avoid the inability to Occurrence of the occlusion phenomenon, get a more natural beauty effect.
  • FIG. 1 is a schematic diagram of a system architecture to which an image processing method provided in an embodiment of the application is applicable;
  • FIG. 2 is a schematic flowchart of an image processing method provided by an embodiment of the application.
  • Fig. 3 is a schematic diagram of an example of eyebrow treatment provided by an embodiment of the application.
  • FIG. 4 is a schematic structural diagram of an image processing device provided by an embodiment of the application.
  • Fig. 5 is a block diagram of an electronic device used to implement the image processing method of an embodiment of the present application.
  • the user when performing beautification processing on a human face, the user can select the beautification material, which can be the material of the various organs of the face, and the terminal device can cover the beautification material on the user's original organs in the image.
  • the beautification material which can be the material of the various organs of the face
  • the terminal device can cover the beautification material on the user's original organs in the image.
  • various types of eyebrow shapes can be provided in the beautifying material, and the user can select one of the eyebrow shapes, and the terminal device can use the eyebrow shape selected by the user to cover the eyebrows in the image to achieve eyebrow beautification.
  • the beauty method that uses beauty materials to cover real organs on the one hand, the integration of the beauty materials and the image is poor, and the image after the beauty processing is not natural enough, which affects the beauty effect.
  • the beauty material cannot be well concealed due to the large or irregular organs in the original image.
  • some eyebrow materials cannot well cover thick eyebrows, invisible, irregular, and other eyebrow shapes.
  • the eyebrow material will be stretched and deformed, and there will be a certain stretch and deformation effect in the image after beautification, resulting in poor beautification effect.
  • the shape of the target object in the face is adjusted according to the shape of the beauty material, and then the adjusted area is sectioned.
  • the solution of the embodiment of the present application that adjusts the shape first and then divides and fills can avoid the occurrence of the incapability phenomenon, and obtain a better result. Natural beauty effect.
  • the beauty materials described in the embodiments of the present application may be pre-drawn or acquired materials.
  • the beauty materials may include eyebrow shapes such as daiyu eyebrows, feather eyebrows, and willow leaf eyebrows.
  • the lip material may include lip shapes such as thin lips and thick lips.
  • the embodiment of this application does not specifically limit the beauty material.
  • the beauty material can be stored in the network or the server, and the terminal device can obtain the beauty material from the network or the server when performing the beautification.
  • the beauty material may also be stored locally in the terminal device, and the embodiment of the present application does not specifically limit the acquisition of the beauty material.
  • the target object described in the embodiment of the present application may be an organ of a human face, such as eyebrows, nose, mouth, or eyes, etc., which is not specifically limited in the embodiment of the present application.
  • the target key point model described in the embodiment of the application may be a neural network model obtained by deep learning training.
  • a sample image including the key points of human organs may be used to train the target key point model, so that the target key point model is After inputting the image, the target key point model can output the key point position of the target object.
  • the method in the embodiments of the present application may be applied to a terminal, and the terminal may include electronic devices such as a mobile phone, a tablet computer, a notebook computer, or a desktop computer.
  • the embodiments of the present application do not specifically limit the specific equipment to be applied.
  • the terminal may provide a graphical user interface (GUI), and controls and switches for receiving user operations may be set in the graphical user interface, so that the user can perform operations in the graphical user interface as in the embodiments of the present application.
  • GUI graphical user interface
  • controls and switches for receiving user operations may be set in the graphical user interface, so that the user can perform operations in the graphical user interface as in the embodiments of the present application. It can be understood that the specific content of the graphical user interface can be determined according to the actual application scenario, which is not specifically limited in the embodiment of the present invention.
  • FIG. 1 is a schematic diagram of an application scenario architecture to which the method provided in an embodiment of the application is applicable.
  • the beauty material and the target key point recognition model are both local to the terminal device, and the terminal device 11 does not need to interact with other devices.
  • the user may trigger the acquisition of an image including a human face on the terminal device 11, and select the beauty material of the target object in the graphical user interface of the terminal device 11.
  • the terminal device 11 may use a pre-trained target key point model to identify the target The position of the key point of the object in the face; and use the position of the key point and the shape of the beauty material to adjust the shape of the target object in the face; divide the adjusted target object in the image area to obtain the divided image Area; fill the divided area according to the beauty material to obtain the target image. That is, the terminal device 11 adopts a solution of first adjusting the shape and then dividing and filling, so as to avoid the occurrence of the incapability phenomenon, and obtain a more natural beauty effect.
  • one or more of the beauty material or the target key point recognition model is set in the server 12, and the terminal device 11 needs to interact with the server 12.
  • the user can trigger the acquisition of an image including a human face on the terminal device 11, and select the beauty material of the target object in the graphical user interface of the terminal device 11.
  • the terminal device 11 can acquire the beauty material from the server 12, or also The image can be sent to the server 12, and the server 12 uses the pre-trained target key point model to identify the key point position of the target object in the face.
  • the terminal device 11 can further obtain the key point position from the server 12, and use the key point position and beauty.
  • the shape of the color material adjusts the shape of the target object in the face; divides the adjusted target object in the image area to obtain the divided area; fills the divided area according to the beauty material to obtain the target image . That is, the solution of first adjusting the shape and then splitting and filling can avoid the occurrence of the inability to block phenomenon and obtain a more natural beauty effect. It can be understood that in a specific application, the number of servers may be any value greater than or equal to 1, which is not specifically limited in the embodiment of the present application.
  • FIG. 2 is a schematic flowchart of an image processing method provided by an embodiment of the application.
  • the method may specifically include:
  • S101 Acquire an image, where the image includes a human face.
  • the image may be selected and uploaded by the user in the local picture library of the terminal device.
  • the image can also be taken by a user using a camera or other equipment of the terminal device.
  • the image can also be downloaded by the user from the network.
  • the embodiment of the present application does not limit the specific method of acquiring the image.
  • the user can open the beauty application (application, APP) in the terminal device, and take images in the APP in real time, or upload the local image of the terminal device in the APP to obtain an image including a face .
  • application application, APP
  • S102 Receive a selection operation of a beauty material of a target object in the human face, where the target object is an organ in the human face.
  • the user can further select the beautification material of the specific target object for beautification.
  • the target object may be an organ in a human face, such as eyebrows, nose, mouth, or Eyes, etc., to achieve the beauty of the target object through the beauty material of the target object.
  • the user interface of the terminal device can display images and facial beauty materials corresponding to the facial organs.
  • the user can click, drag, and voice control to select the beauty materials of the target object. choose.
  • S103 Use a pre-trained target key point model to identify the key point position of the target object in the face.
  • the key point position of the target object in the face can be used to identify the area of the target object in the face.
  • the number and position of the key points are different.
  • the position of the key points of the subject in the face is not specifically limited.
  • the target key point model can be pre-trained and set in the beauty APP, and the target key point model can be used to identify the key point position of the target object in the face.
  • S104 Adjust the shape of the target object in the face by using the position of the key point and the shape of the beauty material.
  • the shape of the target object in the face can be adjusted according to the shape of the beauty material, so that the shape of the target object in the face is the same as the beauty of the face.
  • the shape of the material is adapted, so that subsequent adaptive beauty processing can be performed according to the adjusted target object area.
  • the eyebrow shape of the face in the image may be thick and flat eyebrows, and the beauty material selected by the user is Daiyu eyebrows, you can first adjust the shape of the eyebrows in the face to match the beauty
  • the shape of the material is similar or the same shape.
  • the shape of the target object in the face is adjusted according to the shape of the beauty material, so that the shape of the target object in the face matches the shape of the beauty material.
  • the material is beautifying, it will not appear that the beautifying material cannot cover the target object, etc.
  • the division can be any polygonal division, and the adjusted target object is divided into the area in the image, so that the subsequent filling can be based on the divided area, as opposed to directly using the beauty material
  • the method of covering the target object in the image can achieve a better fusion effect with the image, and the beauty effect is more natural.
  • S106 Fill the divided area according to the beauty material to obtain a target image.
  • the beauty material may include the color and shape of the target object, etc., and the segmented area is filled according to the beauty material to obtain a target image that is better integrated with the original image.
  • the scheme of adjusting the shape first and then dividing and filling in the embodiment of the present application can avoid the occurrence of the inability to occlude. Get a more natural beauty effect.
  • the method further includes: in the case of receiving a sharing operation on the target image, sharing the target based on the path of the sharing operation image.
  • one or more sharing buttons may be provided in the user interface of the terminal device, and each sharing button may correspond to a different sharing path.
  • each sharing button may correspond to a social application. At the time, the target audience will be shared to the corresponding social application based on the share button.
  • the user can trigger the selection of the share button, and then jump to the social application corresponding to the share button, so as to conveniently realize the sharing of the target image.
  • the method further includes: storing the target image when a storing operation on the target image is received.
  • one or more storage buttons may be provided in the user interface of the terminal device, and each storage button may correspond to a different storage path.
  • the storage button may be a button stored locally, and the storage button may also be stored in Cloud buttons are not specifically limited in the embodiment of this application.
  • the user can trigger the selection of the save and share button, which can then conveniently realize the storage of the target image.
  • the adjusting the shape of the target object in the face by using the position of the key point and the shape of the beauty material includes : Obtain the source area of the target object in the face by using the key point position; adjust the shape of the source area to a shape similar to the shape of the beauty material.
  • the source area of the target object in the face can be drawn according to the position of the key point, for example, all the key point positions can be connected to obtain the source area of the target object in the face.
  • the source area of the target object in the human face may also be referred to as the original area of the target object in the human face, and the shape and size of the source area are related to the specific human face.
  • the shape of the source area can be adjusted to a shape similar to the shape of the beauty material, for example, the source area can be stretched and deformed to make the source area consistent in shape with the beauty material , And the size is adapted to the size of the face.
  • a specific method for adjusting the shape of the target object in the face is provided, which can adjust the shape of the target object in the face more conveniently and accurately.
  • the segmentation of the adjusted target object in the image includes: dividing the adjusted target object in the image The area in is triangulated.
  • the triangulation method is adopted to achieve a relatively stable segmentation realization, so that a better processing effect can be achieved when the target object in the image is subsequently processed.
  • Triangulation is a relatively classic method of division, which is not described in detail in the embodiment of the present application.
  • the filling the segmented area according to the beauty material to obtain the target image includes: according to the beauty material in the Texture mapping is performed on the divided area to obtain the target image.
  • texture mapping when filling the segmented area, texture mapping can be performed in the segmented area based on the beauty material to achieve better image fusion and obtain a target image with better fusion effect.
  • the acquiring an image includes: receiving a photographing operation; and photographing the image in response to the photographing operation.
  • the user can trigger the shooting button in the beauty APP to capture the image in real time, and realize the real-time processing of the image.
  • FIG. 3 shows a schematic diagram of an example of eyebrow processing in the image processing method of an embodiment of the present application.
  • the beauty material can be the material drawn in the bottom model (such as the face model).
  • the key to the eyebrows in the image can be identified based on the eyebrow key point model Point position, and adjust the shape of the eyebrows in the image based on the key point position of the eyebrows and the beauty material, and divide the adjusted area based on the segmentation network, and realize the texture of the split area based on the beauty material
  • the fused image is obtained by mapping, and better image fusion is realized, so that a target image with better fusion effect can be obtained.
  • FIG. 4 is a schematic structural diagram of an embodiment of an image processing apparatus provided by this application. As shown in Figure 4, the image processing apparatus provided in this embodiment includes:
  • the processing module 41 is configured to obtain an image, and the image includes a human face;
  • the receiving module 42 is configured to receive a selection operation of beauty materials of a target object in the human face, where the target object is an organ in the human face;
  • the processing module 41 is further configured to use a pre-trained target key point model to identify the key point position of the target object in the face;
  • the processing module 41 is further configured to adjust the shape of the target object in the face by using the position of the key point and the shape of the beauty material;
  • the processing module 41 is also used to segment the adjusted target object in the image area to obtain the segmented area;
  • the processing module 41 is further configured to fill the divided area according to the beauty material to obtain a target image.
  • the processing module is specifically used for:
  • the shape of the source area is adjusted to a shape similar to the shape of the beauty material.
  • the processing module is specifically used for:
  • the processing module is specifically used for:
  • the processing module is specifically used for:
  • the image is photographed in response to the photographing operation.
  • the target object is eyebrows
  • the beauty material is eyebrow material
  • the processing module is further used for:
  • the target image is shared based on the path of the sharing operation.
  • the processing module is further used for:
  • the target image is stored.
  • the scheme of adjusting the shape first and then dividing and filling in the embodiment of the present application can avoid the occurrence of the inability to occlude. Get a more natural beauty effect.
  • the image processing apparatus provided in each embodiment of the present application can be used to execute the method shown in each corresponding embodiment described above, and the implementation manner and principle are the same, and details are not repeated here.
  • the present application also provides an electronic device and a readable storage medium.
  • FIG. 5 it is a block diagram of an electronic device according to an image processing method of an embodiment of the present application.
  • Electronic devices are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.
  • Electronic devices can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices.
  • the components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the application described and/or required herein.
  • the electronic device includes: one or more processors 501, a memory 502, and interfaces for connecting various components, including a high-speed interface and a low-speed interface.
  • the various components are connected to each other using different buses, and can be installed on a common motherboard or installed in other ways as needed.
  • the processor may process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input/output device (such as a display device coupled to an interface).
  • an external input/output device such as a display device coupled to an interface.
  • multiple processors and/or multiple buses can be used with multiple memories and multiple memories.
  • multiple electronic devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).
  • a processor 501 is taken as an example.
  • the memory 502 is a non-transitory computer-readable storage medium provided by this application.
  • the memory stores instructions executable by at least one processor, so that the at least one processor executes the image processing method provided in this application.
  • the non-transitory computer-readable storage medium of the present application stores computer instructions, and the computer instructions are used to make a computer execute the image processing method provided in the present application.
  • the memory 502 can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions/modules corresponding to the image processing method in the embodiment of the present application (for example, The processing module 41 and the receiving module 42 shown in FIG. 4).
  • the processor 501 executes various functional applications and data processing of the server by running non-transitory software programs, instructions, and modules stored in the memory 502, that is, implements the image processing method in the foregoing method embodiment.
  • the memory 502 may include a storage program area and a storage data area.
  • the storage program area may store an operating system and an application program required by at least one function; the storage data area may store data created according to the use of image processing electronic devices.
  • the memory 502 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices.
  • the memory 502 may optionally include a memory remotely provided with respect to the processor 501, and these remote memories may be connected to an image processing electronic device through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
  • the electronic device of the image processing method may further include: an input device 503 and an output device 504.
  • the processor 501, the memory 502, the input device 503, and the output device 504 may be connected by a bus or in other ways. In FIG. 5, the connection by a bus is taken as an example.
  • the input device 503 can receive input digital or character information, and generate key signal input related to the user settings and function control of the image processing electronic device, such as touch screen, keypad, mouse, track pad, touch pad, pointer, a Or multiple mouse buttons, trackballs, joysticks and other input devices.
  • the output device 504 may include a display device, an auxiliary lighting device (for example, LED), a tactile feedback device (for example, a vibration motor), and the like.
  • the display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.
  • Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, the programmable processor It can be a dedicated or general-purpose programmable processor that can receive data and instructions from the storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device. An output device.
  • machine-readable medium and “computer-readable medium” refer to any computer program product, device, and/or device used to provide machine instructions and/or data to a programmable processor ( For example, magnetic disks, optical disks, memory, programmable logic devices (PLD)), including machine-readable media that receive machine instructions as machine-readable signals.
  • machine-readable signal refers to any signal used to provide machine instructions and/or data to a programmable processor.
  • the systems and techniques described here can be implemented on a computer that has: a display device for displaying information to the user (for example, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) ); and a keyboard and a pointing device (for example, a mouse or a trackball) through which the user can provide input to the computer.
  • a display device for displaying information to the user
  • LCD liquid crystal display
  • keyboard and a pointing device for example, a mouse or a trackball
  • Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (for example, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including Acoustic input, voice input, or tactile input) to receive input from the user.
  • the systems and technologies described herein can be implemented in a computing system that includes back-end components (for example, as a data server), or a computing system that includes middleware components (for example, an application server), or a computing system that includes front-end components (for example, A user computer with a graphical user interface or a web browser, through which the user can interact with the implementation of the system and technology described herein), or includes such back-end components, middleware components, Or any combination of front-end components in a computing system.
  • the components of the system can be connected to each other through any form or medium of digital data communication (for example, a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.
  • the computer system can include clients and servers.
  • the client and server are generally far away from each other and usually interact through a communication network.
  • the relationship between the client and the server is generated by computer programs that run on the corresponding computers and have a client-server relationship with each other.
  • the shape of the target object in the face is adjusted according to the shape of the beauty material, and then the adjusted area is sectioned.
  • the solution of the embodiment of the present application that adjusts the shape first and then divides and fills can avoid the occurrence of the incapability phenomenon, and obtain a better result. Natural beauty effect.
  • the embodiments of the present application also provide a computer program, including program code.
  • program code executes the method described in any of the above embodiments.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
  • Image Processing (AREA)

Abstract

本申请实施例提供一种图像处理的方法及装置，涉及图像处理的人工智能领域，具体包括：获取包括人脸的图像；接收对人脸中的目标对象的美颜素材的选择操作，目标对象为人脸中的器官；利用预先训练的目标关键点模型识别目标对象在人脸中的关键点位置；利用关键点位置和美颜素材的形状调整目标对象在人脸中的形状；对调整后的目标对象在图像中的区域进行剖分，得到剖分后的区域；依据美颜素材填充剖分后的区域，得到目标图像。本申请实施例中采用先调整形状再剖分填充的方案，从而能够避免不能遮挡现象的发生，得到较为自然的美颜效果。

Description

图像处理的方法及装置
本申请要求于2020年04月29日提交中国专利局、申请号为CN202010358288.5、申请名称为“图像处理的方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理和计算机技术领域，尤其涉及人工智能和计算机视觉技术领域，具体涉及一种图像处理的方法及装置。
背景技术
随着终端设备的发展，较多的终端设备中包括了拍照设备，适应的美颜应用程序也得到了较多发展。
在美颜应用程序中，通常会提供一些美颜素材，用户可以根据需求选择美颜素材，应用程序可以将美颜素材添加在用户的脸部。例如，对于眉毛美化处理中，用户可以选择应用程序素材中提供的眉型，之后应用程序可以将应用程序素材中的眉型覆盖在用户脸部的眉毛部位，实现眉毛美化。
但是现有技术的眉毛美化通常不够自然，不能达到较好的美化效果。
发明内容
本申请实施例提供一种图像处理的方法及装置，以解决现有技术中美颜处理中美化效果差的技术问题。
本申请实施例第一方面提供一种图像处理的方法，包括：
获取图像，所述图像中包括人脸；
接收对所述人脸中的目标对象的美颜素材的选择操作，所述目标对象为人脸中的器官；
利用预先训练的目标关键点模型识别所述目标对象在所述人脸中的关键点位置；
利用所述关键点位置和所述美颜素材的形状调整所述目标对象在所述人脸中的形状；
对调整后的目标对象在所述图像中的区域进行剖分，得到剖分后的区域；
依据所述美颜素材填充所述剖分后的区域，得到目标图像。
本申请实施例第二方面提供一种图像处理的装置，包括：
处理模块，用于获取图像，所述图像中包括人脸；
接收模块，用于接收对所述人脸中的目标对象的美颜素材的选择操作，所述目标对象为人脸中的器官；
所述处理模块，还用于利用预先训练的目标关键点模型识别所述目标对象在所述人脸中的关键点位置；
所述处理模块，还用于利用所述关键点位置和所述美颜素材的形状调整所述目标对象在所述人脸中的形状；
所述处理模块，还用于对调整后的目标对象在所述图像中的区域进行剖分，得到剖分后的区域；
所述处理模块，还用于依据所述美颜素材填充所述剖分后的区域，得到目标图像。
本申请实施例第三方面提供一种电子设备，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如前述第一方面任一项所述的方法。
本申请实施例第四方面提供一种存储有计算机指令的非瞬时计算机可读存储介质，所述计算机指令用于使所述计算机执行如前述第一方面中任一项所述的方法。
本申请实施例第五方面提供一种计算机程序，包括程序代码，当计算机运行所述计算机程序时，所述程序代码执行如上任一实施例所述的方法。
综上所述，本申请实施例相对于现有技术的有益效果：
本申请实施例中提供了一种图像处理的方法及装置，在识别到目标对象在人脸中的关键点位置后，先依据美颜素材的形状调整目标对象在人脸中的形状，之后再对调整后的区域进行剖分和素材填充，相较于直接将美颜素材覆盖目标对象可能造成的不能完全遮挡的方案，本申请实施例的先调整形状再剖分填充的方案，能够避免不能遮挡现象的发生，得到较为自然的美颜效果。具体实现中，可以获取包括人脸的图像；接收对人脸中的目标对象的美颜素材的选择操作，目标对象为人脸中的器官；利用预先训练的目标关键点模型识别目标对象在人脸中的关键点位置；利用关键点位置和美颜素材的形状调整目标对象在人脸中的形状；对调整后的目标对象在图像中的区域进行剖分，得到剖分后的区域；依据美颜素材填充剖分后的区域，得到目标图像。即采用先调整形状再剖分填充的方案，从而能够避免不能遮挡现象的发生，得到较为自然 的美颜效果。
应当理解，本部分所描述的内容并非旨在标识本公开的实施例的关键或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的说明书而变得容易理解。
附图说明
图1为本申请实施例提供的图像处理的方法适用的系统架构示意图；
图2为本申请实施例提供的图像处理的方法的流程示意图；
图3为本申请实施例提供的眉毛处理示例示意图；
图4为本申请实施例提供的图像处理装置的结构示意图；
图5是用来实现本申请实施例的图像处理的方法的电子设备的框图。
具体实施方式
以下结合附图对本申请的示范性实施例做出说明，其中包括本申请实施例的各种细节以助于理解，应当将它们认为仅仅是示范性的。因此，本领域普通技术人员应当认识到，可以对这里描述的实施例做出各种改变和修改，而不会背离本申请的范围和精神。同样，为了清楚和简明，以下的描述中省略了对公知功能和结构的描述。在不冲突的情况下，下述的实施例及实施例中的特征可以相互组合。
通常的，在对人脸进行美颜处理时，用户可以选择美颜素材，美颜素材可以为人脸各器官的素材，终端设备可以将美颜素材覆盖在图像中用户原有的器官上。例如，在眉毛美化中，美颜素材中可以提供各种类型的眉型，用户可以选择其中的一种眉型，终端设备可以采用用户选择的眉型覆盖图像中本身的眉毛，实现眉毛美化。
但是，采用美颜素材覆盖真实器官的美颜方式，一方面美颜素材与图像的融合度差，美颜处理后图像显得不够自然，影响美颜效果。另一方面，还可能存在由于原图像中的器官较大或不规则，美颜素材不能很好遮挡的现象，例如，一些眉毛素材不能很好的覆盖粗眉、杂乱无形、不规则等眉型，或者，由于普通人的眉型各不相同，眉毛素材会有一定的拉伸变形，美颜后的图像中也会存在一定的拉伸变形效果，导致美颜效果较差。
基于此，本申请实施例中，在识别到目标对象在人脸中的关键点位置后，先依据美颜素材的形状调整目标对象在人脸中的形状，之后再对调整后的区域进行剖分和素材填充，相较于直接将美颜素材覆盖目标对象可能造成的不能完全遮挡的方案，本申请实施例的先调整形状再剖分填充的方案，能够避免不能遮挡现象的发生，得到较为 自然的美颜效果。
本申请实施例所描述的美颜素材可以是预先绘制或获取的素材，例如，以对眉毛美颜的场景为例，美颜素材可以包括黛玉眉、羽玉眉、柳叶眉等眉型。以嘴唇美颜的场景为例，嘴唇素材可以包括薄嘴唇、厚嘴唇等唇型。本申请实施例对美颜素材不做具体限定。美颜素材可以存储在网络或服务器中，则终端设备在进行美颜时，可以从网络或服务器中获取美颜素材。美颜素材也可以存储在终端设备本地，本申请实施例对美颜素材的获取不做具体限定。
本申请实施例所描述的目标对象可以为人脸的器官，例如眉毛、鼻子、嘴巴或眼睛等，本申请实施例对此不做具体限定。
本申请实施例所描述的目标关键点模型可以是采用深度学习训练得到的神经网络模型，例如可以采用包括人体器官关键点标注的样本图像，训练目标关键点模型，使得在向该目标关键点模型输入图像后，该目标关键点模型可以输出目标对象的关键点位置。
本申请实施例的方法可以应用于终端，终端可以包括：手机、平板电脑、笔记本电脑、或者台式电脑等电子设备。本申请实施例对应用的具体设备不作具体限定。
示例性的，终端可以提供图形用户界面(graphical user interface，GUI)，在图形用户界面中可以设置用于接收用户操作的控件、开关等，使得用户可以在图形用户界面中进行如本申请实施例的操作，可以理解，图形用户界面的具体内容可以根据实际的应用场景确定，本发明实施例对此不作具体限定。
如图1所示，图1为本申请实施例提供的方法所适用的应用场景架构示意图。
一种可能的应用场景中，美颜素材和目标关键点识别模型均在终端设备本地，终端设备11不需要与其他设备交互。示例性的，用户可以在终端设备11触发获取包括人脸的图像，并在终端设备11的图形用户界面中选择目标对象的美颜素材，终端设备11可以利用预先训练的目标关键点模型识别目标对象在人脸中的关键点位置；以及利用关键点位置和美颜素材的形状调整目标对象在人脸中的形状；对调整后的目标对象在图像中的区域进行剖分，得到剖分后的区域；依据美颜素材填充剖分后的区域，得到目标图像。即终端设备11采用先调整形状再剖分填充的方案，从而能够避免不能遮挡现象的发生，得到较为自然的美颜效果。
一种可能的应用场景中，美颜素材或目标关键点识别模型的一种或多种设置在服务器12中，终端设备11需要与服务器12交互。示例性的，用户可以在终端设备11触发获取包括人脸的图像，并在终端设备11的图形用户界面中选择目标对象的美颜素 材，终端设备11可以从服务器12获取美颜素材，或者还可以将图像发送给服务器12，由服务器12利用预先训练的目标关键点模型识别目标对象在人脸中的关键点位置，终端设备11可以进一步从服务器12获取关键点位置，以及利用关键点位置和美颜素材的形状调整目标对象在人脸中的形状；对调整后的目标对象在图像中的区域进行剖分，得到剖分后的区域；依据美颜素材填充剖分后的区域，得到目标图像。即采用先调整形状再剖分填充的方案，从而能够避免不能遮挡现象的发生，得到较为自然的美颜效果。可以理解，具体应用中，服务器的数量可以为大于或等于1的任意值，本申请实施例对此不作具体限定。
可以理解，本申请实施例还可能应用于其他的应用场景中，上述场景示例是为了更好的说明本申请实施例，而不是限定本申请实施例。
如图2所示，图2为本申请实施例提供的图像处理的方法的流程示意图。该方法具体可以包括：
S101：获取图像，所述图像中包括人脸。
本申请实施例中，图像可以是用户在终端设备本地的图片库中选择上传的。图像也可以是用户利用终端设备的摄像头等设备拍摄的。图像也可以是用户从网络下载的。本申请实施例对获取图像的具体方式不做限定。
一种可能的实现方式中，用户可以打开终端设备中的美颜应用程序(application，APP)，在该APP中实时拍摄图像，或在APP中上传终端设备本地的图像，得到包括人脸的图像。
S102：接收对所述人脸中的目标对象的美颜素材的选择操作，所述目标对象为人脸中的器官。
本申请实施例中，用户在选择需要美颜的图像后，可以进一步选择具体进行美颜的目标对象的美颜素材，目标对象可以是人脸中的器官，例如可以是眉毛、鼻子、嘴巴或眼睛等，以通过目标对象的美颜素材实现对目标对象的美颜。
一种可能的实现方式中，终端设备的用户界面中可以显示图像以及人脸器官对应的美颜素材，用户可以通过点击、拖动、语音控制等选择操作，实现对目标对象的美颜素材的选择。
可以理解，实际应用中，终端设备的用户界面的具体内容以及用户选择美颜素材需要的执行步骤均可以根据实际的应用场景确定，本申请实施例对此不做具体限定。
S103：利用预先训练的目标关键点模型识别所述目标对象在所述人脸中的关键点位置。
本申请实施例中，目标对象在人脸中的关键点位置可以用于标识目标对象在人脸中的区域，对于不同的目标对象，关键点的数量和位置均不同，本申请实施例对目标对象在人脸中的关键点位置不做具体限定。
一种可能的实现方式中，目标关键点模型可以预先训练并设置在美颜APP中，可以利用目标关键点模型识别目标对象在人脸中的关键点位置。
S104：利用所述关键点位置和所述美颜素材的形状调整所述目标对象在所述人脸中的形状。
本申请实施例中，在获取到目标对象在人脸中的关键点位置后，可以根据美颜素材的形状调整目标对象在人脸中的形状，使得目标对象在人脸中的形状与美颜素材的形状适配，从而后续可以根据调整后的目标对象的区域进行适应的美颜处理。
例如，以目标对象为眉毛为例，图像中人脸的眉型可能为粗平眉，而用户选择的美颜素材为黛玉眉，则可以首先将人脸中的眉毛形状调整为与美颜素材的形状相似或相同的形状。
可以理解，因为本申请实施例中，根据美颜素材的形状调整目标对象在人脸中的形状，使得目标对象在人脸中的形状与美颜素材的形状适配，因此，后续根据美颜素材进行美颜时，不会出现美颜素材无法覆盖目标对象等。
S105：对调整后的目标对象在所述图像中的区域进行剖分，得到剖分后的区域。
本申请实施例中，剖分可以是任意的多边形剖分，对调整后的目标对象在图像中的区域进行剖分，使得后续可以基于剖分后的区域进行填充，相对于直接采用美颜素材在图像中覆盖目标对象的方式，能与图像达到较好的融合效果，美颜效果更自然。
S106：依据所述美颜素材填充所述剖分后的区域，得到目标图像。
本申请实施例中，美颜素材中可以包括目标对象的颜色和形状等，依据美颜素材填充剖分后的区域，可以得到与原图像融合较好的目标图像。
综上所述，本申请实施例中，在识别到目标对象在人脸中的关键点位置后，先依据美颜素材的形状调整目标对象在人脸中的形状，之后再对调整后的区域进行剖分和素材填充，相较于直接将美颜素材覆盖目标对象可能造成的不能完全遮挡的方案，本申请实施例的先调整形状再剖分填充的方案，能够避免不能遮挡现象的发生，得到较为自然的美颜效果。具体实现中，可以获取包括人脸的图像；接收对人脸中的目标对象的美颜素材的选择操作，目标对象为人脸中的器官；利用预先训练的目标关键点模型识别目标对象在人脸中的关键点位置；利用关键点位置和美颜素材的形状调整目标对象在人脸中的形状；对调整后的目标对象在图像中的区域进行剖分，得到剖分后的 区域；依据美颜素材填充剖分后的区域，得到目标图像。即采用先调整形状再剖分填充的方案，从而能够避免不能遮挡现象的发生，得到较为自然的美颜效果。
在图2对应的实施例的基础上，一种可能的实现方式中，该方法还包括：在接收到对所述目标图像的分享操作的情况下，基于所述分享操作的路径分享所述目标图像。
本申请实施例中，终端设备的用户界面中可以提供一个或多个分享按钮，各分享按钮可以对应不同的分享路径，例如，每个分享按钮可以对应一种社交应用程序，在选择该分享按钮时，会基于该分享按钮将目标对象分享到对应的社交应用程序。
具体应用中，用户可以触发选择分享按钮，进而跳转到分享按钮对应的社交应用程序中，便捷的实现对目标图像的分享。
在图2对应的实施例的基础上，一种可能的实现方式中，该方法还包括：在接收到对所述目标图像的存储操作的情况下，存储所述目标图像。
本申请实施例中，终端设备的用户界面中可以提供一个或多个存储按钮，各存储按钮可以对应不同的存储路径，例如，存储按钮可以为存储到本地的按钮，存储按钮也可以为存储到云端的按钮，本申请实施例对此不做具体限定。
具体应用中，用户可以触发选择存储享按钮，进而可以便捷的实现对目标图像的存储。
在图2对应的实施例的基础上，一种可能的实现方式中，所述利用所述关键点位置和所述美颜素材的形状调整所述目标对象在所述人脸中的形状，包括：利用所述关键点位置得到所述目标对象在所述人脸中的源区域；将所述源区域的形状调整为与所述美颜素材的形状相似的形状。
本申请实施例中，可以根据关键点位置绘制目标对象在人脸中的源区域，例如可以连接全部的关键点位置，得到目标对象在人脸中的源区域。可以理解，目标对象在人脸中的源区域也可以称为该人脸中目标对象原本的区域，源区域的形状和大小与具体的人脸相关。
在得到源区域后，可以将源区域的形状调整为与美颜素材的形状相似的形状，例如，可以对源区域进行适应的拉伸、变形等，使得源区域在形状上与美颜素材一致，且在大小上与人脸的大小适配。
本申请实施例中，提供了一种具体的调整目标对象在人脸中形状的方式，可以较为便捷、准确的调整目标对象在人脸中的形状。
在图2对应的实施例的基础上，一种可能的实现方式中，所述对调整后的目标对 象在所述图像中的区域进行剖分，包括：对调整后的目标对象在所述图像中的区域进行三角剖分。
本申请实施例中，采用三角剖分的方式，可以得到较为稳定的剖分实现，从而在后续对图像中的目标对象处理时，能达到较好的处理效果。三角剖分是较为经典的剖分方式，本申请实施例对此不做赘述。
在图2对应的实施例的基础上，一种可能的实现方式中，所述依据所述美颜素材填充所述剖分后的区域，得到目标图像，包括：依据所述美颜素材在所述剖分后的区域进行纹理映射，得到所述目标图像。
本申请实施例中，在填充剖分后的区域时，可以基于美颜素材在剖分后的区域中进行纹理映射，实现较好的图像融合，得到具有较好融合效果的目标图像。
在图2对应的实施例的基础上，一种可能的实现方式中，所述获取图像包括：接收拍摄操作；响应于所述拍摄操作拍摄得到所述图像。
本申请实施例中，用户可以触发美颜APP中的拍摄按钮，实时拍摄得到图像，实现对图像的实时处理。
示例性的，图3示出了本申请实施例的图像处理方法的眉毛处理示例示意图。
美颜素材可以是在底面模特(例如脸部模特)中绘制得到的素材，在通过相机采样前处理(也可以称为相机拍照)得到图像后，可以基于眉毛关键点模型识别图像中眉毛的关键点位置，并基于眉毛的关键点位置和美颜素材对图像中眉毛的形状进行塑形调整，并基于剖分网络对调整后的区域进行剖分，和基于美颜素材实现对剖分区域的纹理映射得到融合后的图像，实现较好的图像融合，从而可以得到具有较好融合效果的目标图像。
图4为本申请提供的图像处理的装置一实施例的结构示意图。如图4所示，本实施例提供的图像处理的装置包括：
处理模块41，用于获取图像，所述图像中包括人脸；
接收模块42，用于接收对所述人脸中的目标对象的美颜素材的选择操作，所述目标对象为人脸中的器官；
所述处理模块41，还用于利用预先训练的目标关键点模型识别所述目标对象在所述人脸中的关键点位置；
所述处理模块41，还用于利用所述关键点位置和所述美颜素材的形状调整所述目标对象在所述人脸中的形状；
所述处理模块41，还用于对调整后的目标对象在所述图像中的区域进行剖分，得 到剖分后的区域；
所述处理模块41，还用于依据所述美颜素材填充所述剖分后的区域，得到目标图像。
一种可能的实现方式中，所述处理模块，具体用于：
利用所述关键点位置得到所述目标对象在所述人脸中的源区域；
将所述源区域的形状调整为与所述美颜素材的形状相似的形状。
一种可能的实现方式中，所述处理模块，具体用于：
对调整后的目标对象在所述图像中的区域进行三角剖分。
一种可能的实现方式中，所述处理模块，具体用于：
依据所述美颜素材在所述剖分后的区域进行纹理映射，得到所述目标图像。
一种可能的实现方式中，所述处理模块，具体用于：
接收拍摄操作；
响应于所述拍摄操作拍摄得到所述图像。
一种可能的实现方式中，所述目标对象为眉毛，所述美颜素材为眉毛素材。
一种可能的实现方式中，所述处理模块，还用于：
在接收到对所述目标图像的分享操作的情况下，基于所述分享操作的路径分享所述目标图像。
一种可能的实现方式中，所述处理模块，还用于：
在接收到对所述目标图像的存储操作的情况下，存储所述目标图像。
综上所述，本申请实施例中，在识别到目标对象在人脸中的关键点位置后，先依据美颜素材的形状调整目标对象在人脸中的形状，之后再对调整后的区域进行剖分和素材填充，相较于直接将美颜素材覆盖目标对象可能造成的不能完全遮挡的方案，本申请实施例的先调整形状再剖分填充的方案，能够避免不能遮挡现象的发生，得到较为自然的美颜效果。具体实现中，可以获取包括人脸的图像；接收对人脸中的目标对象的美颜素材的选择操作，目标对象为人脸中的器官；利用预先训练的目标关键点模型识别目标对象在人脸中的关键点位置；利用关键点位置和美颜素材的形状调整目标对象在人脸中的形状；对调整后的目标对象在图像中的区域进行剖分，得到剖分后的区域；依据美颜素材填充剖分后的区域，得到目标图像。即采用先调整形状再剖分填充的方案，从而能够避免不能遮挡现象的发生，得到较为自然的美颜效果。
本申请各实施例提供的图像处理的装置可用于执行如前述各对应的实施例所示的方法，其实现方式与原理相同，不再赘述。
根据本申请的实施例，本申请还提供了一种电子设备和一种可读存储介质。
如图5所示，是根据本申请实施例的图像处理的方法的电子设备的框图。电子设备旨在表示各种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示各种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本申请的实现。
如图5所示，该电子设备包括：一个或多个处理器501、存储器502，以及用于连接各部件的接口，包括高速接口和低速接口。各个部件利用不同的总线互相连接，并且可以被安装在公共主板上或者根据需要以其它方式安装。处理器可以对在电子设备内执行的指令进行处理，包括存储在存储器中或者存储器上以在外部输入/输出装置(诸如，耦合至接口的显示设备)上显示GUI的图形信息的指令。在其它实施方式中，若需要，可以将多个处理器和/或多条总线与多个存储器和多个存储器一起使用。同样，可以连接多个电子设备，各个设备提供部分必要的操作(例如，作为服务器阵列、一组刀片式服务器、或者多处理器系统)。图5中以一个处理器501为例。
存储器502即为本申请所提供的非瞬时计算机可读存储介质。其中，所述存储器存储有可由至少一个处理器执行的指令，以使所述至少一个处理器执行本申请所提供的图像处理的方法。本申请的非瞬时计算机可读存储介质存储计算机指令，该计算机指令用于使计算机执行本申请所提供的图像处理的方法。
存储器502作为一种非瞬时计算机可读存储介质，可用于存储非瞬时软件程序、非瞬时计算机可执行程序以及模块，如本申请实施例中的图像处理的方法对应的程序指令/模块(例如，附图4所示的处理模块41和接收模块42)。处理器501通过运行存储在存储器502中的非瞬时软件程序、指令以及模块，从而执行服务器的各种功能应用以及数据处理，即实现上述方法实施例中的图像处理的方法。
存储器502可以包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需要的应用程序；存储数据区可存储根据图像处理的电子设备的使用所创建的数据等。此外，存储器502可以包括高速随机存取存储器，还可以包括非瞬时存储器，例如至少一个磁盘存储器件、闪存器件、或其他非瞬时固态存储器件。在一些实施例中，存储器502可选包括相对于处理器501远程设置的存储器，这些远程存储器可以通过网络连接至图像处理的电子设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
图像处理的方法的电子设备还可以包括：输入装置503和输出装置504。处理器501、存储器502、输入装置503和输出装置504可以通过总线或者其他方式连接，图5中以通过总线连接为例。
输入装置503可接收输入的数字或字符信息，以及产生与图像处理的电子设备的用户设置以及功能控制有关的键信号输入，例如触摸屏、小键盘、鼠标、轨迹板、触摸板、指示杆、一个或者多个鼠标按钮、轨迹球、操纵杆等输入装置。输出装置504可以包括显示设备、辅助照明装置(例如，LED)和触觉反馈装置(例如，振动电机)等。该显示设备可以包括但不限于，液晶显示器(LCD)、发光二极管(LED)显示器和等离子体显示器。在一些实施方式中，显示设备可以是触摸屏。
此处描述的系统和技术的各种实施方式可以在数字电子电路系统、集成电路系统、专用ASIC(专用集成电路)、计算机硬件、固件、软件、和/或它们的组合中实现。这些各种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。
这些计算程序(也称作程序、软件、软件应用、或者代码)包括可编程处理器的机器指令，并且可以利用高级过程和/或面向对象的编程语言、和/或汇编/机器语言来实施这些计算程序。如本文使用的，术语“机器可读介质”和“计算机可读介质”指的是用于将机器指令和/或数据提供给可编程处理器的任何计算机程序产品、设备、和/或装置(例如，磁盘、光盘、存储器、可编程逻辑装置(PLD))，包括，接收作为机器可读信号的机器指令的机器可读介质。术语“机器可读信号”指的是用于将机器指令和/或数据提供给可编程处理器的任何信号。
为了提供与用户的交互，可以在计算机上实施此处描述的系统和技术，该计算机具有：用于向用户显示信息的显示装置(例如，CRT(阴极射线管)或者LCD(液晶显示器)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。
可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据 服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(LAN)、广域网(WAN)和互联网。
计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。
根据本申请实施例的技术方案，在识别到目标对象在人脸中的关键点位置后，先依据美颜素材的形状调整目标对象在人脸中的形状，之后再对调整后的区域进行剖分和素材填充，相较于直接将美颜素材覆盖目标对象可能造成的不能完全遮挡的方案，本申请实施例的先调整形状再剖分填充的方案，能够避免不能遮挡现象的发生，得到较为自然的美颜效果。具体实现中，可以获取包括人脸的图像；接收对人脸中的目标对象的美颜素材的选择操作，目标对象为人脸中的器官；利用预先训练的目标关键点模型识别目标对象在人脸中的关键点位置；利用关键点位置和美颜素材的形状调整目标对象在人脸中的形状；对调整后的目标对象在图像中的区域进行剖分，得到剖分后的区域；依据美颜素材填充剖分后的区域，得到目标图像。即采用先调整形状再剖分填充的方案，从而能够避免不能遮挡现象的发生，得到较为自然的美颜效果。
根据本申请实施例的另一个方面，本申请实施例还提供了一种计算机程序，包括程序代码，当计算机运行所述计算机程序时，所述程序代码执行如上任一实施例所述的方法。
应该理解，可以使用上面所示的各种形式的流程，重新排序、增加或删除步骤。例如，本申请中记载的各步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本申请公开的技术方案所期望的结果，本文在此不进行限制。
上述具体实施方式，并不构成对本申请保护范围的限制。本领域技术人员应该明白的是，根据设计要求和其他因素，可以进行各种修改、组合、子组合和替代。任何在本申请的精神和原则之内所作的修改、等同替换和改进等，均应包含在本申请保护范围之内。

Claims (19)

  1. 一种图像处理的方法，所述方法包括：
    获取图像，所述图像中包括人脸；
    接收对所述人脸中的目标对象的美颜素材的选择操作，所述目标对象为人脸中的器官；
    利用预先训练的目标关键点模型识别所述目标对象在所述人脸中的关键点位置；
    利用所述关键点位置和所述美颜素材的形状调整所述目标对象在所述人脸中的形状；
    对调整后的目标对象在所述图像中的区域进行剖分，得到剖分后的区域；
    依据所述美颜素材填充所述剖分后的区域，得到目标图像。
  2. 根据权利要求1所述的方法，其中，所述利用所述关键点位置和所述美颜素材的形状调整所述目标对象在所述人脸中的形状，包括：
    利用所述关键点位置得到所述目标对象在所述人脸中的源区域；
    将所述源区域的形状调整为与所述美颜素材的形状相似的形状。
  3. 根据权利要求1或2所述的方法，其中，所述对调整后的目标对象在所述图像中的区域进行剖分，包括：
    对调整后的目标对象在所述图像中的区域进行三角剖分。
  4. 根据权利要求1-3任一项所述的方法，其中，所述依据所述美颜素材填充所述剖分后的区域，得到目标图像，包括：
    依据所述美颜素材在所述剖分后的区域进行纹理映射，得到所述目标图像。
  5. 根据权利要求1-4任一项所述的方法，其中，所述获取图像包括：
    接收拍摄操作；
    响应于所述拍摄操作拍摄得到所述图像。
  6. 根据权利要求1-5任一项所述的方法，其中，所述目标对象为眉毛，所述美颜素材为眉毛素材。
  7. 根据权利要求1-6任一项所述的方法，所述方法，还包括：
    在接收到对所述目标图像的分享操作的情况下，基于所述分享操作的路径分享所述目标图像。
  8. 根据权利要求1-7任一项所述的方法，所述方法，还包括：
    在接收到对所述目标图像的存储操作的情况下，存储所述目标图像。
  9. 一种图像处理的装置，包括：
    处理模块，用于获取图像，所述图像中包括人脸；
    接收模块，用于接收对所述人脸中的目标对象的美颜素材的选择操作，所述目标对象为人脸中的器官；
    所述处理模块，还用于利用预先训练的目标关键点模型识别所述目标对象在所述人脸中的关键点位置；
    所述处理模块，还用于利用所述关键点位置和所述美颜素材的形状调整所述目标对象在所述人脸中的形状；
    所述处理模块，还用于对调整后的目标对象在所述图像中的区域进行剖分，得到剖分后的区域；
    所述处理模块，还用于依据所述美颜素材填充所述剖分后的区域，得到目标图像。
  10. 根据权利要求9所述的装置，其中，所述处理模块，具体用于：
    利用所述关键点位置得到所述目标对象在所述人脸中的源区域；
    将所述源区域的形状调整为与所述美颜素材的形状相似的形状。
  11. 根据权利要求9或10所述的装置，其中，所述处理模块，具体用于：
    对调整后的目标对象在所述图像中的区域进行三角剖分。
  12. 根据权利要求9-11任一项所述的装置，其中，所述处理模块，具体用于：
    依据所述美颜素材在所述剖分后的区域进行纹理映射，得到所述目标图像。
  13. 根据权利要求9-12任一项所述的装置，其中，所述处理模块，具体用于：
    接收拍摄操作；
    响应于所述拍摄操作拍摄得到所述图像。
  14. 根据权利要求9-13任一项所述的装置，其中，所述目标对象为眉毛，所述美颜素材为眉毛素材。
  15. 根据权利要求9-14任一项所述的装置，其中，所述处理模块，还用于：
    在接收到对所述目标图像的分享操作的情况下，基于所述分享操作的路径分享所述目标图像。
  16. 根据权利要求9-15任一项所述的装置，其中，所述处理模块，还用于：
    在接收到对所述目标图像的存储操作的情况下，存储所述目标图像。
  17. 一种电子设备，包括：
    至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执 行，以使所述至少一个处理器能够执行如权利要求1-8任一项所述的方法的指令。
  18. 一种存储有计算机指令的非瞬时计算机可读存储介质，计算机指令用于使所述计算机执行如权利要求1-8任一项所述的方法。
  19. 一种计算机程序，包括程序代码，当计算机运行所述计算机程序时，所述程序代码执行如权利要求1-8任一项所述的方法。
PCT/CN2020/120102 2020-04-29 2020-10-10 图像处理的方法及装置 WO2021218040A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2021526360A JP7183414B2 (ja) 2020-04-29 2020-10-10 画像処理の方法及び装置
EP20886186.4A EP3933751A4 (en) 2020-04-29 2020-10-10 IMAGE PROCESSING METHOD AND APPARATUS
KR1020217015566A KR20210070377A (ko) 2020-04-29 2020-10-10 이미지 처리 방법 및 장치
US17/330,507 US20210279928A1 (en) 2020-04-29 2021-05-26 Method and apparatus for image processing

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010358288.5A CN111563855B (zh) 2020-04-29 2020-04-29 图像处理的方法及装置
CN202010358288.5 2020-04-29

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/330,507 Continuation US20210279928A1 (en) 2020-04-29 2021-05-26 Method and apparatus for image processing

Publications (1)

Publication Number Publication Date
WO2021218040A1 true WO2021218040A1 (zh) 2021-11-04

Family

ID=72073324

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/120102 WO2021218040A1 (zh) 2020-04-29 2020-10-10 图像处理的方法及装置

Country Status (3)

Country Link
EP (1) EP3933751A4 (zh)
CN (1) CN111563855B (zh)
WO (1) WO2021218040A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115018698A (zh) * 2022-08-08 2022-09-06 深圳市联志光电科技有限公司 一种用于人机交互的图像处理方法及系统

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111563855B (zh) * 2020-04-29 2023-08-01 百度在线网络技术（北京）有限公司 图像处理的方法及装置
CN112150387B (zh) * 2020-09-30 2024-04-26 广州光锥元信息科技有限公司 对照片中的人像增强五官立体感的方法及装置
CN112529808A (zh) * 2020-12-15 2021-03-19 北京映客芝士网络科技有限公司 一种图像的颜色调整方法、装置、设备和介质
CN112766215B (zh) * 2021-01-29 2024-08-09 北京字跳网络技术有限公司 人脸图像处理方法、装置、电子设备及存储介质
CN112766234B (zh) * 2021-02-23 2023-05-12 北京市商汤科技开发有限公司 图像处理方法及装置、电子设备和存储介质
CN113344776B (zh) * 2021-06-30 2023-06-27 北京字跳网络技术有限公司 图像处理方法、模型训练方法、装置、电子设备及介质
CN113781293B (zh) * 2021-09-08 2024-09-06 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备和计算机可读存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120177288A1 (en) * 2009-08-04 2012-07-12 Vesalis Image-processing method for correcting a target image with respect to a reference image, and corresponding image-processing device
CN107464253A (zh) * 2017-07-10 2017-12-12 北京小米移动软件有限公司 眉毛定位方法及装置
CN109063560A (zh) * 2018-06-28 2018-12-21 北京微播视界科技有限公司 图像处理方法、装置、计算机可读存储介质和终端
CN109937434A (zh) * 2017-10-18 2019-06-25 腾讯科技（深圳）有限公司 图像处理方法、装置、终端和存储介质
CN110969120A (zh) * 2019-11-29 2020-04-07 北京达佳互联信息技术有限公司 图像处理方法及装置、电子设备、可读存储介质
CN111563855A (zh) * 2020-04-29 2020-08-21 百度在线网络技术（北京）有限公司 图像处理的方法及装置

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101299267A (zh) * 2008-07-02 2008-11-05 北京中星微电子有限公司 一种人脸图像处理方法及装置
CN105184249B (zh) * 2015-08-28 2017-07-18 百度在线网络技术（北京）有限公司 用于人脸图像处理的方法和装置
CN206729262U (zh) * 2017-04-25 2017-12-12 袁虹冲 多功能人脸型方便口罩
CN107835367A (zh) * 2017-11-14 2018-03-23 维沃移动通信有限公司 一种图像处理方法、装置及移动终端
CN108022207A (zh) * 2017-11-30 2018-05-11 广东欧珀移动通信有限公司 图像处理方法、装置、存储介质和电子设备
CN108509846B (zh) * 2018-02-09 2022-02-11 腾讯科技（深圳）有限公司 图像处理方法、装置、计算机设备、存储介质及计算机程序产品
CN109087239B (zh) * 2018-07-25 2023-03-21 腾讯科技（深圳）有限公司 一种人脸图像处理方法、装置及存储介质
CN109584151B (zh) * 2018-11-30 2022-12-13 腾讯科技（深圳）有限公司 人脸美化方法、装置、终端及存储介质
CN110197462A (zh) * 2019-04-16 2019-09-03 浙江理工大学 一种人脸图像实时美化与纹理合成方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120177288A1 (en) * 2009-08-04 2012-07-12 Vesalis Image-processing method for correcting a target image with respect to a reference image, and corresponding image-processing device
CN107464253A (zh) * 2017-07-10 2017-12-12 北京小米移动软件有限公司 眉毛定位方法及装置
CN109937434A (zh) * 2017-10-18 2019-06-25 腾讯科技（深圳）有限公司 图像处理方法、装置、终端和存储介质
CN109063560A (zh) * 2018-06-28 2018-12-21 北京微播视界科技有限公司 图像处理方法、装置、计算机可读存储介质和终端
CN110969120A (zh) * 2019-11-29 2020-04-07 北京达佳互联信息技术有限公司 图像处理方法及装置、电子设备、可读存储介质
CN111563855A (zh) * 2020-04-29 2020-08-21 百度在线网络技术（北京）有限公司 图像处理的方法及装置

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115018698A (zh) * 2022-08-08 2022-09-06 深圳市联志光电科技有限公司 一种用于人机交互的图像处理方法及系统

Also Published As

Publication number Publication date
EP3933751A1 (en) 2022-01-05
CN111563855A (zh) 2020-08-21
EP3933751A4 (en) 2022-07-27
CN111563855B (zh) 2023-08-01

Similar Documents

Publication Publication Date Title
WO2021218040A1 (zh) 图像处理的方法及装置
US11587300B2 (en) Method and apparatus for generating three-dimensional virtual image, and storage medium
US11854118B2 (en) Method for training generative network, method for generating near-infrared image and device
CN111652828B (zh) 人脸图像生成方法、装置、设备和介质
CN111598818B (zh) 人脸融合模型训练方法、装置及电子设备
JP7110502B2 (ja) 深度を利用した映像背景減算法
CN111783647B (zh) 人脸融合模型的训练方法、人脸融合方法、装置及设备
US11887235B2 (en) Puppeteering remote avatar by facial expressions
CN111294665B (zh) 视频的生成方法、装置、电子设备及可读存储介质
US11074437B2 (en) Method, apparatus, electronic device and storage medium for expression driving
JP7389840B2 (ja) 画像画質補強方法、装置、機器および媒体
CN111709875B (zh) 图像处理方法、装置、电子设备及存储介质
CN110688948A (zh) 视频中人脸性别变换方法、装置、电子设备和存储介质
US20230143452A1 (en) Method and apparatus for generating image, electronic device and storage medium
US20210279928A1 (en) Method and apparatus for image processing
US20230146178A1 (en) Attention based audio adjustment in virtual environments
US20210352181A1 (en) Transparency adjustment method and document camera
CN112381927A (zh) 图像生成的方法、装置、设备以及存储介质
CN113223128B (zh) 用于生成图像的方法和装置
CN114419253A (zh) 一种卡通人脸的构建、直播方法及相关装置
CN113822964A (zh) 图像优化渲染的方法、装置、设备及存储介质
CN116385643B (zh) 虚拟形象生成、模型的训练方法、装置及电子设备
WO2024113779A1 (zh) 图像处理方法、装置及相关设备
CN115546082A (zh) 镜头光晕的生成方法和装置、电子装置

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2021526360

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20217015566

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2020886186

Country of ref document: EP

Effective date: 20210520

NENP Non-entry into the national phase

Ref country code: DE