CN113744127A

CN113744127A - Method and device for realizing beautiful pupil, electronic equipment and readable storage medium

Info

Publication number: CN113744127A
Application number: CN202110994798.6A
Authority: CN
Inventors: 贾辰
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-08-27
Filing date: 2021-08-27
Publication date: 2021-12-03

Abstract

The disclosure provides a method and a device for realizing beautiful pupil, electronic equipment and a readable storage medium, and relates to the technical field of enhancement/virtual reality and image processing, in particular to the technical field of beautiful makeup in live broadcasting. The specific implementation scheme is as follows: determining a beauty pupil image to be added from a multi-frame image containing different beauty pupil images, adding the beauty pupil image to be added to a target image in a target video, and executing the operation for multiple times, so that the beauty pupil images added by different target images are different, and the video added with the beauty pupil image can show a dynamic beauty pupil effect when being played.

Description

Method and device for realizing beautiful pupil, electronic equipment and readable storage medium

Technical Field

The present disclosure relates to the field of augmented/virtual reality and image processing technologies, and in particular to the field of beauty cosmetics technology in live broadcast.

Background

With the development of mobile internet and video technology, live video becomes a popular social activity. Corresponding beauty treatment processing such as large-eye processing, face thinning processing, pupil beautifying processing and the like is usually carried out in the live video.

Disclosure of Invention

The disclosure provides a method and a device for realizing beautiful pupil, an electronic device and a readable storage medium.

According to a first aspect of the present disclosure, a method for implementing beautiful pupil is provided, including:

determining a beautiful pupil image to be added from a plurality of frames of images, wherein the plurality of frames of images comprise at least two different beautiful pupil images;

adding the determined cosmetic pupil image to be added to a pupil area of a target object in a current target image of a target video to obtain a current target image with the cosmetic pupil added;

and executing the operation on the subsequent target image of the target video to obtain the target video with the dynamic pupil beautifying effect.

According to a second aspect of the present disclosure, there is provided a cosmetic pupil implementing apparatus, including:

the device comprises a first determining module, a second determining module and a processing module, wherein the first determining module is used for determining a beautiful pupil image to be added from a multi-frame image, and the multi-frame image comprises at least two different beautiful pupil images;

the adding module is used for adding the determined beautifying pupil image to be added to the pupil area of the target object in the current target image of the target video to obtain the current target image with the added beautifying pupil;

and the execution module is used for executing the operation on the subsequent target image of the target video to obtain the target video with the dynamic pupil beautifying effect.

According to a third aspect of the present disclosure, there is provided an electronic apparatus comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the above method.

According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the above-described method.

The technical scheme provided by the disclosure has the following beneficial effects:

compared with the prior art that only static beautiful pupils can be added, the scheme provided by the embodiment of the disclosure is provided. The method comprises the steps that a beautiful pupil image to be added is determined from a plurality of frames of images, wherein the plurality of frames of images comprise at least two beautiful pupil images which are different from each other; adding the determined cosmetic pupil image to be added to a pupil area of a target object in a current target image of a target video to obtain a current target image with the cosmetic pupil added; and executing the operation on the subsequent target image of the target video to obtain the target video with the dynamic pupil beautifying effect. Namely, the cosmetic pupil image to be added is determined from the multi-frame images containing different cosmetic pupil images, the cosmetic pupil image to be added is added to the target image in the target video, and the operation is executed for multiple times, so that the cosmetic pupil images added by different target images are different, and the video added with the cosmetic pupil image can show a dynamic cosmetic pupil effect when being played.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

figure 1 is a schematic flow diagram of a cosmetic pupil implementation method provided in accordance with the present disclosure;

figure 2 is an exemplary graph of a plain eye state versus a static cosmetic pupil provided in accordance with the present disclosure;

figure 3 is an exemplary graph of a plain eye state versus a dynamic cosmetic pupil provided in accordance with the present disclosure;

FIG. 4 is an illustration of an eye mask graph provided in accordance with the present disclosure;

figure 5 is an illustration of a pupil patch provided in accordance with the present disclosure;

figure 6 is an exemplary diagram of pupil casting provided in accordance with the present disclosure;

FIG. 7 is an exemplary illustration of face scoring provided in accordance with the present disclosure;

FIG. 8 is an exemplary illustration of eye keypoints provided in accordance with the present disclosure;

figure 9 is a schematic structural view of a cosmetic pupil realisation apparatus provided according to the present disclosure;

FIG. 10 is a block diagram of an electronic device used to implement an embodiment of the disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Example one

Fig. 1 shows a method for implementing cosmetic pupils provided by an embodiment of the present disclosure, as shown in fig. 1, including:

step S101, determining a beautiful pupil image to be added from a plurality of frames of images, wherein the plurality of frames of images comprise at least two different beautiful pupil images;

step S102, adding the determined cosmetic pupil image to be added to the pupil area of a target object in the current target image of the target video to obtain the current target image with the cosmetic pupil added;

step S103, the following target images of the target video are subjected to the operation, and the target video with the dynamic pupil beautifying effect is obtained.

The implementation scheme of the beauty pupil in the prior art is static beauty pupil, such as a comparison graph of an original eye state and an added static beauty pupil shown in fig. 2, wherein a left graph in fig. 2 is the original eye state (i.e., the beauty pupil is not added), a right graph is an effect after the static beauty pupil is added (i.e., the static beauty pupil), and the brightness of the pupil is enhanced.

The method disclosed by the invention can be applied to live video scenes or short videos, animations and the like, and can add dynamic beautiful pupils to the eyes of the target object in the videos, wherein the videos disclosed by the invention can comprise animations such as GIF animations.

The multi-frame image can comprise a plurality of frames of American pupil images, and the American pupil images of the frames are different from one another; the multi-frame image may be a sequence frame image having animation effect during playing, and may be different slightly between each frame of beautiful pupil image, and the different beautiful pupil images are played in sequence to have the effect of GIF animation or have certain special effect, such as photoelectric flicker, etc.

The target video in the application can be a live video or an offline video, such as a shot short video. The target object in the target video is not limited to a human being, and may be another animal.

Specifically, when a predetermined condition is met, such as a trigger action of adding a dynamic cosmetic pupil is detected, a cosmetic pupil image to be added is determined from the multi-frame image, and the determined cosmetic pupil image to be added is added to a pupil area of a target object in a current target image of the target video, so that a video frame or a target image of the cosmetic pupil image is obtained. Then, the above actions (i.e. the actions of step S101 and step S102) are repeatedly performed on the subsequent target images of the target video, so that the cosmetic pupil images can be added to the corresponding target images of the target video, and since the cosmetic pupil images in the multi-frame images are different in at least two frames of cosmetic pupil images or are different from each other, when the target video with the cosmetic pupil images added is played, the cosmetic pupils of the target object in the video at different moments are different, thereby presenting a dynamic cosmetic pupil effect.

The multi-frame image can be divided into two sequences respectively aiming at the left eye and the right eye of the target object, wherein the two sequences can be different or the same. In the specific implementation, the cosmetic pupils may be added to both the left and right eyes of the target object, or only one of the cosmetic pupil images may be added.

Specifically, the target image of the target video may include a plurality of video frame images, and the same beautiful pupil image may be added to the plurality of video frame images belonging to the target image; then, the same beautiful pupil image can be added for the subsequent multi-frame video frame images belonging to the same target image of the target video. Because the beautiful pupil images added by different target images are different, when the video added with the beautiful pupil images is played, the eyes of the target object have the effect of dynamic beautiful pupil.

For better understanding of the technical solution of the present disclosure, exemplarily, as shown in fig. 3, a comparison graph of a prime eye state and an added dynamic cosmetic pupil is shown, where a left graph is the prime eye state, and a right graph is an effect graph of the added dynamic cosmetic pupil (photoelectric flicker); the effect diagram of adding dynamic cosmetic pupil (photoelectric flicker) only shows one frame of adding the photoelectric flicker effect, and other related frames are not shown, and it can be understood that a certain dynamic change of photoelectric flicker can be presented in cooperation with other preceding frames or subsequent frames.

The embodiment of the present application provides a possible implementation manner, where determining a cosmetic pupil image to be added from a multi-frame image includes:

and determining a beautiful pupil image from the multi-frame images in a preset sequence as the beautiful pupil image to be added.

Specifically, the cosmetic pupil image may be determined as the cosmetic pupil image to be added from the multi-frame images (specifically, one sequence of frame images having an animation effect) in a predetermined order for enabling the cosmetic pupil image in the multi-frame images to exhibit the animation effect when played.

Illustratively, the beautiful pupil sequence frame image includes A, B, C, D, E five frames of beautiful pupil images, wherein the five frames of beautiful pupil images may constitute one GIF animation. In the dynamic cosmetic pupil adding process, the dynamic cosmetic pupil adding process can be carried out according to the ABCDE sequence. Firstly, an aesthetic pupil image (such as A) to be added is determined, and for a current target image (such as all image frames of the first second) of the target video, A is added to the eyes of a target object in the current target image. Then, the above operation is also performed for subsequent target images of the target video, such as adding the determined B to the target image (all image frames of the second) for the target image (all image frames of the second); adding the determined C to the target image (all image frames of the third second) as for the target image (all image frames of the third second); adding the determined D to the target image (all image frames of the fourth second) as for the target image (all image frames of the fourth second); as for the target image (all image frames of the fifth second), the determined E is added to the target image (all image frames of the fifth second).

Specifically, if the cosmetic pupil image in the multi-frame image or cosmetic pupil sequence frame image is added once, the addition may be performed again from the beginning.

Illustratively, as in the above example, the beautiful pupil images A, B, C, D, E may be added for the 6 th to 10 th second images of the target video, respectively.

The method and the device solve the problem of how to enable the animation effect of the beautiful pupil presentation added by the target object in the target video added with the beautiful pupil image to be better.

The embodiment of the present application provides a possible implementation manner, wherein the determining of the multi-frame image or the cosmetic pupil sequence frame image includes the following steps:

determining an eye mask image;

and respectively sampling preset multi-frame American pupil maps based on the determined eye mask map to obtain the multi-frame image or the American pupil sequence frame image, wherein the multi-frame American pupil maps comprise at least two different American pupil maps.

The eye mask image may be obtained by extracting a standard face image, or may be obtained by performing image analysis and extraction on an eye region of a target object in a target video.

Illustratively, an eye mask map (as shown in fig. 4) is determined, and then predetermined multi-frame cosmetic pupil maps (as shown in fig. 5) are respectively sampled based on the determined eye mask map, so as to obtain an addable cosmetic pupil sequence frame image, wherein the multi-frame cosmetic pupil maps comprise multi-frame cosmetic pupil maps, and the cosmetic pupil maps of the frames are different from each other.

Specifically, the standard face may be further processed by triangle separation through a corresponding face detection algorithm and a triangle separation method to obtain a corresponding pupil separation map (as shown in fig. 6), and the processing may be specifically performed by referring to the technique for obtaining the face separation map (see fig. 7).

Specifically, the cosmetic pupil map may be added to the obtained pupil cast map, and the alignment process is performed, and then the cosmetic pupil map added to the pupil cast map is sampled through the eye mask map, so as to obtain the multi-frame image or the cosmetic pupil sequence frame image.

For the embodiment of the application, the problem of determining the American pupil sequence frame image is solved.

The embodiment of the present application provides a possible implementation manner, where the adding the determined cosmetic pupil image to be added to the pupil area of the target object in the current target image of the target video to obtain the current target image with the cosmetic pupil added includes:

determining the area of the pupil of the target object in the current target image;

and adding the beautiful pupil image to be added to the current target image based on the determined area where the pupil of the target object is located, so as to obtain the current target image with the added beautiful pupil.

Specifically, the region where the pupil of the target object is located in the current target image is identified and determined through a corresponding image identification method (for example, through a trained image detection and identification model).

Specifically, training of the image detection recognition model can be performed by a corresponding supervised learning method through a sample set of labels including an image of the target object and a corresponding pupil region position.

Specifically, a cosmetic pupil image to be added is added to the current target image, so that the current target image with the cosmetic pupil added is obtained.

With the application, the problem of how to add the beautiful pupil image to the current target image is solved.

determining the eye closure degree of a target object in a current target image;

determining a scaling ratio of the cosmetic pupil to be added based on the determined eye closure degree of the current target object;

and adding the to-be-added beautiful pupil image processed according to the scaling to the current target image.

However, when the cosmetic pupil image is added, it is necessary to take into consideration the case where the human eye is closed. When closed or semi-closed, the cosmetic pupil image to be added should be correspondingly reduced until disappearing to ensure that the added cosmetic pupil image does not stick to the eyelid.

Specifically, the eye closure degree of the target object in the current target image may be determined by a human eye closure detection algorithm based on human eye aspect ratio calculation.

According to the application, the beauty pupil to be added is added according to the closeness of the eyes of the target object, so that the problem that when the eyes are not completely opened, obvious attaching traces such as saw teeth or color mutation exist around the pupil part is solved.

The embodiment of the present application provides a possible implementation manner, where the determining an eye closure degree of a target object in a current target image includes:

identifying eye key points of eyes of the target object in the current target image;

determining eye closure of the target object in the current target image based on the eye keypoints determined by the identification.

The face key point detection is also called face key point detection, positioning or face alignment, and refers to positioning key region positions of a face, including eyebrows, eyes, a nose, a mouth, a face contour and the like, of a given face image.

Specifically, the face key point algorithm may be adjusted accordingly, and may be used as the eye key point detection of the present application.

Regarding the degree of closure of the human eye, illustratively, as shown in fig. 8, the key points of the positions of the corresponding eyes can be identified first. Taking the selected left eye as an example, the degree of opening (or closing) of the eye is calculated by calculating the ordinate of

points

15 and 19 and the abscissa of

points

13 and 17. The formula is adopted: 1/2 (19.y-15.y)/(17.x-13.x), and comparing the result with the calculated value of the target subject when the eyes are closed and/or the eyes are maximally open, thereby determining whether the current eye is open or closed. According to the obtained closing degree, the beautifying pupil image to be added is scaled in proportion, so that the function that the beautifying pupil is only displayed on the pupil when the eye is in a semi-closed state is realized, and the problem that the beautifying pupil can be displayed on the eyelid when the eye is in the semi-closed state is solved. Wherein, the closing degree is the eye closing period from large to small, and the eye opening period from small to large.

With the embodiment of the application, the problem of determining the degree of closure of the human eye is solved.

The embodiment of the present application provides a possible implementation manner, wherein the adding the determined cosmetic pupil image to be added to the pupil area of the target object in the current target image of the target video to obtain the current target image with the cosmetic pupil added later includes:

performing fusion processing on the current target image added with the beautiful pupil based on a fusion technology, wherein the fusion technology comprises at least one of the following items: positive film folding, darkening, brightening, color filtering, linear deepening, linear weakening and soft light.

The positive bottom-stacking technology is a color mixing scheme, and the colors of the two image layers can be mixed into a whole without floating on the surface.

Specifically, a positive film bottom-stacking technology can be adopted to mix the beauty pupil image to be added with the current target image, so that the colors of the beauty pupil image to be added and the current target image are fused, and the beauty pupil effect is improved.

Specifically, the current target image with the cosmetic pupil added can be further subjected to corresponding other processing, such as darkening, brightening, color filtering, linear deepening, linear dimming and soft lighting, so that the cosmetic pupil effect is further improved.

Example two

The embodiment of the present disclosure provides a pupil beautifying implementation device, as shown in fig. 9, including:

a first determining module 901, configured to determine a cosmetic pupil image to be added from multiple frames of images, where the multiple frames of images include at least two different cosmetic pupil images;

an adding module 902, configured to add the determined cosmetic pupil image to be added to a pupil area of a target object in a current target image of a target video, so as to obtain a current target image with a cosmetic pupil added;

an executing module 903, configured to execute the above operation on a subsequent target image of the target video, so as to obtain the target video with a dynamic pupil beautifying effect.

The embodiment of the present application provides a possible implementation manner, wherein the first determining module is specifically configured to determine a cosmetic pupil image from the cosmetic pupil sequence frame image according to a predetermined sequence, as the cosmetic pupil image to be added.

The embodiment of the present application provides a possible implementation manner, wherein the apparatus further includes:

a second determining module, configured to determine an eye mask image;

the sampling module is used for respectively sampling preset multi-frame American pupil maps based on the determined eye mask map to obtain the American pupil sequence frame image, wherein the multi-frame American pupil maps comprise at least two different American pupil maps.

The embodiment of the present application provides a possible implementation manner, wherein the adding module includes:

the first determining unit is used for determining the area where the pupil of the target object is located in the current target image;

and the first adding unit is used for adding the beautiful pupil image to be added to the current target image based on the determined area where the pupil of the target object is located, so as to obtain the current target image with the beautiful pupil added.

a second determination unit, configured to determine an eye closure degree of the target object in the current target image;

a third determination unit for determining a scaling ratio of the cosmetic pupil to be added based on the determined eye closure degree of the current target object;

and the second adding unit is used for adding the cosmetic pupil image to be added after the cosmetic pupil image is processed according to the scaling ratio to the current target image.

The embodiment of the present application provides a possible implementation manner, wherein the second determining unit is configured to identify and determine an eye key point of an eye of a target object in the current target image; and means for determining eye closure of the target object in the current target image based on the identified eye keypoints.

the fusion module is used for carrying out fusion processing on the current target image added with the beautiful pupil based on a fusion technology, and the fusion technology comprises at least one of the following items: positive film folding, darkening, brightening, color filtering, linear deepening, linear weakening and soft light.

For the embodiment of the present application, the beneficial effects achieved by the embodiment of the present application are the same as those of the embodiment of the method described above, and are not described herein again.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and regulations, and do not violate the good customs of the public order.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

The electronic device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as provided by the embodiments of the present disclosure.

Compared with the prior art, the electronic equipment only can add static beautiful pupils. The method comprises the steps that a beautiful pupil image to be added is determined from a plurality of frames of images, wherein the plurality of frames of images comprise at least two beautiful pupil images which are different from each other; adding the determined cosmetic pupil image to be added to a pupil area of a target object in a current target image of a target video to obtain a current target image with the cosmetic pupil added; and executing the operation on the subsequent target image of the target video to obtain the target video with the dynamic pupil beautifying effect. Namely, the cosmetic pupil image to be added is determined from the multi-frame images containing different cosmetic pupil images, the cosmetic pupil image to be added is added to the target image in the target video, and the operation is executed for multiple times, so that the cosmetic pupil images added by different target images are different, and the video added with the cosmetic pupil image can show a dynamic cosmetic pupil effect when being played.

The readable storage medium is a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform a method as provided by an embodiment of the present disclosure.

The readable storage medium is comparable to the prior art where only static cosmetic pupils can be added. The method comprises the steps that a beautiful pupil image to be added is determined from a plurality of frames of images, wherein the plurality of frames of images comprise at least two beautiful pupil images which are different from each other; adding the determined cosmetic pupil image to be added to a pupil area of a target object in a current target image of a target video to obtain a current target image with the cosmetic pupil added; and executing the operation on the subsequent target image of the target video to obtain the target video with the dynamic pupil beautifying effect. Namely, the cosmetic pupil image to be added is determined from the multi-frame images containing different cosmetic pupil images, the cosmetic pupil image to be added is added to the target image in the target video, and the operation is executed for multiple times, so that the cosmetic pupil images added by different target images are different, and the video added with the cosmetic pupil image can show a dynamic cosmetic pupil effect when being played.

The computer program product comprising a computer program which, when executed by a processor, implements a method as shown in the first aspect of the disclosure.

Compared with the prior art, the computer program product can only add static beautiful pupils. The method comprises the steps that a beautiful pupil image to be added is determined from a plurality of frames of images, wherein the plurality of frames of images comprise at least two beautiful pupil images which are different from each other; adding the determined cosmetic pupil image to be added to a pupil area of a target object in a current target image of a target video to obtain a current target image with the cosmetic pupil added; and executing the operation on the subsequent target image of the target video to obtain the target video with the dynamic pupil beautifying effect. Namely, the cosmetic pupil image to be added is determined from the multi-frame images containing different cosmetic pupil images, the cosmetic pupil image to be added is added to the target image in the target video, and the operation is executed for multiple times, so that the cosmetic pupil images added by different target images are different, and the video added with the cosmetic pupil image can show a dynamic cosmetic pupil effect when being played.

FIG. 10 illustrates a schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 10, the apparatus 1000 includes a computing unit 1001 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)1002 or a computer program loaded from a storage unit 1008 into a Random Access Memory (RAM) 1003. In the RAM 1003, various programs and data necessary for the operation of the device 1000 can also be stored. The calculation unit 1001, the ROM 1002, and the RAM 1003 are connected to each other by a bus 1004. An input/output (I/O) interface 1007 is also connected to bus 1004.

A number of components in device 1000 are connected to I/O interface 1005, including: an input unit 1006 such as a keyboard, a mouse, and the like; an output unit 1007 such as various types of displays, speakers, and the like; a storage unit 1008 such as a magnetic disk, an optical disk, or the like; and a communication unit 1009 such as a network card, a modem, a wireless communication transceiver, or the like. The communication unit 1009 allows the device 1000 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

Computing unit 1001 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 1001 executes the respective methods and processes described above, such as the method cosmetic pupil implementation method. For example, in some embodiments, the method aesthetic pupil implementation may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 1007. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 1000 via ROM 1002 and/or communications unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the method cosmetic pupil implementation method described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured by any other suitable means (e.g., by means of firmware) to perform the method aesthetic pupil implementation method.

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method for realizing beautiful pupil comprises the following steps:

2. The method of claim 1, wherein the determining from the plurality of frame images that a cosmetic pupil image is to be added comprises:

3. The method of claim 1, wherein the determination of the plurality of frame images comprises the steps of:

determining an eye mask image;

and respectively sampling preset multi-frame American pupil maps based on the determined eye mask map to obtain the multi-frame image, wherein the multi-frame American pupil maps comprise at least two different American pupil maps.

4. The method according to claim 1, wherein the adding the determined cosmetic pupil image to be added to a pupil area of a target object in a current target image of a target video to obtain the current target image with the cosmetic pupil added comprises:

5. The method according to any one of claims 1 to 4, wherein the adding the determined cosmetic pupil image to be added to a pupil area of a target object in a current target image of a target video to obtain a current target image with a cosmetic pupil added comprises:

6. The method of claim 5, wherein the determining eye closure of the target object in the current target image comprises:

7. The method according to claim 1, wherein the determined cosmetic pupil image to be added is added to a pupil area of a target object in a current target image of the target video, resulting in the current target image with cosmetic pupil added, and then comprising:

8. A cosmetic pupil implementing device, comprising:

9. The apparatus according to claim 8, wherein the first determining module is specifically configured to determine a cosmetic pupil image from the multiple frame images as the cosmetic pupil image to be added in a predetermined order.

10. The apparatus of claim 8, wherein the apparatus further comprises:

a second determining module, configured to determine an eye mask image;

the sampling module is used for respectively sampling preset multi-frame American pupil maps based on the determined eye mask map to obtain the multi-frame image, wherein the multi-frame American pupil maps comprise at least two different American pupil maps.

11. The apparatus of claim 8, wherein the adding means comprises:

a first determination unit configured to determine a region where a pupil of a target object is located in a current target image;

12. The apparatus according to claim 8, wherein the adding the determined cosmetic pupil image to be added to a pupil area of a target object in a current target image of a target video to obtain a current target image with a cosmetic pupil added comprises:

13. The apparatus according to claim 12, wherein the second determining unit is configured to identify eye keypoints for determining an eye of a target object in the current target image; and means for determining eye closure of the target object in the current target image based on the identified eye keypoints.

14. The apparatus of claim 8, wherein the apparatus further comprises:

15. An electronic device, comprising:

at least one processor; and

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-7.

17. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-7.