WO2021047433A1

WO2021047433A1 - Image processing method and system in live streaming

Info

Publication number: WO2021047433A1
Application number: PCT/CN2020/112970
Authority: WO
Inventors: 王云
Original assignee: 广州华多网络科技有限公司
Priority date: 2019-09-10
Filing date: 2020-09-02
Publication date: 2021-03-18
Also published as: CN110490828A; CN110490828B

Abstract

An image processing method and system in live streaming. The method comprises: extracting a video image from a live video; recognizing a target area of the video image to obtain first image contour information, and shaping the video image according to the first image contour information to generate a first image; performing secondary recognition on the target area in the first image to obtain second image contour information of the first image, and superposing a beauty makeup sticker on the first image according to the second image contour information to generate a second image; and replacing the video image in the live video with the second image to obtain a target live video, and outputting the target live video. According to the method above, the secondary recognition of the target area is performed after shaping, the second image contour information obtained after the secondary recognition can accurately describe the contour in the shaped image, and image contour matching between the beauty makeup sticker and the first image is performed according to the second image contour information, thereby improving the matching effect between shaping and beauty makeup special effects in the live video.

Description

Image processing method and system in live video broadcast

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on September 10, 2019, the application number is 201910854350.7, and the invention title is "Image Processing Method and System in Live Video Broadcasting", the entire content of which is incorporated herein by reference. Applying.

Technical field

This application relates to the technical field of image processing. Specifically, this application relates to an image processing method in live video broadcasting, an image processing system in live video broadcasting, computer equipment, storage media, and terminals.

Background technique

In image processing, the typical way of realizing special effects for beauty makeup is mainly to paste pre-designed translucent special effects pictures on corresponding parts of the image.

With the increasing progress in the pursuit of beauty in live video, a better beauty effect can be obtained through the superimposed combination of multiple beauty image processing. For example, after the person in the image is processed with beauty special effects, the image will continue to be reshaped Processing to further optimize the beauty effect of the characters in the image. Among them, in image processing, the way of realizing the special effects of beauty is mainly to paste pre-designed translucent special effect pictures on corresponding parts of the image.

However, after the image processing is performed with the special effects of beauty makeup and plastic surgery, the image processing is superimposed in sequence, and obvious strange graphics often appear, and the matching effect of image processing between beauty special effects and plastic surgery is poor.

Summary of the invention

Based on this, it is necessary to provide an image processing method, system, computer equipment, storage medium, and terminal in live video broadcasting in response to the above technical shortcomings, especially the poor matching effect of image processing between beauty special effects and plastic surgery.

An image processing method in live video broadcast includes the following steps:

Extract video images from live video;

Recognizing the target area of the video image to obtain first image contour information, and shaping the video image according to the first image contour information to generate a first image;

Perform secondary recognition on the target area in the first image to obtain second image contour information of the first image, and superimpose a beauty map on the first image according to the second image contour information And generate a second image;

The second image is used to replace the video image in the live video to obtain and output the target live video.

In an embodiment, the step of extracting a video image from a live video includes:

Obtain the live video, and determine whether the video image in the live video has been subjected to texture processing; if so, determine whether the target area of the texture processing and the target area to be reshaped are overlapped; if they overlap, call the pre-texture And extract the video image from the original live video.

In one embodiment, the video image is an image containing a human face, the target area includes a human face area, and the image contour information is a human face feature point;

The step of recognizing the target area of the video image to obtain first image contour information, and shaping the video image according to the first image contour information to generate the first image includes:

Identifying the face area in the video image, detecting the face area and obtaining a first face feature point; determining a plastic part corresponding to the plastic type from the face area according to the first face feature point; The contour of the plastic part is adjusted to obtain the primary beauty image.

In an embodiment, the step of adjusting the contour of the shaping part includes:

Extracting the current contour of the shaping object according to the first facial feature points; adjusting the current contour to the shaping contour corresponding to the shaping type;

The second recognition of the target area in the first image is performed to obtain second image contour information of the first image, and a beauty texture is superimposed on the first image according to the second image contour information. The steps of generating a second image on the image include:

Detect the reshaped face area to obtain a second face feature point; retrieve a texture image matching the second face feature point; fuse the texture image in the target area to make the texture image The contour coincides with the shaping contour.

In an embodiment, after the step of superimposing a beauty texture map on the first image according to the contour information of the second image, the method further includes:

According to the contour information of the second image, the skin area for whitening and dermabrasion in the first image is identified, and the dermabrasion area in the first image on which the beauty map is superimposed is subjected to whitening and dermabrasion to generate the second image. image.

In one embodiment, the plastic surgery includes any one or more of face thinning, nose reduction, lip augmentation, eye enlargement, plumping apple muscles, and smiling lips.

In one embodiment, the texture includes any one or more of foundation, nose shadow, lips, eyebrows, eye shadow, cosmetic contact lenses, silkworm, and blush.

An image processing system in live video broadcasting, including:

Extraction module for extracting video images from live video;

A shaping module, configured to identify the target area of the video image, obtain first image contour information, and shape the video image according to the first image contour information to generate a first image;

The mapping module is used to perform secondary recognition on the target area in the first image, obtain second image contour information of the first image, and superimpose the beauty texture on the first image according to the second image contour information. On the first image and generating a second image;

The video module is used to replace the video image in the live video with the second image to obtain and output the target live video.

A computer device, including a memory, a processor, and a computer program stored on the memory and capable of running on the processor. The processor executes the computer program to implement the image processing method in the live video broadcast of any one of the above embodiments A step of.

A computer-readable storage medium has a computer program stored thereon, and when the computer program is executed by a processor, the steps of the image processing method in the live video broadcast of any one of the above embodiments are realized.

A terminal, which includes:

One or more processors;

Memory

One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to: execute The image processing method in the live video broadcast described in any of the above embodiments.

The above-mentioned image processing method, system, computer equipment, storage medium and terminal in the live video broadcast extract the video image in the live video, and after the target area of the video image is recognized for the first time, the video image is reshaped, and after the reshaping, the second target area is performed. Second recognition, the second image contour information obtained after the second recognition can accurately describe the contour in the reshaped image. According to the second image contour information, the beauty texture matching with the first image can be processed, and the beauty texture is similar to the first image. The contour matching of the image, avoiding the appearance of unmatched strange graphics near the contour, and improving the coordination effect between the plastic surgery and the beauty special effects in the live video.

The additional aspects and advantages of the present application will be partly given in the following description, which will become obvious from the following description, or be learned through practice.

Description of the drawings

The above and/or additional aspects and advantages will become obvious and easy to understand from the following description of the embodiments in conjunction with the accompanying drawings, in which:

FIG. 1 is an implementation environment diagram of an image processing method in a live video broadcast provided in an embodiment;

Fig. 2 is a flowchart of an image processing method in live video broadcasting in an embodiment;

FIG. 3 is an effect diagram of a beauty treatment effect of applying images first and then shaping in an embodiment;

Fig. 4 is a flowchart of an image processing method in a live video broadcast in another embodiment;

FIG. 5 is a schematic diagram of the principle of facial feature points in an embodiment;

Fig. 6 is a schematic structural diagram of an image processing system in a live video broadcast in an embodiment;

Figure 7 is a schematic diagram of the internal structure of a computer device in an embodiment;

Fig. 8 is a schematic diagram of the internal structure of a terminal in an embodiment.

detailed description

The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, in which the same or similar reference numerals denote the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the drawings are exemplary, and are only used to explain the present application, and cannot be construed as a limitation to the present application.

Those skilled in the art can understand that, unless specifically stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in this specification refers to the presence of the described features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, integers, Steps, operations, elements, components, and/or groups of them. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or intervening elements may also be present. In addition, “connected” or “coupled” used herein may include wireless connection or wireless coupling. The term "and/or" as used herein includes all or any unit and all combinations of one or more of the associated listed items.

Those skilled in the art can understand that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meanings as those commonly understood by those of ordinary skill in the art. It should also be understood that terms such as those defined in general dictionaries should be understood to have a meaning consistent with the meaning in the context of the prior art, and unless specifically defined as here, they will not be idealized or overly Explain the formal meaning.

As shown in FIG. 1, FIG. 1 is an implementation environment diagram of an image beauty processing method provided in an embodiment. The implementation environment includes a host 110, a live broadcast platform 120, and an audience 130. The host captures live video through the host 110 camera or captures the host screen, and uploads the live video to the live room 121 of the live broadcast platform 120. The live broadcast platform 120 can report to the audience according to the viewing needs of users in the live room 121 130 transmits the live video of the live room 121.

It should be noted that the anchor terminal 110 or the audience terminal 130 can be installed on a smart phone, a tablet computer, a notebook computer, or a desktop computer. The live broadcast platform 120 may run on a computer device, a server device, or a server device group. The client 110 and the live broadcast platform 120 and between the audience 130 and the live broadcast platform 120 can be connected via a network, which is not limited in this application.

In an embodiment, as shown in FIG. 2, FIG. 2 is a flowchart of an image beauty processing method in an embodiment. In this embodiment, an image beauty processing method is proposed. The method can be applied to the host 110 or the live broadcast platform 120 described above, and the processor in the host 110 or the live broadcast platform 120 executes the steps of the image beauty processing method. The host 110 can collect live video and perform beauty processing on the images in the collected video, or the live broadcast platform 120 can perform beauty processing on the images in the live video after receiving the live video uploaded by the host 110 . The image beauty processing method may specifically include the following steps:

Step S210: Extract a video image from the live video.

In this step, the live video contains multiple picture frames, and the processor can retrieve the picture frames of the live video as video images.

Step S220: Recognizing the target area of the video image to obtain first image contour information, and shaping the video image according to the first image contour information to generate a first image.

In this step, the processor may have the function of identifying a specific target area in the video image. The processor may detect image features from the video image, identify the target area based on the image features, and generate first image contour information. The image contour information can be used to record image features, and the image contour information can also characterize contour lines in the image. According to the first image contour information, the processor can perform image processing on the pixels in the target area part of the video image, adjust the contour lines specified in the target area according to the shaping characteristics, and display the contour lines in the target area by adjusting the contour lines in the target area. effect.

The video image can be an image that includes the face and/or the torso of the body. The target area in the video image may be a face area or a human body area, such as a human face, torso, limbs, and other parts.

Taking a video image containing a human face as an example, in one embodiment, the video image is an image containing a human face, the target area includes a human face area, and the image contour information is a human face feature point. The processor detects the facial feature points of the video image, and detects the facial feature points in the original image. For example, the processor may call a face recognition algorithm to perform face detection and output the coordinates of 106 facial feature points. The shaping of the face area can include any one or more types of face thinning, nose reduction, lip augmentation, enlarged eyes, plump apple muscle, and smiling lips. The texture of the face area may include any one or more of makeup special effects from foundation, nose shadow, lips, eyebrows, eye shadow, cosmetic contact lenses, silkworm, and blush.

In addition, the processor can also identify the limbs and torso of the human body in the video image, and use them as the target area to be processed. For example, if there is a leg area in a video image, the processor can detect the contour lines in the video image, and record the detected contour lines through the image contour information; the processor can identify the characteristic areas of the legs according to the contour lines and determine the video image The leg area in the middle; image processing of the leg area to change the contour lines of the leg, the leg in this area realizes the shaping of the thin leg or the stretched leg, or the shaping of the leg to achieve thick or fattening. Moreover, the processor can also have the function of detecting the skin area in the video image, such as filtering out the smooth skin area in the image with the help of a filter, identifying the contour lines of the skin area, and further determining whether the skin area is based on the contour lines. It is the leg area, which is conducive to accurately identifying the leg area.

In the image processing of the pixels in the video image to realize the adjustment of the contour lines, take the face-lift shaping acting on the chin contour as an example, the target area is the face area, and the chin part to be reshaped in the face area can be identified through the image contour information. The image pixels of the chin part will be "squeezed" from the two sides to the middle chin contour, especially the closer the chin contour is, the stronger the squeezing will be, and finally the chin contour will be adjusted to the target contour of the shaping.

Step S230: Perform secondary recognition on the target area in the first image to obtain second image contour information of the first image, and superimpose the beauty texture on the first image according to the second image contour information to generate a second image.

In this step, the second image contour information of the first image is extracted, the target area is recognized again, and the reshaped target area is updated. The recognition mode of the secondary recognition in this step can be the same as the recognition mode in step S220. The processor can detect image features from the first image, recognize the target area based on the image features, and generate second image contour information. The processor can select the beauty texture map matching the target area according to the contour information of the second image, and determine the accurate area to be superimposed, superimpose the beauty texture map on the first image and generate a second image, thereby obtaining accurate beauty makeup effects. The second image together.

Step S240: Use the second image to replace the video image in the live video to obtain and output the target live video.

In this step, the video image of the live video is updated to the processed second image, the reshaping of the live video and the processing of the beauty texture map are realized, and the target live video is obtained and output.

The image processing method in the above video live broadcast extracts the video image in the live video, performs the video image shaping after the first recognition of the target area of the video image, and then performs the second recognition of the target area after the reshaping, and the second recognition is obtained after the second recognition. The contour information of the image can accurately describe the contour in the reshaped image. According to the contour information of the second image, the beauty texture matching the first image can be processed. The beauty texture matches the contour of the first image to avoid mismatches near the contour. The bizarre graphics enhance the coordination effect between plastic surgery and beauty special effects in live video.

In an embodiment, the step of extracting a video image from a live video may include:

Step S241: Obtain the live video, and determine whether the video image in the live video has been subjected to texture processing; if so, determine whether the target area of the texture processing overlaps with the target area to be shaped.

Step S242: If they overlap, retrieve the live video before the texture, and obtain the video image from the original live video.

In this step, if the live video has been subjected to texture processing, and there is an overlap between the texture area and the target area to be reshaped, it is necessary to retrieve the original live video before texture, and retrieve the picture frame in the original live image as the video image .

The original live video may refer to the original video captured by the host's camera or captured by the host's screen. The original live video may also be the live video that was initially uploaded to the live broadcast platform by the host. At this time, the host does not perform image processing on the original live video.

The image processing method in the above video live broadcast determines whether the video image in the live video has been subjected to texture processing, and whether there is an overlap between the texture processed area and the shaping target area. If there is an overlap, it needs to be recalled without texture processing. The processed original image can prevent the mismatched area between the original texture and the contour before the reshaping from being enlarged after the reshaping, thereby improving the matching effect between the reshaping and the beauty special effects.

Take the target area in face-lift plastic surgery as an example to illustrate the overlap of the target area. For example, there is a clear overlap between the target area of the foundation texture and the target area of face-lifting. In addition, the target area of face-lift plastic surgery is not only in the chin area, but the target area of face-lift plastic surgery also affects the change of lip shape and position. Therefore, the target area of face-lift plastic surgery also overlaps with the target area of lip texture.

The foregoing embodiments describe the process of extracting video images. The following embodiments will take the shaping of the face area as an example to illustrate the shaping of the face area.

In one embodiment, the video image is an image containing a human face, the target area includes a human face area, and the image contour information is a human face feature point.

In step S220, the step of identifying the target area of the video image to obtain first image contour information, and shaping the video image according to the first image contour information to generate the first image may include:

Step S221: Identify the face area in the video image, detect the face area, and obtain the first face feature point.

The face area of the video image is recognized, and the face feature points are detected on the face area to obtain the first face feature point. For example, the processor may call a face recognition algorithm to perform face detection and output the coordinates of 106 facial feature points.

Step S222: Determine a plastic part corresponding to the plastic type from the face area according to the first facial feature point.

According to the relationship between the facial feature points and the plastic parts corresponding to the plastic surgery type, the region to be shaped can be determined according to the first facial feature points.

The type of plastic surgery for the face area may include any one or more types of face-lifting, reduced nose, enlarged lips, enlarged eyes, plumped apple muscles, and smiling lips. When performing face-lifting, the chin area can be determined according to the contour of the facial feature points in the chin area, and then the chin area in the original image can be reshaped. In addition, the contours of various parts can be determined with the help of facial feature points. For example, facial feature points can determine the contours of the face shape, nose shape, lip shape, and eye shape.

Step S223: Adjust the contour of the plastic part to obtain a primary beauty image.

The image processing method in the above-mentioned live video broadcast recognizes the plastic parts corresponding to the plastic type, and adjusts the contours of the plastic parts to realize the plastic parts in the target area.

The adjustment of the contour of the reshaping part can be realized by means of image processing. The contour of the reshaping part can be adjusted by locally "squeezing" and locally "stretching" the image pixels of the shaping part. Take the face-lifting type as an example, the image pixels of the chin part will be "squeezed" from the two sides to the middle chin contour, and the image pixels outside the chin part will be "stretched" outwards, especially the area closer to the chin contour The stronger the squeezing strength, the final adjustment of the chin contour to the target contour of the shaping, the stronger the stretching of the area closer to the chin contour, and the outward stretching can avoid obvious deformation in other areas of the image.

It is precisely because the closer the area is to the contour, the stronger the intensity of the deformation in image processing. In the operation of mapping first and then shaping, the gap between the edge of the texture and the contour will be enlarged when deformed, which leads to obvious strange shapes near the contours of the characters. As shown in Fig. 3, Fig. 3 is an effect drawing of the beauty treatment after applying the texture first and then shaping. The foundation texture is to cover the entire face with a layer of translucent face skin, the foundation texture of the chin and the original chin The gap between the contours is stretched and enlarged under the face-lifting, and weird graphics appear at the contour of the chin, as indicated by the arrow in Figure 3. At this time, the matching effect of the image processing between the foundation effect and the face-lifting is poor.

In an embodiment, the step of adjusting the contour of the shaping part in step S223 may include:

Step S2231: Extract the current contour of the plastic object according to the first face feature point; Step S2232: Adjust the current contour to the plastic contour corresponding to the plastic type.

In this step, the current contour in the original image is adjusted to the plastic contour to obtain a primary beauty image.

In step S230, the target area in the first image is identified twice to obtain second image contour information of the first image, and a beauty texture is superimposed on the first image according to the second image contour information. The step of generating a second image on one image may include:

Step S231: Detect the face area after shaping to obtain a second face feature point;

In this step, the second face recognition is performed to obtain the second face feature points in the primary beauty image.

Step S232: Retrieve a texture image matching the second face feature point; Step S233: Fusion the texture image in the target area, so that the contour of the texture image coincides with the shaping contour.

The second face feature points after the secondary face recognition are fitted to the plastic contour, so that the contour of the texture acquired according to the second facial feature coincides with the plastic contour.

The above-mentioned image processing method in the live video broadcast can make the texture image and the plastic contour overlap, and improve the matching effect of the image processing between the special effects of the beauty makeup and the plastic surgery.

In an embodiment, after the step of superimposing a beauty texture map on the first image according to the second image contour information, the method may further include:

Step S260: Recognizing the skin-whitening skin area in the first image according to the contour information of the second image, and performing the skin-whitening skin skinning area on the first image with the superimposed beauty texture map to generate a second image.

In the above-mentioned live video image processing method, after the beauty mapping, the image contour information in the second image has not changed. The dermabrasion area identified by continuing to use the second image contour information is also accurate, and the dermabrasion area can be accelerated at this time Recognition efficiency.

In an implementation example, take thin face and foundation texture as examples, as shown in Figures 4 and 5. Figure 4 is a flowchart of an image processing method in live video in another embodiment, and Figure 5 is a face in an embodiment Schematic diagram of the principle of feature points. The image processing method in the live video broadcast provided by this embodiment example includes: collecting live video through a USB external camera device, extracting video images from the live video; performing the first face recognition on the video image, and detecting the first facial feature point, As shown by the black feature points in Figure 5, the face area is determined by face recognition, and after the first face recognition, face-lifting of the face area is performed to obtain the first image; then the second face recognition is performed, Detect the second face feature points on the first image, as shown by the white feature points in Figure 5, to obtain more accurate face feature points after shaping, re-determine the face area for the first image, and according to the second face Recognize and apply the beauty texture to obtain the target beauty image, and enhance the fit effect of the beauty texture on the edge. The foundation texture will not be stretched and deformed during the face-lifting process, avoiding the gap between the foundation texture and the original chin contour. It is magnified under face-lifting and shaping to improve the coordination effect of image processing between foundation special effects and face-lifting. After the beauty stickers are applied, you can continue to perform whitening and dermabrasion on the target beauty image based on the second face recognition to speed up the recognition efficiency of the dermabrasion area.

In an embodiment, as shown in FIG. 6, FIG. 6 is a schematic structural diagram of an image processing system in a live video broadcast in an embodiment. In this embodiment, an image beauty processing system is provided, which may specifically include an extraction module 610, The shaping module 620, the texture module 630 and the video module 640, of which:

The extraction module 610 is used to extract video images from the live video.

In the extraction module 610, the live video contains multiple picture frames, and the processor can retrieve the picture frames of the live video as video images.

The shaping module 620 is configured to identify the target area of the video image, obtain first image contour information, and shape the video image according to the first image contour information to generate a first image.

The processor may have a function of identifying a specific target area in the video image, and the processor may detect image features from the video image, identify the target area based on the image features, and generate first image contour information. The image contour information can be used to record image features, and the image contour information can also characterize contour lines in the image. According to the first image contour information, the processor can perform image processing on the pixels in the target area part of the video image, adjust the contour lines specified in the target area according to the shaping characteristics, and display the contour lines in the target area by adjusting the contour lines in the target area. effect.

The video image can be an image that includes the face and/or the torso of the body. The target area in the video image can be the face area or the human body area, such as the human face, torso, limbs and other parts of the area.

The mapping module 630 is configured to perform secondary recognition on the target area in the first image, obtain second image contour information of the first image, and superimpose the beauty texture on the second image contour information according to the second image contour information. And generate a second image on the first image.

In the mapping module 630, the contour information of the second image of the first image is extracted, the target area is recognized again, and the reshaped target area is updated. The recognition method of the secondary recognition in this step can be the same as the recognition method in step S220. The processor can detect image features from the first image, identify the target area based on the image features, and generate second image contour information. The processor can select the beauty texture map matching the target area according to the contour information of the second image, and determine the accurate area to be superimposed, superimpose the beauty texture map on the first image and generate a second image, thereby obtaining accurate beauty makeup effects. The second image together.

The video module 640 is configured to replace the video image in the live video with the second image to obtain and output the target live video.

In the video module 640, the video image of the live video is updated to the processed second image, the reshaping of the live video and the beauty mapping process are realized, and the target live video is obtained and output.

The image processing system in the above video live broadcast extracts the video image in the live video, reshapes the video image after the first recognition of the target area of the video image, and then performs the second recognition of the target area after the reshaping, and the second recognition is obtained after the second recognition. The contour information of the image can accurately describe the contour in the reshaped image. According to the contour information of the second image, the beauty texture matching the first image can be processed. The beauty texture matches the contour of the first image to avoid mismatches near the contour. The bizarre graphics enhance the coordination effect between plastic surgery and beauty special effects in live video.

For the specific definition of the image processing system in the live video broadcast, please refer to the above definition of the image processing method in the live video broadcast, which will not be repeated here. The various modules in the image processing system in the above-mentioned live video broadcast can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

As shown in FIG. 7, FIG. 7 is a schematic diagram of the internal structure of a computer device in an embodiment. The computer device includes a processor, a non-volatile storage medium, a memory, and a network interface connected through a system bus. Among them, the non-volatile storage medium of the computer device stores an operating system, a database, and a computer program. The database may store control information sequences. When the computer program is executed by the processor, the processor can realize a live video broadcast. Image processing method. The processor of the computer equipment is used to provide computing and control capabilities, and supports the operation of the entire computer equipment. A computer program can be stored in the memory of the computer device, and when the computer program is executed by the processor, the processor can execute an image processing method in live video broadcasting. The network interface of the computer device is used to connect and communicate with the terminal. Those skilled in the art can understand that the structure shown in FIG. 7 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may Including more or fewer parts than shown in the figure, or combining some parts, or having a different arrangement of parts.

In one embodiment, a computer device is proposed. The computer device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, the video of any of the above embodiments is implemented. The steps of the image processing method in the live broadcast.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the image processing method in the live video broadcast of any of the above embodiments are realized.

In one embodiment, a terminal is provided, which includes: one or more processors; a memory; one or more application programs, wherein one or more application programs are stored in the memory and configured to be operated by one or more Multiple processors are executed, and one or more programs are configured to execute the image processing method in the live video broadcast according to any one of the foregoing embodiments.

An embodiment of the present application also provides a mobile terminal, as shown in FIG. 8, which is a schematic diagram of the internal structure of the terminal in an embodiment. For ease of description, only the parts related to the embodiments of the present application are shown. For specific technical details that are not disclosed, please refer to the method part of the embodiments of the present application. The terminal can be any terminal device including a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales, sales terminal), a car computer, etc. Take the terminal as a mobile phone as an example:

FIG. 8 shows a block diagram of a part of the structure of a mobile phone related to a terminal provided in an embodiment of the present application. Referring to FIG. 8, the mobile phone includes: a radio frequency (RF) circuit 810, a memory 820, an input unit 830, a display unit 840, a sensor 850, an audio circuit 860, a wireless fidelity (Wi-Fi) module 870, a processing 880, power supply 890 and other components. Those skilled in the art can understand that the structure of the mobile phone shown in FIG. 8 does not constitute a limitation on the mobile phone, and may include more or less components than those shown in the figure, or a combination of some components, or different component arrangements.

In the embodiment of the present application, the processor 880 included in the terminal also has the following functions: extracting video images from live video; identifying the target area of the video image to obtain first image contour information, and according to the first image An image contour information reshapes the video image to generate a first image; performs secondary recognition on the target area in the first image to obtain the second image contour information of the first image, according to the The second image contour information superimposes the beauty texture map on the first image to generate a second image; replace the video image in the live video with the second image to obtain and output the target live video. That is, the processor 880 has the function of executing the image processing method in the live video broadcast of any of the foregoing embodiments, and details are not described herein again.

It should be understood that although the various steps in the flowchart of the drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless explicitly stated in this article, the execution of these steps is not strictly limited in order, and they can be executed in other orders. Moreover, at least part of the steps in the flowchart of the drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the order of execution is also It is not necessarily performed sequentially, but may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

The above are only part of the implementation of this application. It should be pointed out that for those of ordinary skill in the art, without departing from the principles of this application, several improvements and modifications can be made, and these improvements and modifications are also Should be regarded as the scope of protection of this application.

Claims

An image processing method in live video broadcast, characterized in that it comprises the following steps:

Extract video images from live video;

Recognizing the target area of the video image to obtain first image contour information, and shaping the video image according to the first image contour information to generate a first image;

Perform secondary recognition on the target area in the first image to obtain second image contour information of the first image, and superimpose a beauty map on the first image according to the second image contour information And generate a second image;

The second image is used to replace the video image in the live video to obtain and output the target live video.
The method for image processing in a live video broadcast according to claim 1, wherein the step of extracting a video image from the live video comprises:

Obtain the live video, and determine whether the video image in the live video has been subjected to texture processing; if so, determine whether the target area of the texture processing overlaps with the target area to be reshaped;

If they overlap, the original live video before the texture is retrieved, and the video image is extracted from the original live video.
The method for image processing in live video broadcast according to claim 1, wherein the video image is an image containing a human face, the target area includes a human face area, and the image contour information is a human face feature point;

The step of recognizing the target area of the video image to obtain first image contour information, and shaping the video image according to the first image contour information to generate the first image includes:

Identifying the face area in the video image, detecting the face area, and obtaining a first face feature point;

Determining a plastic part corresponding to a plastic plastic type from the human face area according to the first facial feature point;

The contour of the plastic part is adjusted to obtain the primary beauty image.
The image processing method in live video broadcast according to claim 3, wherein the step of adjusting the contour of the shaping part comprises:

Extracting the current contour of the shaping object according to the first face feature point;

Adjusting the current contour to a shaping contour corresponding to the shaping type;

The second recognition of the target area in the first image is performed to obtain second image contour information of the first image, and a beauty texture is superimposed on the first image according to the second image contour information. The steps of generating a second image on the image include:

Detect the face area after shaping to obtain the second face feature point;

Retrieve a texture image matching the second face feature point;

The texture image is merged in the target area so that the contour of the texture image coincides with the shaping contour.
The method for image processing in live video broadcasting according to claim 1, wherein after the step of superimposing a beauty texture map on the first image according to the second image contour information, the method further comprises:

According to the contour information of the second image, the skin area for whitening and dermabrasion in the first image is identified, and the dermabrasion area in the first image on which the beauty map is superimposed is subjected to whitening and dermabrasion to generate the second image. image.
The image processing method in live video according to claim 3, wherein the shaping includes any one or more of face-lifting, nasal reduction, lip augmentation, enlarged eyes, full apple muscle, smiling lips, and/or The texture includes any one or more of foundation, nose shadow, lips, eyebrows, eye shadow, cosmetic contact lenses, silkworm, and blush.
An image processing system in live video broadcasting, which is characterized in that it comprises:

Extraction module for extracting video images from live video;

A shaping module, configured to identify the target area of the video image, obtain first image contour information, and shape the video image according to the first image contour information to generate a first image;

The mapping module is used to perform secondary recognition on the target area in the first image, obtain second image contour information of the first image, and superimpose the beauty texture on the first image according to the second image contour information. On the first image and generating a second image;