CN115908185A

CN115908185A - Processing method and processing device

Info

Publication number: CN115908185A
Application number: CN202211524797.6A
Authority: CN
Inventors: 周梦涵; 于海峰
Original assignee: Lenovo Beijing Ltd
Current assignee: Lenovo Beijing Ltd
Priority date: 2022-11-30
Filing date: 2022-11-30
Publication date: 2023-04-04

Abstract

The embodiment of the application provides a processing method and a processing device, wherein the method comprises the following steps: obtaining a first frame image, wherein the first frame image comprises a target object; obtaining a first input track; determining a target contour of the target object based on the first input trajectory, wherein the target contour of the target object matches a path formed by the first input trajectory; determining a second contour according to the first input track; and adjusting the target contour to be the second contour.

Description

Processing method and processing device

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a processing method and a processing apparatus.

Background

Some users need to beautify through the beautifying function when taking a picture or broadcasting on the internet. At present, a user can perform corresponding beautifying processing by setting corresponding grades of beauty or beauty types, but preset beauty or beauty types cannot be adjusted according to actual feature conditions of the user, and cannot meet the beauty effect of the user on processing specific features, so that the actual effect of shot photos or videos cannot meet the use requirements of the user.

Disclosure of Invention

In one aspect, an embodiment of the present application provides a processing method, where the method includes:

obtaining a first frame image, wherein the first frame image comprises a target object;

obtaining a first input track;

determining a target contour of the target object based on the first input trajectory, wherein the target contour of the target object matches a path formed by the first input trajectory;

determining a second contour according to the first input track;

and adjusting the target contour to be the second contour.

In some embodiments, said determining a second contour from said first input trajectory comprises:

determining the second contour according to the first input track and a first condition, wherein the first condition is used for restricting the bending degree of the second contour.

In some embodiments, said determining a target contour of said target object based on said first input trajectory comprises:

determining a first contour of the target object, wherein the first contour comprises the target contour;

and determining the target contour from the first contour according to the distance relation between the first contour and the path of the first input track.

In some embodiments, the method further comprises:

displaying the first input track;

canceling the display of the first input trajectory after the target contour is adjusted to the second contour for a preset time period.

In some embodiments, prior to determining the target contour of the target object based on the first input trajectory, the method further comprises:

obtaining a second input track;

determining a target area from the first frame image according to the second input track, wherein the target contour is located in the target area.

In some embodiments, after adjusting the target contour to the second contour, the method further comprises:

obtaining a first adjusting parameter according to the second contour;

and processing the target object according to the first adjusting parameter.

In some embodiments, the first frame image is an image in a target video stream, the method further comprising:

obtaining a second adjusting parameter according to the second contour;

acquiring a second frame image from the target video stream;

and processing the second frame image according to the second adjusting parameter.

In some embodiments, the second adjustment parameter includes first key point information corresponding to the target object; the processing the second frame image according to the second adjustment parameter includes:

determining a second object corresponding to the target object from the second frame image;

determining second key point information corresponding to the first key point information in the second object;

and processing the second frame image according to the second key point information and the second adjusting parameter.

determining a second input track symmetrical to the first input track according to the first axis;

determining the target contour of the target object according to the first input trajectory and the second input trajectory.

On the other hand, an embodiment of the present application further provides a processing apparatus, including:

a first obtaining module configured to obtain a first frame image, the first frame image including a target object;

a second obtaining module configured to obtain a first input trajectory;

a first determination module configured to determine a target contour of the target object based on the first input trajectory, wherein the target contour of the target object matches a path formed by the first input trajectory;

a second determination module configured to determine a second contour from the first input trajectory;

an adjustment module configured to adjust the target profile to the second profile.

The embodiment of the present application further provides an electronic device, which at least includes a memory, a processor and a bus, where the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the processor and the memory communicate with each other through the bus, and when the machine-readable instructions are executed by the processor, the steps of the processing method provided in any embodiment of the present application are performed.

The embodiment of the present application further provides a storage medium, where one or more programs are stored, and when the one or more programs are executed by a processor, the steps of the processing method provided in any of the above embodiments of the present application are implemented.

Drawings

In order to more clearly illustrate the embodiments of the present disclosure or technical solutions in related arts, the drawings used in the description of the embodiments or related arts will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present disclosure, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

Fig. 1 is a flowchart of a processing method provided in an embodiment of the present application;

FIG. 2 is a block diagram of a processing device provided in an embodiment of the present application;

fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings of the embodiments of the present application. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the described embodiments of the application without any inventive step, are within the scope of protection of the application.

Unless defined otherwise, technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application belongs. As used in this application, the terms "first," "second," and the like do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", and the like are used merely to indicate relative positional relationships, and when the absolute position of the object being described is changed, the relative positional relationships may also be changed accordingly.

Detailed descriptions of known functions and known components are omitted in the present application in order to keep the following description of the embodiments of the present application clear and concise.

When shooting or live broadcasting, the user can adjust the shooting effect by adjusting the beauty grade. However, the beauty level is adjusted based on a preset adjustment strategy, and cannot be adjusted correspondingly for the portion actually required to be adjusted by each user, and the requirements of different users cannot be flexibly met, so that the shooting effect of the photos or videos is not ideal. Based on this, the application provides a processing method, by obtaining the shot first frame image and the first input track of the user, the target contour that the user wants to adjust for the target object can be determined according to the first input track, and the target contour is adjusted to the second contour determined according to the first input track, so that the corresponding adjustment can be performed according to the input of the user, the accuracy of beautifying and adjusting the target object is improved, the effect of the shot picture or video can be improved according to the actual needs of the user, and the user experience is good.

The processing method provided by the embodiment of the present application may be applied to a client of an electronic device, where the electronic device may include, but is not limited to, a mobile phone, a tablet, a smart watch, a music playing device, a laptop, a notebook, a desktop computer, a monitoring device, and the like, and the client may be an application capable of taking a photo or a video, for example, at least one of a camera application, a live application, or an instant messaging application.

Fig. 1 shows a flowchart of a processing method provided in an embodiment of the present application. As shown in fig. 1, the processing method provided by the present application includes the following steps S100-S500.

S100, a first frame image is obtained, and the first frame image comprises a target object.

In this step, the target object may be determined according to a portion that needs to be adjusted by the user, for example, the target object may be a main shooting object, or an eye, a nose, and the like of the shooting object, where the user needs to adjust, which is not limited in this embodiment of the present application. In some practical applications, the electronic device may select based on a locally stored picture or video to obtain a first frame image including the target object; of course, the first frame image including the target object may also be acquired by previewing through a camera of the electronic device, and the method for acquiring the first frame image is not limited in the present application.

And S200, obtaining a first input track.

In this step, the first input trajectory is a trajectory formed by the user operating on the target object, for example, when the target object is an eye of a shooting object, the user may perform an input operation on the shape of the eye according to the need of self-adjustment to form the first input trajectory. In some practical applications, a user may perform a touch input operation on a touch display screen of the electronic device for a target object to form a first input track, or perform an input operation on the target object by using a mouse or the like of the electronic device, and the like.

S300, determining a target contour of the target object based on the first input track, wherein the target contour of the target object is matched with a path formed by the first input track.

In this step, the electronic device may identify the target object based on the first input trajectory, and determine a target contour corresponding to the target object. Since the first input track is formed by the user performing an input operation on the target object, and the first input track corresponds to the target object, a path formed by the first input track matches a target contour of the target object. In some specific applications, the electronic device may identify and confirm the target object according to a path formed by the first input trajectory, and obtain a target contour of the target object. For example, when the target object is an eye of the shooting object, after the user performs an input operation on the eye, the electronic device may record a path formed by the first input track, and identify the contour of the eye based on the association between the path and the eye, so as to obtain the target contour of the eye.

In some embodiments, the step S300 may be implemented as:

In this embodiment, the shape formed by the first contour may include a target contour, and for example, in the case where the target object is an eye, in order to determine a target contour corresponding to the eye, the electronic device may construct the entire region including the eyebrow and the eye as the first contour based on the eye. In this way, after the first contour is determined, the electronic device may determine a corresponding position between the first contour and the path of the first input track according to the corresponding relationship between the first contour and the path of the first input track, so as to obtain a distance relationship between the first contour and the path of the first input track, and then determine a target contour of a target object for which the first input track of the user is directed according to the distance relationship between the first contour and the path of the first input track.

Further, the processing method of the present application further includes:

obtaining a second input track;

In this embodiment, the electronic device may determine, according to the second input trajectory input by the user, a target area that the user needs to adjust, so as to adjust the target contour based on the target area accurately in the following. In some practical applications, the manner of obtaining the second input track by the electronic device may refer to the manner of obtaining the first input track, and this embodiment is not described herein again. In this embodiment, after the user performs the second input operation based on the target object, the electronic device may obtain a second input track corresponding to the user input operation, determine a corresponding area from the first frame image as the target area according to a shape of the second input track, where a shape defined by the second input track includes a target contour of the target object. In an exemplary case where eyes are used as the target object, after determining the target contour corresponding to the eyes according to the first input track, the electronic device obtains the second input track through the input operation of the user to avoid mistaken adjustment of other areas, and surrounds the target contour of the eyes through the shape defined by the second input track, so that a target area for adjustment processing is determined on the first frame of image, and the target contour of the target object can be accurately adjusted in the following process.

In some embodiments, the step S300 may be further implemented as:

In some practical applications, in order to perform symmetry adjustment on the target object as a whole, the object may be detected by a face recognition technique, a first axis of the face of the object is determined, and thus symmetry adjustment operation may be performed based on the first axis. In this embodiment, when adjustment needs to be performed on a target object having symmetry on a face of a photographed object, after obtaining a first input trajectory, the electronic device may determine a second input trajectory that is symmetrical to the first input trajectory based on the first axis, so that a target contour of the target object may be determined according to the first input trajectory and the second input trajectory, so as to facilitate subsequent adjustment and processing of an image of the target object.

S400, determining a second contour according to the first input track.

This step is intended to determine a second contour to which the user needs to adjust for the target object based on the first input trajectory. In some specific applications, the electronic device may record the first input trajectory and obtain the corresponding second profile based on a predetermined algorithm. For example, after the electronic device records the first input track, the electronic device may use a preset algorithm to construct the first input track to determine the second contour of the target object.

In some embodiments, the step S400 may be implemented as:

In this embodiment, the first condition may be set according to the target object by using a corresponding algorithm, so as to obtain a second contour corresponding to the shape of the target object. For example, for an eye, the first condition may be an algorithm corresponding to the contour of the eye to constrain the degree of curvature of the second contour when determining the contour to construct the shape of the eye.

S500, adjusting the target contour to be the second contour.

In this step, the electronic device may adjust the target contour of the target object according to the determined second contour, so as to adjust the actual contour of the target object to the contour required by the user. In some specific applications, the electronic device may determine an adjustment parameter for adjusting the target contour according to the association between the second contour and the target contour, so as to adjust the target contour to the second contour; the method also can determine a plurality of key points according to the target contour of the target object, the key points can be selected and identified through face detection, then the adjustment parameters are determined according to the distance between the selected key points and the corresponding positions of the second contour, the key points are adjusted, and then the adjusted key points are constructed according to the second contour, so that the target object is beautified according to the needs of a user, and the effect of shot photos or videos is improved.

In some embodiments, in order to facilitate observation and comparison by a user, the first input track is displayed on a display screen of the electronic device; canceling the display of the first input trajectory after the target contour is adjusted to the second contour for a preset time period.

In this embodiment, the preset duration may be set according to the needs of the user, which is not limited in this application. In some practical applications, the electronic device may determine the display time according to a preset duration set by a user, record and display the first input track according to an input operation of the user, after the target contour of the target object is adjusted to the second contour, continue to display the first input track within the time range of the preset duration, so that the user can check and compare the second contour based on the first input track, and further enable the user to further adjust the second contour as required, and after the preset duration is over, the display of the first input track may be cancelled, so that the user can check the effect of the adjusted image.

In some embodiments, after adjusting the target profile to the second profile, the method further comprises:

obtaining a first adjusting parameter according to the second contour;

and processing the target object according to the first adjusting parameter.

The present embodiment is directed to processing the entire target object based on the adjustment processing of the target contour to the second contour. In this embodiment, after the target contour is the second contour, the corresponding first adjustment parameter is determined by using the algorithm corresponding to the target object according to the second contour, and the target object is adjusted by the first adjustment parameter, so that the overall effect of the target object is better. For example, in the case that the target object is an eye, after the target contour of the eye is adjusted to the second contour, a first adjustment parameter for the whole eye, for example, parameters such as a degree for an eye angle, a size of an eyeball, a brightness or contrast of a pupil, and the like, is determined according to the second contour and a preset algorithm for the eye, so that when the eye is zoomed in or zoomed out, a good shape of the eye is maintained, and a better effect is obtained when a picture or a video is taken.

obtaining a second adjustment parameter according to the second contour;

acquiring a second frame image from the target video stream;

In some practical applications, for example, when the method and the device are applied to a live broadcast application program, when a user acquires a video stream by using a camera of electronic equipment for live broadcast, the live broadcast video stream is a target video stream; or when the user carries out video call with the friend through the instant messaging application program, a call video stream is collected through a camera of the electronic equipment, and the call video stream is the target video stream. In this embodiment, the second adjustment parameter may be the same as or different from the first adjustment parameter. For example, the second adjustment parameter may be an adjustment parameter that is adjusted for a target contour of an object corresponding to the target object in the second frame image, where the second adjustment parameter is different from the first adjustment parameter; or the second adjustment parameter may be an adjustment parameter that is adjusted on the whole for an object corresponding to the target object in the second frame image, and in this case, the second adjustment parameter may be the same as the first adjustment parameter.

Schematically, the following describes two cases of the second adjustment parameter. When the second adjustment parameter is an adjustment parameter for adjusting the target contour of the target object in the second frame image, after the electronic device adjusts the target contour of the target object to the second contour in the first frame image, the second adjustment parameter may be determined according to a distance between the target contour of the target object and the second contour. Then, the electronic device obtains a second frame image played next from the target video stream, performs face detection on the second frame image according to a target object adjusted by a user for the first frame image, determines a second object corresponding to the target object from the second frame image, determines outline information corresponding to the second object, and adjusts the outline information of the second object of the second frame image by using a second adjustment parameter to process the second frame image, so that the processed image is propagated in real time through the video image.

When the second adjustment parameter is an adjustment parameter for adjusting the target object in the second frame image as a whole, the electronic device further processes the target object by using the first adjustment parameter after adjusting the target contour of the target object in the first frame image to the second contour. Therefore, in this embodiment, the electronic device may process the second frame image with the first adjustment parameter as the second adjustment parameter. In some specific applications, the electronic device obtains a second frame image played next from the target video stream, performs face detection on the second frame image according to a target object adjusted by a user for the first frame image, determines a second object corresponding to the target object from the second frame image, and directly adjusts the second object of the second frame image on the whole by using a second adjustment parameter to process the second frame image, so that the processed image is propagated in real time through the video image.

In the application, when the target object in the first frame image is adjusted, the electronic device may identify and adjust the key points of the target object by using a preset algorithm, and the key points may be selected according to the detail characteristics of the target object. Therefore, when the second adjustment parameter is obtained based on the second contour, the first key point information corresponding to the target object is included. In the embodiment of the present application, the second object corresponds to the target object, and for example, when the target object is an eye of the object to be photographed in the first frame image, the second object is an eye of the object to be photographed in the second frame image. In some specific applications, in order to process the second frame image, the electronic device may perform face detection on the second frame image based on the target object of the first frame image, and determine the second object corresponding to the target object. Then, the electronic device determines second keypoint information corresponding to the first keypoint information on the second object according to the corresponding relationship between the target object and the second object, so that the electronic device can adjust the outline of the second object according to the second keypoint information and a second adjustment parameter, or adjust the second object as a whole, so as to implement processing of a second frame of image.

For example, when the target object is an eye of the object to be photographed in the first frame image, the second adjustment parameter includes first key point information corresponding to the eye, and at this time, the second object is an eye of the object to be photographed in the second frame image, and there is a correspondence between key points of the eyes of the object to be photographed in the first frame image and the second frame image, the electronic device may determine second key point information of the eye in the second frame image according to the first key point information of the eye in the first frame image. Therefore, the electronic equipment can determine second key point information corresponding to the eyes of the shot object under the condition of acquiring the second frame image, adjust corresponding key points in the second key point information according to the second adjustment parameter, and then connect the adjusted key points by using the curve to construct the outline of the second object to obtain the adjusted outline of the second object; or the second object is constructed on the whole to obtain the adjusted second object, so that the second frame image is correspondingly processed.

Based on the same inventive concept, the present application further provides a processing apparatus, and fig. 2 is a block diagram of the processing apparatus provided in the embodiment of the present application. As shown in fig. 2, the processing apparatus includes:

a first obtaining module 10 configured to obtain a first frame image, wherein the first frame image includes a target object;

a second obtaining module 20 configured to obtain a first input trajectory;

a first determination module 30 configured to determine a target contour of the target object based on the first input trajectory, wherein the target contour of the target object matches a path formed by the first input trajectory;

a second determining module 40 configured to determine a second contour from the first input trajectory;

an adjusting module 50 configured to adjust the target profile to the second profile.

The processing device in the embodiment of the present application can implement the processing method provided in any embodiment of the present application through the configured functional module of the processing device.

The embodiment of the present application further provides an electronic device, which at least includes a memory 901, a processor 902, and a bus (not shown), where a schematic structural diagram of the electronic device may be as shown in fig. 3, the memory 901 stores machine-readable instructions executable by the processor 902, when the electronic device runs, the processor 902 and the memory 901 communicate through the bus, and the machine-readable instructions, when executed by the processor, perform the steps of the processing method provided in any embodiment of the present application.

Since the electronic device described in the embodiment of the present application is an electronic device provided with a memory for implementing the processing method disclosed in the embodiment of the present application, based on the processing method described in the embodiment of the present application, a person skilled in the art can understand the structure and the variation of the electronic device described in the embodiment of the present application, and thus details are not described here.

The embodiment of the present application further provides a storage medium, where one or more programs are stored, and when the one or more programs are executed by a processor, the steps of the processing method provided in any embodiment of the present application are implemented.

The storage medium in the present embodiment may be one contained in an electronic device/system; or may exist alone without being assembled into an electronic device/system. The storage medium stores one or more programs, and when the one or more programs are executed, the steps of the processing method provided according to the embodiment of the application are realized.

According to embodiments of the present application, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. Optionally, the specific examples in this embodiment may refer to examples described in any embodiment of this application, and this embodiment is not described herein again. It will be apparent to those skilled in the art that the modules or steps of the present application described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.

Moreover, although exemplary embodiments have been described herein, the scope thereof includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., of various embodiments across), adaptations or alterations. The elements of the claims are to be interpreted broadly based on the language employed in the claims and not limited to examples described in the present specification or during the prosecution of the application, which examples are to be construed as non-exclusive. It is intended, therefore, that the specification and examples be considered as exemplary only, with a true scope and spirit being indicated by the following claims and their full scope of equivalents.

The above description is intended to be illustrative and not restrictive. For example, the above-described examples (or one or more versions thereof) may be used in combination with each other. For example, other embodiments may be used by those of ordinary skill in the art upon reading the above description. In addition, in the above detailed description, various features may be grouped together to streamline the application. This should not be interpreted as an intention that a disclosed feature not claimed is essential to any claim. Rather, subject matter of the present application can lie in less than all features of a particular disclosed embodiment. Thus, the following claims are hereby incorporated into the detailed description as examples or embodiments, with each claim standing on its own as a separate embodiment, and it is contemplated that these embodiments may be combined with each other in various combinations or permutations. The scope of the application should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

The embodiments of the present application have been described in detail, but the present application is not limited to these specific embodiments, and those skilled in the art can make various modifications and changes based on the concept of the present application, and these modifications and changes should fall within the scope of the present application.

Claims

1. A method of processing, the method comprising:

obtaining a first input track;

determining a second contour according to the first input track;

and adjusting the target contour to be the second contour.

2. The method of claim 1, the determining a second contour from the first input trajectory comprising:

3. The method of claim 1, the determining a target contour of the target object based on the first input trajectory, comprising:

4. The method of claim 1, further comprising:

displaying the first input track;

5. The method of claim 1, prior to determining a target contour of the target object based on the first input trajectory, the method further comprising:

obtaining a second input track;

6. The method of claim 1, after adjusting the target profile to the second profile, the method further comprising:

obtaining a first adjusting parameter according to the second contour;

and processing the target object according to the first adjusting parameter.

7. The method of any of claims 1-6, the first frame image being an image in a target video stream, the method further comprising:

obtaining a second adjusting parameter according to the second contour;

acquiring a second frame image from the target video stream;

and processing the second frame image according to the second adjustment parameter.

8. The method according to claim 7, wherein the second adjustment parameter includes first key point information corresponding to the target object; the processing the second frame image according to the second adjustment parameter includes:

9. The method of claim 1, the determining a target contour of the target object based on the first input trajectory, comprising:

10. A processing apparatus, comprising:

a second obtaining module configured to obtain a first input trajectory;