CN110365903A

CN110365903A - Object processing method, device, equipment and readable storage medium storing program for executing based on video

Info

Publication number: CN110365903A
Application number: CN201910676167.2A
Authority: CN
Inventors: 田元
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2019-07-25
Filing date: 2019-07-25
Publication date: 2019-10-22
Anticipated expiration: 2039-07-25
Also published as: CN110365903B

Abstract

This application discloses a kind of object processing method based on video, device, equipment and readable storage medium storing program for executing, are related to field of video processing.It include target object in the video content of video flowing this method comprises: obtaining the video flowing of video；Crucial point tracking is carried out to target object, target object is corresponding with the m key point for constructing profile；Receive key point adjustment signal；It is adjusted according to position of the key point adjustment signal to the first key point；According to the first key point position adjusted, contour area corresponding with the first key point on target object is handled.By carrying out key point identification to the target object in video flowing, obtain corresponding key point at the profile of the target object, and the first key point on the profile is adjusted by receiving key point adjustment signal, the overall profile of the target object is adjusted to realize, the accuracy beautified to target object is improved, the efficiency for carrying out integrated regulation to target object is improved.

Description

Object processing method, device, equipment and readable storage medium storing program for executing based on video

Technical field

The invention relates to field of video processing, in particular to a kind of object processing method based on video, device, Equipment and readable storage medium storing program for executing.

Background technique

Video is a kind of multimedia presentation comprising the more information content, including film, TV play, advertisement video, The diversified forms such as small video, ordinary video.Optionally, which, which can be, is shot as shooting center using object, can also Being shot using other objects as shooting center.When video is shot using object as shooting center, usually deposit In the demand beautified to object.

In the related technology, when beautifying to object, the beautification mode elongated object leg, client are provided After end is by video identifying the corresponding region in object leg, the selection of degree is adjusted for leg to leg according to user Portion is elongated to length corresponding with the adjustment degree.

However, after being adjusted to leg length, leg length and body can be generated when beautifying through the above way to object The problem of material ratio is uncoordinated, adjustment result distortion, namely the accuracy beautified through the above way to object are lower.

Summary of the invention

The embodiment of the present application provides a kind of object processing method based on video, device, equipment and readable storage medium storing program for executing, After can solve be adjusted long to leg, it can generate that leg length is uncoordinated with stature ratio, and adjustment result distortion carries out object beautiful The lower problem of the accuracy of change.The technical solution is as follows:

On the one hand, a kind of object processing method based on video is provided, which comprises

The video flowing of the video is obtained, includes target object in the video content of the video flowing；

Crucial point tracking is carried out to the target object, the target object is corresponding with n key point, and described n crucial It include the m key point for constructing the profile of the target object, 0 < m≤n in point；

Key point adjustment signal is received, the key point adjustment signal is used to indicate the profile progress to the target object Adjustment；

It is adjusted according to position of the key point adjustment signal to the first key point in the m key point, institute Rheme is set for indicating the relative position between first key point and other key points；

According to the first key point position adjusted, to corresponding with first key point on the target object Contour area is handled.

On the other hand, the object handles device based on video is provided, described device includes:

Module is obtained, includes target object in the video content of the video flowing for obtaining the video flowing of the video；

Identification module, for carrying out crucial point tracking to the target object, the target object is corresponding with n key Point includes the m key point for constructing the profile of the target object, 0 < m≤n in the n key point；

Receiving module, for receiving key point adjustment signal, the key point adjustment signal is used to indicate to the target The profile of object is adjusted；

Module is adjusted, for the position according to the key point adjustment signal to the first key point in the m key point It sets and is adjusted, the position is used to indicate the relative position between first key point and other key points；

The adjustment module, is also used to according to the first key point position adjusted, on the target object with The corresponding contour area of first key point is handled.

On the other hand, a kind of computer equipment is provided, the computer equipment includes processor and memory, described to deposit Be stored at least one instruction, at least a Duan Chengxu, code set or instruction set in reservoir, at least one instruction, it is described extremely A few Duan Chengxu, the code set or instruction set are loaded by the processor and are executed to realize as in above-mentioned the embodiment of the present application The object processing method based on video provided.

On the other hand, a kind of computer readable storage medium is provided, at least one finger is stored in the storage medium Enable, at least a Duan Chengxu, code set or instruction set, at least one instruction, an at least Duan Chengxu, the code set or Instruction set is loaded by processor and is executed to realize the object handles side based on video as provided in above-mentioned the embodiment of the present application Method.

On the other hand, a kind of computer program product is provided, when the computer program product is run on computers When, so that computer is executed as described in above-mentioned the embodiment of the present application based on the object processing method of video.

Technical solution bring beneficial effect provided by the embodiments of the present application includes at least:

By carrying out key point identification to the target object in video flowing, corresponding pass at the profile of the target object is obtained Key point, and the first key point on the profile is adjusted by receiving key point adjustment signal, to realize to the target The overall profile of object is adjusted, and avoid causes stature ratio not assisted and elongating leg in the related technology It adjusts, object beautifies the lower problem of accuracy, improves the beautification accuracy beautified to target object, and improve pair The efficiency of target object progress integrated regulation.

Detailed description of the invention

In order to more clearly explain the technical solutions in the embodiments of the present application, make required in being described below to embodiment Attached drawing is briefly described, it should be apparent that, the drawings in the following description are only some examples of the present application, for For those of ordinary skill in the art, without creative efforts, it can also be obtained according to these attached drawings other Attached drawing.

Fig. 1 is one exemplary embodiment offer of the application to the pass obtained after target object progress key point identification The schematic diagram of key point；

Fig. 2 is the schematic diagram for the beautification method of object in the related technology that one exemplary embodiment of the application provides；

Fig. 3 is the implementation environment schematic diagram that one exemplary embodiment of the application provides；

Fig. 4 is the flow chart for the object processing method based on video that one exemplary embodiment of the application provides；

Fig. 5 is the schematic diagram of the profile that target object is obtained according to key point provided based on the embodiment shown in Fig. 4；

Fig. 6 is the flow chart for the object processing method based on video that another exemplary embodiment of the application provides；

Fig. 7 is the schematic diagram of the adjustment process of the key point provided based on the embodiment shown in Fig. 6；

Fig. 8 be the adjustment result according to key point that is provided based on the embodiment shown in Fig. 6 to the profile of target object into The schematic diagram of row adjustment；

Fig. 9 is the flow chart for the object processing method based on video that another exemplary embodiment of the application provides；

Figure 10 is the schematic diagram of the face feature method of adjustment provided based on the embodiment shown in Fig. 9；

Figure 11 is the flow chart for the object processing method based on video that another exemplary embodiment of the application provides；

Figure 12 is the flow chart for the object processing method based on video that another exemplary embodiment of the application provides；

Figure 13 is the structural block diagram for the object handles device based on video that one exemplary embodiment of the application provides；

Figure 14 is the structural frames for the object handles device based on video that another exemplary embodiment of the application provides Figure；

Figure 15 is the structural block diagram for the terminal that one exemplary embodiment of the application provides.

Specific embodiment

To keep the purposes, technical schemes and advantages of the application clearer, below in conjunction with attached drawing to the application embodiment party Formula is described in further detail.

Firstly, simply being introduced noun involved in the embodiment of the present application:

Artificial intelligence (Artificial Intelligence, AI): being to utilize digital computer or digital computer control Machine simulation, extension and the intelligence for extending people of system, perception environment obtain knowledge and the reason using Knowledge Acquirement optimum By, method, technology and application system.In other words, artificial intelligence is a complex art of computer science, it attempts to understand The essence of intelligence, and produce a kind of new intelligence machine that can be made a response in such a way that human intelligence is similar.Artificial intelligence The design principle and implementation method for namely studying various intelligence machines make machine have the function of perception, reasoning and decision.

Artificial intelligence technology is an interdisciplinary study, is related to that field is extensive, and the technology of existing hardware view also has software layer The technology in face.Artificial intelligence basic technology generally comprise as sensor, Special artificial intelligent chip, cloud computing, distributed storage, The technologies such as big data processing technique, operation/interactive system, electromechanical integration.Artificial intelligence software's technology mainly includes computer Several general orientation such as vision technique, voice processing technology, natural language processing technique and machine learning/deep learning.

Key point: referring to during Object identifying, the mark for being identified to characteristic portion detected Point, optionally, this feature position include position, the position where object leg, face five where object outline, object arm At least one of position where official, wherein when in the key point including the position where human face five-sense-organ, the key point is logical It is frequently located in around face mask, around looks, around nose, around lip, around ear etc..Optionally, at the human face five-sense-organ Key point include face mask key point, looks key point, nose key point, lip key point, in ear key point extremely Few one kind includes optionally eyebrow key point and eyes key point in looks key point, and eyes key point includes upper eyelid key Point and palpebra inferior key point, lip key point includes upper lip key point and lower lip key point.

Optionally, which can be detected by critical point detection algorithm, such as: be based on convolutional neural networks The key point homing method of (Convolutional Neural Networks, CNN).Optionally, the number of the key point may be used also To be configured by designer, each key point is corresponding with a crucial point identification, for the same target object in two frames For the two groups of key points identified in image there are corresponding relationship, which realized according to crucial point identification, signal Property, identification obtains key point 1 to 90 in image A, and identification obtains key point 1 to 90 in image B, then the pass in image A Key point 1 in 1 correspondence image B of key point, the key point 17 in 17 correspondence image B of key point in image A, the pass in image A Key point 81 in 81 correspondence image B of key point.

Optionally, which in practical applications, can be used for the application such as object beautification, face U.S. face, three-dimensional reconstruction, Wherein, object beautification includes weight reducing, adjustment height, adjustment body curvature etc.；Face U.S. face includes thin face, eye amplification, eyebrow Adjustment etc., pendant for being attached to around organ by face's pendant according to the position where each organ, such as: cat ear is attached to face Above profile, three-dimensional reconstruction is used to construct threedimensional model according to object outline, each body part of object, face's organ.

Schematically, referring to FIG. 1, including target object 110 in image 100, which is corresponding with arm 111, leg 112, waist 113, chest 114 and head 115 are distributed with detection around the profile of the target object 110 and obtain Key point 120, wherein in around the profile include arm 111 around, around leg 112, around chest 114 and head Around 115.

In the related technology, when beautifying to the object in video, the beautification elongated object leg is provided Mode after client is by video identifying the corresponding region in object leg, adjusts degree to leg according to user Selection leg is elongated to length corresponding with the adjustment degree, schematically, referring to FIG. 2, showing view in client The broadcast interface 200 of frequency includes object 210 in the broadcast interface 200, and the object 210 keeps whole body to be in broadcast interface 200 In, it further include special efficacy trigger control 220 in the broadcast interface 200, when user selects in the special efficacy trigger control 220 When, leg elongation is carried out to the object 210 in the broadcast interface 200 and shows elongation ratio control 230, by the ratio of elongation Elongation ratio is adjusted on control 230, so that the elongation degree long to leg controls.

Optionally, the application scenarios of the object processing method provided by the embodiments of the present application based on video include following scene At least one of:

First, being somebody's turn to do the object processing method based on video can be applied in camera application program, the camera application program Video flowing is acquired by camera first in shooting process, and the corresponding key point of target object is examined in video streaming It surveys, to carry out beautification adjustment to target object by the adjustment to key point, wherein beautification adjustment includes to target object The part that is adjusted of profile；

Second, being somebody's turn to do the object processing method based on video can be applied in live video application program, and user is by being somebody's turn to do When live video application program is broadcast live, video flowing is acquired by terminal camera, which passes through inspection The corresponding key point of target object in video flowing is surveyed, in each frame video image, by the adjustment to key point position to mesh Mark object carries out beautification adjustment, wherein beautification adjustment includes the part being adjusted to the profile of target object, and terminal will be beautiful After live video stream after change is sent to server, the live video stream after beautification is forwarded to viewing terminal by server, by seeing See that terminal plays out the live video stream after beautification；

Third, being somebody's turn to do the object processing method based on video can be applied in instant messaging application program, and user is by being somebody's turn to do During instant messaging application program and good friend carry out video calling, call video stream is acquired by the camera of terminal, it should Instant messaging application program is by the corresponding key point of target object in detection call video stream, in each frame video image, Beautification adjustment is carried out to target object by the adjustment to key point position, wherein beautification adjustment includes to target object The part that profile is adjusted, after the call after beautification is gone back video stream to server by terminal, server will be after beautification Call video circulation is sent to the corresponding terminal of good friend, is played out by the corresponding terminal of good friend to the call video stream after beautification.

Optionally, the object processing method provided by the embodiments of the present application based on video, can be applied in terminal, can also To be applied in the scene of terminal and server interaction.

When the object processing method based on video is applied in terminal, includes video processing module in the terminal, lead to It crosses video processing module and landscaping treatment is carried out to the target object in video flowing, the video flowing after landscaping treatment can be obtained.

When in the scene that the object processing method based on video is applied to terminal and server interaction, Fig. 3 is this Shen Please an exemplary embodiment provide implementation environment schematic diagram, as shown in figure 3, in the implementation environment include terminal 310, clothes Business device 320 and communication network 330；

Wherein, terminal 310 is for being acquired video flowing by camera, and the video flowing collected is passed through and is led to Communication network 330 is sent to server 320；Or, being downloaded to video flowing, and the video flowing that downloading obtains is passed through into communication network 330 are sent to server 320；Or, pre-stored video flowing in the terminal 310 is sent to service by communication network 330 Device 320.

Optionally, above-mentioned camera application program, live video application program, instant messaging application are installed in terminal 310 At least one of program.

It optionally, include video processing module 321 in server 320, by video processing module 321 in video flowing After target object carries out key point identification, key point is adjusted according to key point adjustable strategies, to realize in the video flowing Target object beautification.Optionally, the video flowing after beautification is fed back to terminal 310,310 pairs of terminal beautifications by server 320 Video flowing afterwards plays out.

It is worth noting that, above-mentioned terminal 310 can be implemented as mobile phone, plate, smartwatch, above-knee portable notebook The mobile terminals such as computer, music player devices also can be implemented as the terminals such as desktop computer, monitoring device, the embodiment of the present application This is not limited.

Above-mentioned server 320 can be implemented as a server, also can be implemented as one group that multiple servers are built into Server can be implemented as physical server, also can be implemented as Cloud Server, and the embodiment of the present application is not limited this.

In conjunction with above-mentioned noun brief introduction and application scenarios, to the object handles side provided by the embodiments of the present application based on video Method is illustrated, and Fig. 4 is the flow chart for the object processing method based on video that one exemplary embodiment of the application provides, with This method is applied to be illustrated in terminal, as shown in figure 4, this method comprises:

Step 401, the video flowing of video is obtained, includes target object in the video content of the video flowing.

Optionally, which can be at least one of personage, animal, cartoon character, cartoon figure, the target Object is also implemented as at least one of building, plant, ornament, affaires, this is not added in the embodiment of the present application To limit.

Optionally, terminal can directly acquire stored video flowing, can also be clapped by camera video flowing It takes the photograph, video flowing can also be downloaded from server, the application is not limited the acquisition modes of video flowing.

Optionally, it may include single target object in the video content of the video flowing, also may include multiple targets pair As.Optionally, the identifiable target object of single is the object within preset quantity, schematically, video flowing in the video flowing The first video frame in include 8 objects, 5 objects in 8 objects are then determined as target object by preset quantity 5.

Optionally it is determined that the mode of target object includes such as any one under type:

First, target object is determined according to the size of object shared display area in the video frame

Schematically, when preset quantity is 5, and the number of objects in video frame is greater than 5, by display area maximum 5 A object is determined as target object.

Second, target object is determined according to the display scale of object in the video frame；

Wherein, it is illustrated so that target object is embodied as personage as an example, the display scale is for indicating personage in video frame The area of middle display accounts for the percentage of whole body area, such as: preset quantity 3, and display scale is 45% to personage A in the video frame (namely 45% display of personage's A whole body is in the video frame), the display scale of personage B in the video frame is 47%, and personage C exists Display scale in video frame is 85%, and the display scale of personage D in the video frame is 90%, personage E in the video frame aobvious Show that ratio is 100%, then personage C, personage D and personage E is determined as target person.

Third randomly selects object corresponding with preset quantity as target object from all objects of video frame.

Optionally, each frame all includes above-mentioned target object in the video flowing, or, in the partial video frame in the video flowing Including above-mentioned target object.

Step 402, crucial point tracking is carried out to target object, which is corresponding with n key point, this n crucial It include the m key point for constructing the profile of target object, 0 < m≤n in point.

It optionally, can be by each frame video frame when carrying out crucial point tracking to the target object in video flowing In target object carry out key point identification realize, can also to the target object in key frame (I frame) carry out key point identification It realizes, schematically, is illustrated respectively in conjunction with above two mode:

First, crucial point tracking is realized by carrying out key point identification to the target object in each frame video frame；

Optionally, each frame video frame in the video flowing is obtained, and the target object in each frame video frame is carried out Key point identification, wherein (wherein wrapped to identifying to obtain the corresponding one group of set of keypoints of the target object for every frame video frame Include n key point), and there are corresponding relationships for the corresponding n key point of every two frames video frame, schematically, know for video frame 1 Set of keypoints a is not obtained, obtains set of keypoints b for the identification of video frame 2, the key point in set of keypoints a and key There are corresponding relationships for key point in point set b, and optionally, which is one-to-one relationship, that is, set of keypoints A key point in a corresponds to a key point in set of keypoints b.It is right by the successively acquisition to each frame video frame Key point in each frame video frame is successively identified, to realize the tracking to key point.

Second, crucial point tracking is realized by carrying out key point identification to the target object in key frame (I frame)；

Optionally, the key frame in the video flowing is obtained, key point identification is carried out to the target object in key frame, is obtained N key point corresponding with target object in every frame key frame, wherein n key point in different key frames has corresponding close System, optionally, the corresponding relationship are one-to-one relationship.Optionally, according to the key point determination and the I identified in I frame Key point in the corresponding non-key frame of frame (P frame and B frame).Optionally, non-key frame is according to I frame and based on I frame The video frame that pixel changes, the change of the pixel where the key point identified in I frame and the key point Change (such as: change in location) to determine in non-key frame and work as key point position.

Optionally, including the key point at the target object profile in the n key point, optionally, in the n key point It further include at least one of the corresponding key point of target object face face, the corresponding key point of target object minutia.

Wherein, the leg profile key point in the key point at the target object profile including target object, target object Buttock contour key point, the waist profile key point of target object, the thoracic contours key point of goal task, target object At least one in the contouring head key point of arm geometry key point, the neck profile key point of target object and target object Kind；The corresponding key point of subject face face include looks key point, nose key point, lip key point, in ear key point At least one；The corresponding key point of target object minutia includes finger key point, hair line key point, ankle key point At least one of.

It optionally, include the m key points being distributed at the profile of target object, the m key point in the n key point It is constructed for the profile to target object, optionally, sequentially connecting the m key point in a manner of curve connection The profile of the target object is constructed.

Schematically, it referring to FIG. 5, including target object 510 in picture frame 500, detects and obtains from the picture frame 500 The key point 511 being distributed at 510 profile of target object is sequentially connected with key point 511 by smooth curve, obtains The corresponding outline 512 of the target object, the outline 521 are to determine obtained contour curve by key point 511.? It changes that is, the outline 512 is corresponding with the variation of 511 position of key point.

Step 403, key point adjustment signal is received.

Optionally, which, which is used to indicate, is adjusted the profile of target object.Optionally, the key Point adjustment signal is also used to indicate to be adjusted the face of target object, and/or, which is also used to indicate The minutia of target object is adjusted.

Optionally, which, which can be, adjusts operation triggering according to the key point of user, is also possible to The recording of video flowing is automatically generated according to user.

Wherein, when key point adjustment operation triggering of the key point adjustment signal by user, user can be directly to this Key point carries out drag operation to realize the adjustment to the key point, can also be selected on beautification control automatically, from And trigger the adjust automatically to key point.

Step 404, it is adjusted according to position of the key point adjustment signal to the first key point in m key point.

Optionally, which can be implemented as a key point, also can be implemented as one group of key point (such as: one The continuous key point of group), it is also implemented as multiple groups key point (such as: the continuous key point of multiple groups).

Optionally, which is used to indicate the relative position between first key point and other key points.Optionally, often A key point there are relative positional relationship, adjusts between adjacent key point according to the position to the first key point, to being adjusted The key point key point adjacent with the key point between relative positional relationship be adjusted.

Optionally, there are left-right symmetric relations in object outline for the m key point, that is, in the object outline of left side Point A it is corresponding with the point B in the object outline of right side, when point A according to key point adjustment signal carry out position adjustment when, the point B with The point A carries out corresponding symmetrical adjustment, and e.g., when point A carries out position adjustment to the left, point B carries out position adjustment to the right；Point A When carrying out position adjustment upwards, point B also carries out position adjustment upwards.

Optionally, when the key point adjustment signal be according to drag operation generate when, according to drag operation to this first The position of key point is adjusted；It is controlled when key point adjustment operates be automatically generated or user by the automatic beautification of triggering When part generates, the position adjustment of the key point can also be by default adjustment model realization, that is, according to key point adjustment signal By the m default adjustment model of key point input, and pass through m key point after default adjustment model output is adjusted, wherein The first key point being adjusted including position, wherein the default adjustment model is by the corresponding key point training of sample object It obtains.

Step 405, according to the first key point position adjusted, to profile corresponding with the first key point on target object Region is handled.

Optionally, which is the key point on target object profile, and the profile of the target object is by being located at The first key point on profile connects to obtain by smooth curve, that is, when the position of the first key point changes, it should Profile that first key point connects correspondence changes, then is adjusted by the position to first key point, drive this The corresponding contour area of one key point is adjusted correspondingly.

In conclusion the object processing method provided in this embodiment based on video, by the target pair in video flowing As carrying out key point identification, corresponding key point at the profile of the target object is obtained, and by receiving key point adjustment signal The first key point on the profile is adjusted, the overall profile of the target object is adjusted to realize, is avoided Cause stature ratio uncoordinated and elongating leg in the related technology, object beautifies the lower problem of accuracy, mentions The high beautification accuracy that target object is beautified, and improve the efficiency that integrated regulation is carried out to target object.

In an alternative embodiment, above-mentioned key point adjustment signal can be generated by receiving drag operation, and Fig. 6 is The flow chart for the object processing method based on video that another exemplary embodiment of the application provides is applied to eventually in this way It is illustrated in end, as shown in fig. 6, this method comprises:

Step 601, the video flowing of video is obtained, includes target object in the video content of the video flowing.

Optionally, it may include single target object in the video content of the video flowing, also may include multiple targets pair As.

Optionally, the mode for obtaining the video flowing has been carried out detailed description in above-mentioned steps 401, no longer superfluous herein It states.

Step 602, crucial point tracking is carried out to target object, which is corresponding with n key point, this n crucial It include the m key point for constructing the profile of target object, 0 < m≤n in point.

It optionally, can be by each frame video frame when carrying out crucial point tracking to the target object in video flowing In target object carry out key point identification realize, can also to the target object in key frame (I frame) carry out key point identification It realizes.

Step 603, the drag operation acted in the first key point is received.

Optionally, the initial position of the drag operation is located at around the first key point in the first preset range.Optionally, should Refer in the first preset range around first key point using first key point as the center of circle, first distance is the circular scope of radius It is interior.Optionally, for corresponding first preset range of each key point.Optionally, it is not deposited between each first preset range In intersection, when the initial position of the drag operation received is located in the first preset range of the first key point, by the dragging It operates as the drag operation for being directed to first key point.Optionally, when between two neighboring first preset range there are intersection, And the initial position of drag operation in the intersection range between two neighboring first preset range when, which is made For the drag operation for corresponding two the first key points of the intersection range.

Optionally, the dragging signal directly carries out the key point of position adjustment according to above-mentioned first key point, and being located at should The second key point around first key point in the second preset range is to carry out position indirectly under the drive of first key point The key point of adjustment.

Optionally, the end position of the drag operation is the relative position for initial position, that is, the dragging is grasped The end position of work is indicated by the displacement of initial position and the initial position；Or, the end position of the drag operation passes through Coordinate representation in terminal interface.

It is worth noting that, above-mentioned n key point may be visible in the broadcast interface of video flowing, it is also possible to not Visible, the embodiment of the present application is not limited this.

Step 604, key point adjustment signal is generated according to drag operation.

It optionally, include the knot of corresponding first key point of drag operation and drag operation in the key point adjustment signal Beam position, or, including the first key point, dragging displacement in the key point adjustment signal.

Optionally, which can be the operation carried out when video flowing stops at a certain frame video frame, can also be with It is the operation carried out in the playing process of video flowing.Optionally, when the drag operation is to hold in the playing process of video flowing When capable operation, the dragging displacement directly according to the drag operation on terminal display screen generates key point adjustment signal.

Step 605, the first key point in m key point is adjusted to end position according to key point adjustment signal.

Optionally, the mode being adjusted to first key point includes any one in following which:

First, according to the end position in above-mentioned key point adjustment signal, which is adjusted to the stop bits Coordinate points where setting；

Schematically, the initial position of drag operation is (10,7), and end position is (10,8), then according to the initial position After determining corresponding first key point of the drag operation, which is adjusted at end position (10,8).

Second, it include displacement information of the end position based on initial position in above-mentioned end position, according to the displacement information To the mobile displacement corresponding with the end position in the position of first key point；

Schematically, the initial position of drag operation is (10,7), and end position is (10,8), then true according to initial position After determining corresponding first key point (10.5,7) of drag operation, by the mobile position corresponding with the end position of first key point It sets, the first key point (10.5,8) after being adjusted.

Step 606, the key point being located at around the first key point in the second preset range in m key point is carried out and the The corresponding adjustment of one key point.

Optionally, it connects to obtain by m crucial point curve due to the profile of the target object, when the distance between key point When smaller, for the drag operation of single first key point, to the second key point around first key point, there are corresponding shadows It rings, the positional relationship between influence degree and second key point and the first key point is related.

Optionally, the direction that initial position in drag operation is moved to end position is determined as drawing direction, there is starting Position to the distance between end position is determined as dragging distance, to the be located at around the first key point in the second preset range Two key points determine the positional relationship of the second key point Yu the first key point, are put to dragging distance with corresponding with positional relationship Contracting ratio carries out scaling, obtains scaling distance, is adjusted with drawing direction and scaling distance to the position of the second key point.

Optionally, above-mentioned second key point and the positional relationship in the first key point may include any in following relationship It is a kind of:

First, which is the distance between the second key point and the first key point relationship；

Optionally, to the second key point being located at around the first key point within the scope of pre-determined distance, the second key point is determined The distance between first key point relationship, and scaling is carried out to dragging distance and the corresponding scaling ratio of distance relation.It is optional Ground, distance is remoter, and scaling ratio is higher, and the size for the scaling distance that scaling obtains is smaller；Distance is closer, and scaling ratio is smaller, The size for the scaling distance that scaling obtains is bigger.

Second, spaced relationship of the positional relationship between the second key point and the first key point.

Optionally, to be located at the first key point around, and be separated by keypoint quantity be less than preset quantity the second key point, Determine the keypoint quantity being separated by between the second key point and the first key point, and to the keypoint quantity for dragging distance with being separated by Corresponding scaling ratio carries out scaling.Optionally, the keypoint quantity being separated by is more, and scaling ratio is higher, and what scaling obtained puts The size of contracting distance is smaller；The keypoint quantity being separated by is fewer, and scaling ratio is smaller, and the size for the scaling distance that scaling obtains is got over Greatly.

Schematically, referring to FIG. 7, including target object 710 in picture frame 700, image knowledge is carried out to picture frame 700 After not, the corresponding key point 720 of target object 710 is obtained, which is located at around 710 profile of target object, reception pair After the drag operation of key point 721, which is subjected to position corresponding with the drag operation and is adjusted, and should to being located at Key point 722, key point 723, key point 724 and key point 725 around key point 721 carry out corresponding position adjustment, Wherein, 0 key point in interval between key point 722 and key point 723 and key point 721, and key point 724 and key point 1 key point is spaced between 725 and key point 721, therefore the first shift length of the key point 722 and key point 723 closes relatively The second displacement of key point 724 and key point 725 is apart from larger, and the first shift length phase of key point 722 and key point 723 It is smaller to the shift length of key point 721.

Step 607, according to the first key point position adjusted, to profile corresponding with the first key point on target object Region is handled.

Optionally, after by contour area corresponding with first key point on target object according to first key point adjustment Position it is corresponding carry out profile adjustment, optionally, using the profile that the first key point adjusted connects as adjusted Profile.

Schematically, referring to FIG. 8, including target object 810 in picture frame 800, the target object 810 correspondence is related Key point 820, after executing drag operation to the key point 820, key point 830 after being adjusted, in conjunction with the company of the key point 830 Connect the profile 840 of the target object after curve is adjusted.

Optionally, for drag operation after the frame that plays or one group of video frame, execute above-mentioned steps 605 to step 607, and for the video frame after a frame or one group of video frame, according to the position of above-mentioned first key frame and the second key frame Adjustment carries out corresponding adjustment.

Method provided in this embodiment is realized by drag operation and carries out position to the key point on the profile of target object Adjustment improves the tune to the whole body curvature of target object so that the profile of target object be driven to carry out corresponding adjustment Whole accuracy rate.

It in an alternative embodiment, further include k corresponding with target object face crucial in above-mentioned n key point Point, the key point adjustment signal are also used to indicate to be adjusted the face of target object, 0 < k < n.Fig. 9 is that the application is another The flow chart for the object processing method based on video that a exemplary embodiment provides, for being applied in terminal in this way into Row explanation, as shown in figure 9, this method comprises:

Step 901, video flowing is obtained, includes target object in the video content of the video flowing.

Step 902, crucial point tracking is carried out to the target object in video flowing, wherein the target object is corresponding with n pass Key point includes k key point corresponding with target object face, 0 < k < n in the n key point.

It optionally, include the corresponding k key point of target object face face in the n key point.Optionally, the k It include at least one of looks key point, nose key point, lip key point, ear key point in key point.

Step 903, key point adjustment signal is received.

Optionally, which, which is used to indicate, is adjusted the face of target object.

Step 904, it is adjusted according to position of the key point adjustment signal to the third key point in k key point.

Optionally, this according to position of the key point adjustment signal to third key point in k key point be adjusted including Such as at least one of under type:

First, the drag operation to third key point in the k key point is received, and according to drag operation to third key The position of point is adjusted；

Specific dragging process please refers to above-mentioned steps 603 to step 606.

Second, the trigger action to face's adjustment control is received, and the k key point is inputted by face according to the trigger action Portion adjusts model, and adjusts k key point after model output is adjusted by face.

Step 905, according to third key point position adjusted, the corresponding face feature of third key point is adjusted It is whole.

Optionally, it is adjusted by the position to third key point in the k key point, the face of target object is wrapped Include at least one of looks amplification, the adjustment of nose Adjusting Shape, lip shape, ear shape adjustment adjustment.

Optionally, it for the adjustment mode of the k key point, can also be adjusted in the following way:

1, k key point is divided according to face feature position, obtains each face feature position corresponding first Set of keypoints；

Optionally, after which being divided according to face feature, the corresponding first crucial point set of looks is obtained Close 1, corresponding first set of keypoints 2 of nose, corresponding first set of keypoints 3 of lip and corresponding first key of ear Point set 4.

2, the first adjustment signal to target signature position is received, which is used to indicate to target signature portion Position carries out independent adjustment；

Optionally, which can be target signature position is selected after, according in target signature portion The signal that drag operation at position generates, the adjustment operation being also possible on the adjustment control for target signature part generate Signal.

3, it is clicked through according to the 4th in the first adjustment signal the first set of keypoints corresponding to target signature position is crucial Line position sets adjustment；

4, according to the 4th key point position adjusted, target signature position is adjusted.

Schematically, referring to FIG. 10, including that position selects control in the adjustment interface 1010 of video flowing, including The corresponding selection control 1011 of looks, the corresponding selection control 1012 of nose, the corresponding selection control 1013 of lip and ear Corresponding selection control 1014, schematically, when receiving the selection operation of selection control 1011 corresponding to looks, at this The looks part 1020 of displaying target object in the adjustment interface 1010 of video flowing, and show that looks adjusts control 1030, pass through Adjustment to looks adjustment control 1030, is adjusted the parameters such as looks size, eyes spacing, pupil size；Or, directly existing Drag operation is carried out on looks part 1020, and looks size is adjusted according to drag operation.

Method provided in this embodiment obtains target pair by carrying out key point identification to the target object in video flowing The corresponding k key point of the face of elephant, and operated by the adjustment to the k key point, to realize to the target object The adjustment of face feature improves the accuracy and accuracy beautified to target object.

In an alternative embodiment, above-mentioned n key point can be divided directly according to body part, and according to The difference of body part carries out corresponding adjustment, and Figure 11 is pair based on video that another exemplary embodiment of the application provides As the flow chart of processing method, it is applied to carry out or illustrate in terminal in this way, as shown in figure 11, this method comprises:

Step 1101, video flowing is obtained, includes target object in the video content of the video flowing.

Step 1102, crucial point tracking is carried out to the target object in video flowing, wherein the target object is corresponding with n Key point, n are positive integer.

Step 1103, n key point is divided according to body part, obtains each body part corresponding second and closes Key point set.

Optionally, the n key point is divided according to body part, obtains corresponding second set of keypoints in head (including the corresponding key point of face's face), corresponding second set of keypoints of neck, corresponding second key point of chest The corresponding second crucial point set of set, corresponding second set of keypoints of arm, corresponding second set of keypoints of waist, buttocks At least one of second set of keypoints such as conjunction, corresponding second set of keypoints in leg.

Step 1104, the second adjustment signal to target body portion is received.

Optionally, which, which is used to indicate, carries out independent adjustment to target body portion.

Optionally, which can be target body portion is selected after, according in intended body portion Respectively drag operation generate signal, be also possible to for target body portion adjustment control on adjustment operation generate Signal.

Step 1105, according to the 5th in second adjustment signal pair the second set of keypoints corresponding with target body portion Key point carries out position adjustment.

Step 1106, according to the 5th key point position adjusted, the target body portion on target object is adjusted It is whole.

It schematically, include that body part selects control in the adjustment interface of video flowing, it is corresponding including head Select control a, the corresponding selection control b of neck, the corresponding selection control c of chest, the corresponding selection control d of waist, arm pair Selection control e, the corresponding selection control f in leg answered, schematically, when the choosing for receiving selection control f corresponding to leg When selecting operation, the leg area of displaying target object in the adjustment interface of the video flowing, and show that leg adjusts control, pass through Adjustment to leg adjustment control, is adjusted the parameters such as leg length, leg fineness；Or, directly enterprising in leg area Row drag operation, and to leg, long, leg fineness is adjusted according to drag operation.

Figure 12 is the flow chart for the object processing method based on video that another exemplary embodiment of the application provides, with The object is target person, is applied to be illustrated in terminal in this way, this method comprises:

Step 1201, client obtains the personage's stature and face information in video flowing by terminal.

Optionally, after client obtains the video flowing that terminal is acquired by camera, to the target person in the video flowing Key point identification is carried out, and determines the stature and face characteristic of target person according to the key point that identification obtains.

Step 1202, landscaping treatment is carried out by the tactful personage's stature and face in video flowing of video beautification, obtained To beautification video data.

Optionally, video beautification strategy includes: to execute drag operation to video to carry out at beautification to target person Reason carries out triggering to automatic beautification control to carry out at least one of landscaping treatment to target person.

Step 1203, beautification video data is uploaded to server, so that vlewer terminals receive the beautification that server is sent Video data.

Optionally, it is illustrated so that above-mentioned client is live streaming application program as an example, client flows into the live video After row landscaping treatment, the live video stream after landscaping treatment is uploaded to server, and by server by the view after landscaping treatment Frequency stream is sent to vlewer terminals and plays out.

Step 1204, the video messaging that server is sent is received, and beautification video data is played out.

Figure 13 is the structural block diagram for the object handles device based on video that one exemplary embodiment of the application provides, such as Shown in Figure 13, which includes: to obtain module 1310, identification module 1320, receiving module 1330 and adjustment module 1340；

Module 1310 is obtained, includes target in the video content of the video flowing for obtaining the video flowing of the video Object；

Identification module 1320, for carrying out crucial point tracking to the target object, the target object is corresponding with n pass Key point includes the m key point for constructing the profile of the target object, 0 < m≤n in the n key point；

Receiving module 1330, for receiving key point adjustment signal, the key point adjustment signal is used to indicate to described The profile of target object is adjusted；

Adjust module 1340, for according to the key point adjustment signal to the first key point in the m key point Position be adjusted, the position is used to indicate relative position between first key point and other key points；

The adjustment module 1340 is also used to according to the first key point position adjusted, to the target object Upper contour area corresponding with first key point is handled.

In an alternative embodiment, the receiving module 1330 is also used to reception and acts on first key point On drag operation, the initial position of the drag operation is located at around first key point in the first preset range；According to The drag operation generates the key point adjustment signal, includes that the drag operation is corresponding in the key point adjustment signal The end position of first key point and the drag operation.

In an alternative embodiment, the adjustment module 1340, being also used to will according to the key point adjustment signal The first key point in the m key point is adjusted to the end position；It is closed to being located at described first in the m key point Key point around key point in the second preset range carries out adjustment corresponding with first key point.

In an alternative embodiment, as shown in figure 14, the adjustment module 1340, comprising:

Submodule 1341 is determined, for initial position described in the drag operation to be moved to the side of the end position To drawing direction is determined as, the distance between described initial position to the end position is determined as dragging distance；

The determining submodule 1341 is also used to second be located at around first key point in the second preset range Key point determines the positional relationship between second key point and first key point；

The determining submodule 1341 is also used to the dragging distance with scaling ratio corresponding with the positional relationship Scaling is carried out, scaling distance is obtained；

Adjusting submodule 1342, for the position with the drawing direction and the scaling distance to second key point It sets and is adjusted.

In an alternative embodiment, the adjustment module 1340, comprising:

Input submodule 1343, for the m key point to be inputted default adjustment according to the key point adjustment signal Model；

Output sub-module 1344, for exporting the m key point after being adjusted by the default adjustment model, Wherein, the default adjustment model is obtained by the corresponding key point training of sample object.

It in an alternative embodiment, further include k corresponding with the target object face in the n key point Key point, the key point adjustment signal are also used to indicate that the face to the target object is adjusted, 0 < k < n；

The adjustment module 1340, is also used to according to the key point adjustment signal to the third in the k key point The position of key point is adjusted；According to the third key point position adjusted, face corresponding to the third key point Portion's feature is adjusted.

In an alternative embodiment, described device further include:

Division module 1350 obtains each face for dividing the k key point according to face feature position Corresponding first set of keypoints of characteristic portion；

The receiving module 1330 is also used to receive the first adjustment signal to target signature position, the first adjustment Signal, which is used to indicate, carries out independent adjustment to the target signature position；

The adjustment module 1340 is also used to according to the first adjustment signal to the corresponding institute in the target signature position The 4th key point stated in the first set of keypoints carries out position adjustment；It is right according to the 4th key point position adjusted The target signature position is adjusted.

In an alternative embodiment, described device further include:

Division module 1350 obtains each body for dividing the n key point according to body part Corresponding second set of keypoints in part；

The receiving module 1330 is also used to receive the second adjustment signal to target body portion, the second adjustment Signal, which is used to indicate, carries out independent adjustment to the target body portion；

The adjustment module 1340 is also used to corresponding with the target body portion according to the second adjustment signal pair The 5th key point in second set of keypoints carries out position adjustment；According to the 5th key point position adjusted, The target body portion is adjusted.

In an alternative embodiment, the acquisition module 1310, is also used to obtain the key frame in the video flowing；

The identification module 1320 is also used to carry out key point identification to the target object in the key frame, obtain The n key point corresponding with the target object into every frame key frame, wherein the n key point in different key frames There are corresponding relationships.

In conclusion the object handles device provided in this embodiment based on video, by the target pair in video flowing As carrying out key point identification, corresponding key point at the profile of the target object is obtained, and by receiving key point adjustment signal The first key point on the profile is adjusted, the overall profile of the target object is adjusted to realize, is avoided Cause stature ratio uncoordinated and elongating leg in the related technology, object beautifies the lower problem of accuracy, mentions The high beautification accuracy that target object is beautified, and improve the efficiency that integrated regulation is carried out to target object.

It should be understood that the object handles device provided by the above embodiment based on video, only with above-mentioned each function mould The division progress of block can according to need and for example, in practical application by above-mentioned function distribution by different functional modules It completes, i.e., the internal structure of equipment is divided into different functional modules, to complete all or part of the functions described above. In addition, the object handles device provided by the above embodiment based on video belongs to the object processing method embodiment based on video Same design, specific implementation process are detailed in embodiment of the method, and which is not described herein again.

Figure 15 shows the structural block diagram of the terminal 1500 of an illustrative embodiment of the invention offer.The terminal 1500 can To be: smart phone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III, dynamic image expert's compression standard audio level 3), MP4 (Moving Picture Experts Group Audio Layer IV, dynamic image expert's compression standard audio level 4) player, laptop or desktop computer.Terminal 1500 is also Other titles such as user equipment, portable terminal, laptop terminal, terminal console may be referred to as.

In general, terminal 1500 includes: processor 1501 and memory 1502.

Processor 1501 may include one or more processing cores, such as 4 core processors, 8 core processors etc..Place Reason device 1501 can use DSP (Digital Signal Processing, Digital Signal Processing), FPGA (Field- Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array, may be programmed Logic array) at least one of example, in hardware realize.Processor 1501 also may include primary processor and coprocessor, master Processor is the processor for being handled data in the awake state, also referred to as CPU (Central Processing Unit, central processing unit)；Coprocessor is the low power processor for being handled data in the standby state.? In some embodiments, processor 1501 can be integrated with GPU (Graphics Processing Unit, image processor), GPU is used to be responsible for the rendering and drafting of content to be shown needed for display screen.In some embodiments, processor 1501 can also be wrapped AI (Artificial Intelligence, artificial intelligence) processor is included, the AI processor is for handling related machine learning Calculating operation.

Memory 1502 may include one or more computer readable storage mediums, which can To be non-transient.Memory 1502 may also include high-speed random access memory and nonvolatile memory, such as one Or multiple disk storage equipments, flash memory device.In some embodiments, the non-transient computer in memory 1502 can Storage medium is read for storing at least one instruction, at least one instruction performed by processor 1501 for realizing this Shen Please in embodiment of the method provide the object processing method based on video.

In some embodiments, terminal 1500 is also optional includes: peripheral device interface 1503 and at least one periphery are set It is standby.It can be connected by bus or signal wire between processor 1501, memory 1502 and peripheral device interface 1503.It is each outer Peripheral equipment can be connected by bus, signal wire or circuit board with peripheral device interface 1503.Specifically, peripheral equipment includes: In radio circuit 1504, touch display screen 1505, camera 1506, voicefrequency circuit 1507, positioning component 1508 and power supply 1509 At least one.

Peripheral device interface 1503 can be used for I/O (Input/Output, input/output) is relevant outside at least one Peripheral equipment is connected to processor 1501 and memory 1502.In some embodiments, processor 1501, memory 1502 and periphery Equipment interface 1503 is integrated on same chip or circuit board；In some other embodiments, processor 1501, memory 1502 and peripheral device interface 1503 in any one or two can be realized on individual chip or circuit board, this implementation Example is not limited this.

Radio circuit 1504 is for receiving and emitting RF (Radio Frequency, radio frequency) signal, also referred to as electromagnetic signal. Radio circuit 1504 is communicated by electromagnetic signal with communication network and other communication equipments.Radio circuit 1504 is by telecommunications Number being converted to electromagnetic signal is sent, alternatively, the electromagnetic signal received is converted to electric signal.Optionally, radio circuit 1504 include: antenna system, RF transceiver, one or more amplifiers, tuner, oscillator, digital signal processor, volume solution Code chipset, user identity module card etc..Radio circuit 1504 can by least one wireless communication protocol come with it is other Terminal is communicated.The wireless communication protocol includes but is not limited to: WWW, Metropolitan Area Network (MAN), Intranet, each third generation mobile communication network (2G, 3G, 4G and 5G), WLAN and/or WiFi (Wireless Fidelity, Wireless Fidelity) network.In some implementations In example, radio circuit 1504 can also include that NFC (Near Field Communication, wireless near field communication) is related Circuit, the application are not limited this.

Display screen 1505 is for showing UI (User Interface, user interface).The UI may include figure, text, Icon, video and its their any combination.When display screen 1505 is touch display screen, display screen 1505 also there is acquisition to exist The ability of the touch signal on the surface or surface of display screen 1505.The touch signal can be used as control signal and be input to place Reason device 1501 is handled.At this point, display screen 1505 can be also used for providing virtual push button and/or dummy keyboard, it is also referred to as soft to press Button and/or soft keyboard.In some embodiments, display screen 1505 can be one, and the front panel of terminal 1500 is arranged；Another In a little embodiments, display screen 1505 can be at least two, be separately positioned on the different surfaces of terminal 1500 or in foldover design； In still other embodiments, display screen 1505 can be flexible display screen, is arranged on the curved surface of terminal 1500 or folds On face.Even, display screen 1505 can also be arranged to non-rectangle irregular figure, namely abnormity screen.Display screen 1505 can be with Using LCD (Liquid Crystal Display, liquid crystal display), OLED (Organic Light-Emitting Diode, Organic Light Emitting Diode) etc. materials preparation.

CCD camera assembly 1506 is for acquiring image or video.Optionally, CCD camera assembly 1506 includes front camera And rear camera.In general, the front panel of terminal is arranged in front camera, the back side of terminal is arranged in rear camera.? In some embodiments, rear camera at least two is that main camera, depth of field camera, wide-angle camera, focal length are taken the photograph respectively As any one in head, to realize that main camera and the fusion of depth of field camera realize background blurring function, main camera and wide Pan-shot and VR (Virtual Reality, virtual reality) shooting function or other fusions are realized in camera fusion in angle Shooting function.In some embodiments, CCD camera assembly 1506 can also include flash lamp.Flash lamp can be monochromatic temperature flash of light Lamp is also possible to double-colored temperature flash lamp.Double-colored temperature flash lamp refers to the combination of warm light flash lamp and cold light flash lamp, can be used for Light compensation under different-colour.

Voicefrequency circuit 1507 may include microphone and loudspeaker.Microphone is used to acquire the sound wave of user and environment, and It converts sound waves into electric signal and is input to processor 1501 and handled, or be input to radio circuit 1504 to realize that voice is logical Letter.For stereo acquisition or the purpose of noise reduction, microphone can be separately positioned on the different parts of terminal 1500 to be multiple. Microphone can also be array microphone or omnidirectional's acquisition type microphone.Loudspeaker is then used to that processor 1501 or radio frequency will to be come from The electric signal of circuit 1504 is converted to sound wave.Loudspeaker can be traditional wafer speaker, be also possible to piezoelectric ceramics loudspeaking Device.When loudspeaker is piezoelectric ceramic loudspeaker, the audible sound wave of the mankind can be not only converted electrical signals to, can also be incited somebody to action Electric signal is converted to the sound wave that the mankind do not hear to carry out the purposes such as ranging.In some embodiments, voicefrequency circuit 1507 may be used also To include earphone jack.

Positioning component 1508 is used for the current geographic position of positioning terminal 1500, to realize navigation or LBS (Location Based Service, location based service).Positioning component 1508 can be the GPS (Global based on the U.S. Positioning System, global positioning system), China dipper system or Russia Galileo system positioning group Part.

Power supply 1509 is used to be powered for the various components in terminal 1500.Power supply 1509 can be alternating current, direct current Electricity, disposable battery or rechargeable battery.When power supply 1509 includes rechargeable battery, which can be line charge Battery or wireless charging battery.Wired charging battery is the battery to be charged by Wireline, and wireless charging battery is to pass through The battery of wireless coil charging.The rechargeable battery can be also used for supporting fast charge technology.

In some embodiments, terminal 1500 further includes having one or more sensors 1510.One or more sensing Device 1510 includes but is not limited to: acceleration transducer 1511, gyro sensor 1512, pressure sensor 1513, fingerprint sensing Device 1514, optical sensor 1515 and proximity sensor 1516.

Acceleration transducer 1511 can detecte the acceleration in three reference axis of the coordinate system established with terminal 1500 Size.For example, acceleration transducer 1511 can be used for detecting component of the acceleration of gravity in three reference axis.Processor The 1501 acceleration of gravity signals that can be acquired according to acceleration transducer 1511, control touch display screen 1505 with transverse views Or longitudinal view carries out the display of user interface.Acceleration transducer 1511 can be also used for game or the exercise data of user Acquisition.

Gyro sensor 1512 can detecte body direction and the rotational angle of terminal 1500, gyro sensor 1512 Acquisition user can be cooperateed with to act the 3D of terminal 1500 with acceleration transducer 1511.Processor 1501 is according to gyro sensors The data that device 1512 acquires, following function may be implemented: action induction (for example changing UI according to the tilt operation of user) is clapped Image stabilization, game control and inertial navigation when taking the photograph.

The lower layer of side frame and/or touch display screen 1505 in terminal 1500 can be set in pressure sensor 1513.When When the side frame of terminal 1500 is arranged in pressure sensor 1513, user can detecte to the gripping signal of terminal 1500, by Reason device 1501 carries out right-hand man's identification or prompt operation according to the gripping signal that pressure sensor 1513 acquires.Work as pressure sensor 1513 when being arranged in the lower layer of touch display screen 1505, is grasped by processor 1501 according to pressure of the user to touch display screen 1505 Make, realization controls the operability control on the interface UI.Operability control include button control, scroll bar control, At least one of icon control, menu control.

Fingerprint sensor 1514 is used to acquire the fingerprint of user, is collected by processor 1501 according to fingerprint sensor 1514 Fingerprint recognition user identity, alternatively, by fingerprint sensor 1514 according to the identity of collected fingerprint recognition user.Knowing Not Chu the identity of user when being trusted identity, authorize the user to execute relevant sensitive operation by processor 1501, which grasps Make to include solving lock screen, checking encryption information, downloading software, payment and change setting etc..Fingerprint sensor 1514 can be set Set the front, the back side or side of terminal 1500.When being provided with physical button or manufacturer Logo in terminal 1500, fingerprint sensor 1514 can integrate with physical button or manufacturer Logo.

Optical sensor 1515 is for acquiring ambient light intensity.In one embodiment, processor 1501 can be according to light The ambient light intensity that sensor 1515 acquires is learned, the display brightness of touch display screen 1505 is controlled.Specifically, work as ambient light intensity When higher, the display brightness of touch display screen 1505 is turned up；When ambient light intensity is lower, the aobvious of touch display screen 1505 is turned down Show brightness.In another embodiment, the ambient light intensity that processor 1501 can also be acquired according to optical sensor 1515, is moved The acquisition parameters of state adjustment CCD camera assembly 1506.

Proximity sensor 1516, also referred to as range sensor are generally arranged at the front panel of terminal 1500.Proximity sensor 1516 for acquiring the distance between the front of user Yu terminal 1500.In one embodiment, when proximity sensor 1516 is examined When measuring the distance between the front of user and terminal 1500 and gradually becoming smaller, by processor 1501 control touch display screen 1505 from Bright screen state is switched to breath screen state；When proximity sensor 1516 detect the distance between front of user and terminal 1500 by When gradual change is big, touch display screen 1505 is controlled by processor 1501 and is switched to bright screen state from breath screen state.

It, can be with it will be understood by those skilled in the art that the restriction of the not structure paired terminal 1500 of structure shown in Figure 15 Including than illustrating more or fewer components, perhaps combining certain components or being arranged using different components.

The embodiment of the present application also provides a kind of computer equipment, which includes memory and processor, storage Be stored at least one instruction, at least a Duan Chengxu, code set or instruction set in device, at least one instruction, an at least Duan Chengxu, Code set or instruction set are loaded by processor and realize the above-mentioned object processing method based on video.

The embodiment of the present application also provides a kind of computer readable storage medium, and at least one is stored in the readable storage medium storing program for executing Item instruction, at least a Duan Chengxu, code set or instruction set, at least one instruction, an at least Duan Chengxu, the code Collection or instruction set are loaded by the processor and are executed to realize the above-mentioned object processing method based on video.

Present invention also provides a kind of computer program products to make when computer program product is run on computers It obtains computer and executes the object processing method based on video that above-mentioned each embodiment of the method provides.

Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of above-described embodiment is can It is completed with instructing relevant hardware by program, which can be stored in a computer readable storage medium, the meter Calculation machine readable storage medium storing program for executing can be computer readable storage medium included in the memory in above-described embodiment；It can also be with It is individualism, without the computer readable storage medium in supplying terminal.Be stored in the computer readable storage medium to Few an instruction, at least a Duan Chengxu, code set or instruction set, it is at least one instruction, an at least Duan Chengxu, described Code set or instruction set are loaded by the processor and are executed to realize the above-mentioned object processing method based on video.

Optionally, the computer readable storage medium may include: read-only memory (ROM, Read Only Memory), Random access memory (RAM, Random Access Memory), solid state hard disk (SSD, Solid State Drives) or light Disk etc..Wherein, random access memory may include resistive random access memory body (ReRAM, Resistance Random Access Memory) and dynamic random access memory (DRAM, Dynamic Random Access Memory).Above-mentioned Apply for that embodiment sequence number is for illustration only, does not represent the advantages or disadvantages of the embodiments.

Those of ordinary skill in the art will appreciate that realizing that all or part of the steps of above-described embodiment can pass through hardware It completes, relevant hardware can also be instructed to complete by program, the program can store in a kind of computer-readable In storage medium, storage medium mentioned above can be read-only memory, disk or CD etc..

The foregoing is merely the preferred embodiments of the application, not to limit the application, it is all in spirit herein and Within principle, any modification, equivalent replacement, improvement and so on be should be included within the scope of protection of this application.

Claims

1. a kind of object processing method based on video, which is characterized in that the described method includes:

Crucial point tracking carried out to the target object, the target object is corresponding with n key point, in the n key point The m key point including the profile for constructing the target object, 0 < m≤n；

Key point adjustment signal is received, the key point adjustment signal, which is used to indicate, adjusts the profile of the target object It is whole；

It is adjusted according to position of the key point adjustment signal to the first key point in the m key point, institute's rheme It sets for indicating the relative position between first key point and other key points；

According to the first key point position adjusted, to profile corresponding with first key point on the target object Region is handled.

2. the method according to claim 1, wherein the reception key point adjustment signal, comprising:

The drag operation acted in first key point is received, the initial position of the drag operation is located at described first and closes Around key point in the first preset range；

The key point adjustment signal is generated according to the drag operation, includes that the dragging is grasped in the key point adjustment signal Make the end position of corresponding first key point and the drag operation.

3. according to the method described in claim 2, it is characterized in that, described a to the m according to the key point adjustment signal The position of the first key point in key point adjusts, comprising:

The first key point in the m key point is adjusted to the end position according to the key point adjustment signal；

The key point being located at around first key point in the second preset range in the m key point is carried out and described the The corresponding adjustment of one key point.

4. according to the method described in claim 3, it is characterized in that, described close in the m key point positioned at described first Key point around key point in the second preset range carries out adjustment corresponding with first key point, comprising:

The direction that initial position described in the drag operation is moved to the end position is determined as drawing direction, described Beginning position is determined as dragging distance to the distance between the end position；

To the second key point being located at around first key point in the second preset range, second key point and institute are determined State the positional relationship between the first key point；

Scaling is carried out with scaling ratio corresponding with the positional relationship to the dragging distance, obtains scaling distance；

The position of second key point is adjusted with the drawing direction and the scaling distance.

5. the method according to claim 1, wherein described a to the m according to the key point adjustment signal The position of the first key point in key point is adjusted, comprising:

The m key point is inputted into default adjustment model according to the key point adjustment signal；

The m key point after being adjusted by the default adjustment model output, wherein the default adjustment model is It is obtained by the corresponding key point training of sample object.

6. method according to any one of claims 1 to 5, which is characterized in that further include and the mesh in the n key point Mark the corresponding k key point of subject face, the key point adjustment signal be also used to indicate to the face of the target object into Row adjustment, 0 < k < n；

After the reception key point adjustment signal, further includes:

It is adjusted according to position of the key point adjustment signal to the third key point in the k key point；

According to the third key point position adjusted, the corresponding face feature of the third key point is adjusted.

7. according to the method described in claim 6, it is characterized in that, the method also includes:

The k key point is divided according to face feature position, each face feature position corresponding first is obtained and closes Key point set；

The first adjustment signal to target signature position is received, the first adjustment signal is used to indicate to the target signature portion Position carries out independent adjustment；

It is closed according to the 4th in the first adjustment signal first set of keypoints corresponding to the target signature position Key clicks through line position and sets adjustment；

According to the 4th key point position adjusted, the target signature position is adjusted.

8. method according to any one of claims 1 to 5, which is characterized in that the method also includes:

The n key point is divided according to body part, obtains corresponding second key point of each body part Set；

The second adjustment signal to target body portion is received, the second adjustment signal is used to indicate to the intended body portion Divide and carries out independent adjustment；

According to the 5th in the second adjustment signal pair second set of keypoints corresponding with the target body portion Key point carries out position adjustment；

According to the 5th key point position adjusted, the target body portion is adjusted.

9. method according to any one of claims 1 to 5, which is characterized in that described to carry out key point to the target object Tracking, comprising:

Obtain the key frame in the video flowing；

To in the key frame the target object carry out key point identification, obtain in every frame key frame with the target object Corresponding n key point, wherein there are corresponding relationships for the n key point in different key frames.

10. a kind of object handles device based on video, which is characterized in that described device includes:

Identification module, for carrying out crucial point tracking to the target object, the target object is corresponding with n key point, institute Stating includes m key point for constructing the profile of the target object in n key point, 0 < m≤n；

Receiving module, for receiving key point adjustment signal, the key point adjustment signal is used to indicate to the target object Profile be adjusted；

Adjust module, for according to the key point adjustment signal to the position of the first key point in the m key point into Row adjustment, the position are used to indicate the relative position between first key point and other key points；

The adjustment module, is also used to according to the first key point position adjusted, on the target object with it is described The corresponding contour area of first key point is handled.

11. device according to claim 10, which is characterized in that the receiving module, be also used to receive act on it is described The initial position of drag operation in first key point, the drag operation is located at the first default model around first key point In enclosing；The key point adjustment signal is generated according to the drag operation, includes the dragging in the key point adjustment signal Operate the end position of corresponding first key point and the drag operation.

12. device according to claim 11, which is characterized in that the adjustment module is also used to according to the key point Adjustment signal adjusts the first key point in the m key point to the end position；To being located in the m key point Key point around first key point in the second preset range carries out adjustment corresponding with first key point.

13. device according to claim 12, which is characterized in that the adjustment module, comprising:

Determine submodule, the direction for initial position described in the drag operation to be moved to the end position is determined as Drawing direction, the distance between described initial position to the end position are determined as dragging distance；

The determining submodule is also used to the second key point being located at around first key point in the second preset range, Determine the positional relationship between second key point and first key point；

The determining submodule is also used to put the dragging distance with scaling ratio corresponding with the positional relationship Contracting, obtains scaling distance；

Adjusting submodule, for being adjusted with the drawing direction and the scaling distance to the position of second key point It is whole.

14. a kind of computer equipment, which is characterized in that the computer equipment includes processor and memory, the memory In be stored at least one instruction, at least a Duan Chengxu, code set or instruction set, at least one instruction, an at least Duan Cheng Sequence, code set or instruction set are loaded by the processor and are executed as described in any one of claim 1 to 9 based on video to realize Object processing method.

15. a kind of computer readable storage medium, which is characterized in that be stored at least one finger in the readable storage medium storing program for executing Enable, at least a Duan Chengxu, code set or instruction set, at least one instruction, at least a Duan Chengxu, code set or instruction set by Processor is loaded and is executed to realize the object processing method as described in any one of claim 1 to 9 based on video.