CN110852934A

CN110852934A - Image processing method and apparatus, image device, and storage medium

Info

Publication number: CN110852934A
Application number: CN201810956680.2A
Authority: CN
Inventors: 李通; 钱晨; 刘文韬
Original assignee: Beijing Sensetime Technology Development Co Ltd
Current assignee: Beijing Sensetime Technology Development Co Ltd
Priority date: 2018-08-21
Filing date: 2018-08-21
Publication date: 2020-02-28

Abstract

The embodiment of the application discloses an image processing method and device, an image device and a storage medium. The method comprises the following steps: carrying out integral deformation processing on a target object of the image; performing a first local deformation process on a first portion of the target object; wherein the first local deformation process is configured to at least partially counteract a deformation of the first portion by the global deformation process.

Description

Image processing method and apparatus, image device, and storage medium

Technical Field

The present application relates to the field of information technology, and in particular, to an image processing method and apparatus, an image device, and a storage medium.

Background

The image deformation processing can be applied to the image beautifying or dramatizing processing process. For example, in the process of beautifying the image, the portrait in the image is subjected to slimming treatment.

However, in the prior art, a general algorithm is adopted for image deformation processing, and the general deformation processing performed by the general deformation algorithm cannot meet individual requirements of different users, and the image processing effect still cannot achieve the expected effect.

Disclosure of Invention

The embodiment of the application aims to provide an image processing method and device, an image device and a storage medium.

The technical scheme of the application is realized as follows:

in a first aspect, an embodiment of the present application provides an image processing method, including:

carrying out integral deformation processing on a target object of the image;

performing a first local deformation process on a first portion of the target object; wherein the first local deformation process is configured to at least partially counteract a deformation of the first portion by the global deformation process.

Based on the above scheme, the method further comprises:

and performing the first part second local deformation processing based on the original size of the first local.

Based on the above scheme, the method further comprises:

acquiring a first size parameter of a second part of the target object after the overall deformation treatment, wherein the second part is different from the first part;

determining a first deformation parameter according to the first size parameter;

the performing, based on the original size of the first part, the first partial second local deformation processing includes:

and performing the second local deformation treatment on the first part according to the first deformation parameter.

Based on the above scheme, the performing the first partial second local deformation processing based on the original size of the first local includes:

performing local deformation processing on the first part in a first direction;

performing local deformation processing on the first part in a second direction, wherein the second direction is perpendicular to the second direction; the deformation ratio in the first direction is equal to the deformation ratio in the second direction.

Based on the above scheme, the performing the local deformation processing on the first portion in the first direction includes:

moving coordinates of grid points of the warped mesh of the first portion in the first direction;

the performing local deformation processing on the first portion in a second direction includes:

moving coordinates of grid points of the warped mesh of the first portion in the second direction; and the second center point of the deformed grid of the first part after the coordinate movement in the second direction is superposed with the first center point of the deformed grid of the first part after the coordinate movement in the first direction.

Based on the scheme, the method comprises the following steps:

determining an overall deformation parameter of the overall deformation treatment according to the second dimension parameter of the first part;

the overall deformation processing of the target object of the image comprises the following steps:

and carrying out integral deformation processing on the target object according to the integral deformation parameters.

Based on the above scheme, the method further comprises:

acquiring the pose of the first part in the image;

and if the posture of the first part in the image is a first preset posture, determining the second size parameter according to the size of the first part in the image.

Based on the above scheme, the method further comprises:

if the posture of the first part in the image is a second preset posture, acquiring at least two alternative size parameters of the first part according to the incidence relation between the second part of the target object and the first part;

and selecting the second size parameter from the candidate size parameters according to a preset strategy.

Based on the scheme, the target object is a portrait;

the first portion includes: head portrait;

the second portion of the target object comprises: and the person image is other than the head image in the person image.

In a second aspect, an embodiment of the present application provides an image processing apparatus, including:

the integral deformation module is used for carrying out integral deformation processing on a target object of the image;

a first local deformation module for performing a first local deformation process on a first portion of the target object; wherein the first local deformation process is configured to at least partially counteract a deformation of the first portion by the global deformation process.

Based on the above scheme, the apparatus further comprises:

and the second local deformation module is used for performing the first part of second local deformation processing based on the original size of the first local.

Based on the above scheme, the apparatus further comprises:

a first obtaining module, configured to obtain a first size parameter of a second portion of the target object after the overall deformation process, where the second portion is different from the first portion;

the first determining module is used for determining a first deformation parameter according to the first size parameter;

the second local deformation module is specifically configured to perform the second local deformation processing on the first portion according to the first deformation parameter.

Based on the above scheme, the second local deformation module is specifically configured to perform local deformation processing on the first portion in the first direction; performing local deformation processing on the first part in a second direction, wherein the second direction is perpendicular to the second direction; the deformation ratio in the first direction is equal to the deformation ratio in the second direction.

Based on the above solution, the second local deformation module is specifically configured to move coordinates of grid points of the deformed grid of the first portion in the first direction; moving coordinates of grid points of the warped mesh of the first portion in the second direction; and the second center point of the deformed grid of the first part after the coordinate movement in the second direction is superposed with the first center point of the deformed grid of the first part after the coordinate movement in the first direction.

Based on the above scheme, the device comprises:

a second determining module, configured to determine an overall deformation parameter of the overall deformation process according to a second size parameter of the first portion;

and the integral deformation module is used for carrying out integral deformation processing on the target object according to the integral deformation parameters.

Based on the above scheme, the apparatus further comprises:

a second acquisition module, configured to acquire a pose of the first portion in the image;

a third determining module, configured to determine the second size parameter according to a size of the first portion in the image if the pose of the first portion in the image is a first preset pose.

Based on the above scheme, the third determining module is specifically configured to, if the posture of the first portion in the image is a second preset posture, obtain at least two candidate size parameters of the first portion according to an association relationship between the second portion of the target object and the first portion;

Based on the scheme, the target object is a portrait;

the first portion includes: head portrait;

In a third aspect, an embodiment of the present application provides a computer storage medium, where the computer storage medium stores computer executable code; after being executed, the computer executable code can implement the image processing method provided by any technical scheme of the first aspect.

In a fourth aspect, embodiments of the present application provide a computer program product comprising computer-executable instructions; after being executed, the computer-executable instructions can be used in the image processing method provided by any technical scheme of the first aspect.

In a fifth aspect, an embodiment of the present application provides an image apparatus, including:

a memory for storing information;

and the processor is connected with the memory and used for realizing the image processing method provided by any technical scheme of the first aspect by executing the computer-executable instructions stored on the memory.

According to the image processing method and device, the image equipment and the storage medium, the target object is subjected to overall deformation processing, and after the overall deformation processing, the first part is subjected to local deformation processing; compared with the first aspect in which each part in the second part except the first part is directly subjected to deformation processing one by one, the first aspect can reduce the number of times of deformation processing, reduce the calculation amount of deformation processing, and improve the deformation efficiency. In the second aspect, deformation processing is performed on all parts in the second part except the first part one by one, so that the problems that deformation amounts are inconsistent and deformation splicing marks before different parts are obvious due to the problems of deformation errors and the like in the one-by-one deformation processing are solved. In a third aspect, the deformation of the first part caused by the global deformation process can be at least partially counteracted by a first local deformation process of the first part; aiming at the first part which does not need to be deformed or has smaller deformation amount, the deformation can be reduced as much as possible, and even the independent deformation of the second part except the first part is realized, thereby meeting different deformation requirements of users for the image.

Drawings

Fig. 1 is a schematic flowchart of a first image processing method according to an embodiment of the present application;

fig. 2 is a schematic flowchart of a second image processing method according to an embodiment of the present application;

fig. 3A is a schematic diagram illustrating a size comparison of different parts according to an embodiment of the present disclosure;

fig. 3B is a schematic diagram illustrating an effect of the target object of fig. 3A after the target object is subjected to the overall deformation processing according to the embodiment of the present application;

FIG. 3C is a schematic illustration of a size comparison of different regions according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of an image transformation using a warped mesh according to an embodiment of the present application;

FIG. 5 is a schematic diagram of another image transformation using a warped mesh according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of another image apparatus according to an embodiment of the present application.

Detailed Description

The technical solution of the present application is further described in detail with reference to the drawings and specific embodiments of the specification.

As shown in fig. 1, the present embodiment provides an image processing method, including:

step S110: carrying out integral deformation processing on a target object of the image;

step S120: performing a first local deformation process on a first portion of the target object; wherein the first local deformation process is configured to at least partially counteract a deformation of the first portion by the global deformation process.

The image processing method provided by the embodiment can be a method applied to an image device. The image device may be various types of electronic devices; such as a stationary device or a mobile device. The fixing device may include: desktop computers or servers, etc. The mobile device may include: vehicle-mounted equipment, man-mounted equipment, robots, and the like. The human-borne device may include: a mobile phone, a tablet computer or a wearable device, etc.

The image in step S110 may be: an image to be processed; for example, raw images acquired by a camera; or an image to be morphed received from other electronic devices.

The target object can be the imaging of various acquisition objects; the acquiring the object may include: a human, an animal, a scene, or a vehicle, etc.

Taking a human as an example, the target object may be a human image. The figure may be divided into at least a first portion and a second portion other than the first portion. The first portion comprises at least a part of the portrait; the second portion may comprise at least a part of the portrait. In some embodiments, the first portion includes a smaller local number of figures than the second portion.

For example, the portrait may include: imaging of several parts of the head, neck, shoulders, chest, waist, abdomen, crotch, legs and hands; the first portion may include: head portrait formed by head imaging; the second portion includes imaging of all portions other than the head, for example, a portrait other than the avatar.

In other embodiments, the first portion may include: imaging of the head and neck, the second portion may comprise: imaging of other parts than the head and neck.

In other embodiments, the target object may also be a person (i.e., a portion other than the head); the first portion of the person may include: neck and/or limbs (hands and legs); the second portion may include: shoulder, chest, waist, abdomen, crotch, etc.

In some embodiments, the first portion may include a plurality of portions that are continuously distributed or discretely distributed.

For example, the first portion may be an avatar of a person's head; the second portion may be a portion other than an avatar, for example, an image of a person other than an avatar, including: imaging of parts of the shoulder, waist, crotch and legs.

In this embodiment, the overall deformation process may include: and carrying out deformation processing of size adjustment on the target object. In some embodiments, in step S110, the overall deformation processing may be performed according to a predetermined standard size parameter and a size parameter of a second portion currently presented in the image before deformation, so that a size of the second portion in the image is close to the standard size parameter, or the size parameter of the second portion in the image after the overall deformation processing is directly the standard size parameter.

The global deformation parameters may be: the overall deformation parameter may be, for example, a difference between the pre-deformation dimension parameter and the standard dimension parameter, which is determined according to the pre-deformation dimension parameter and the standard dimension parameter.

In other embodiments, the method further comprises: acquiring a first deformation instruction; the deformation instruction can be a first deformation instruction input by a user and received from a human-computer interaction interface, or the first deformation instruction is received from other equipment; the deformation instructions give the desired dimensional parameters. For example, the user indicates the desired size parameter by a finger operation on a slide operation on which the image is displayed. In some embodiments, the overall deformation parameter is determined based on the pre-deformation dimensional parameter and the desired dimensional parameter. For example, the dimensional parameter before deformation and the expected dimensional parameter are subjected to difference processing, and the overall deformation parameter is obtained through the difference processing.

In some embodiments, the method may further comprise: in this embodiment, the overall deformation parameter corresponding to the second deformation instruction may be directly left without detecting the size parameter of the second portion before deformation, so that the second portion exhibits the required size.

For example, the image apparatus displays an image, detects a pushing operation applied to the first image with a display interaction unit (e.g., a display touch screen or a display proximity screen), and determines the overall deformation parameter based on an operation parameter of the pushing operation. Determining a deformation amount of the overall deformation parameter, for example, based on a pushing distance of the pushing operation; and/or determining the size reduction and/or the direction of the overall deformation parameter based on the pushing direction of the pushing operation. For example, in a plane coordinate system with the central point of the image as the origin, a connecting line between a pushing starting point and a pushing end point of the pushing operation forms an angle of Y with the X axis of the plane coordinate system where the image is located; the size of the target object is reduced or enlarged in the Y-angle based on the global deformation parameter.

The display touch screen needs an operation body such as a user finger or a touch pen to directly contact the screen, and the pushing operation can be detected; the pushing operation can be detected when the display proximity screen operable body is close enough to the screen, and the operation body is not required to be attached to the screen.

In such an embodiment, therefore, the overall deformation parameter is determined according to the operating parameter corresponding to the second deformation instruction, independently of the dimensional parameter before deformation.

The value of the integral deformation parameter can be positive or negative; and if the value of the overall deformation parameter is positive, the size of the second part is increased, the value of the overall deformation parameter is negative, the size of the second part is decreased, and if the value of the overall deformation parameter is negative, the size of the second part is increased, the value of the overall deformation parameter is positive, the size of the second part is decreased.

In other embodiments, the method further comprises:

and acquiring a first morphological parameter of the second part, wherein the first morphological parameter can be used for characterizing the morphology of the second part in the image. The different forms of the second part in the image have different corresponding sizes in the image coordinate system corresponding to the image.

In this embodiment, when performing the overall size deformation processing, it is necessary to determine the standard size parameter and/or the expected size parameter according to the presented form in consideration of the form of the second part presented in the image, so that the current overall deformation processing makes the second part present the size desired by the user on the one hand, and the presented size is adapted to the form of the second part in the image on the other hand, which can improve the image processing effect.

While the above embodiments describe the overall deformation process being used to perform the size adjustment on the entire target object, in other embodiments, the overall deformation process may also perform the overall shape adjustment process on the target object, for example, the lines of the entire face may be made to have different line shapes by the overall deformation process.

In the global deformation process, the image device deforms all parts of the target object at the same time, for example, simultaneously deforms the first part and the second part of the target object, so that both the first part and the second part of the target object are deformed, but the deformation of the first part may not be desired.

However, in some specific implementations, the first portion is the portion that needs to be maintained and the second portion is the portion that needs to be deformed; alternatively, the amount of deformation required for the first portion is less than the amount of deformation required for the second portion.

For example, the user a may think that his body is somewhat fat but his head is not fat, and thus a deformation process of slimming the body in his picture is generated. If each part of the whole body is adjusted one by one in the slimming process, on one hand, the whole slimming effect is poor due to the problem of different slimming degrees caused by different local adjustments, and on the other hand, each part of the body can be processed one by one, so that the processing time delay of the image equipment is long, and the delay of a user waiting to see an ideal photo is long. However, with the image processing method provided by this embodiment, an overall deformation process may be adopted, which is to perform a tuning process on the entire portrait first, and then combine with the deformation process of the avatar, so that the avatar is restored to the original size in the image, thereby achieving the effect of simply slimming the portrait in the portrait. On one hand, the requirement of only carrying out slimming treatment on the human body direction is met, and on the other hand, the method has the characteristic of small treatment time delay.

In some embodiments, the first portion may be a predefined portion.

Since the second part includes a large number of parts, which requires a uniform deformation process, and if the parts are processed one by one, which may result in a large amount of data processing and a low efficiency, in this embodiment, the entire target object is deformed first, and since the first part includes a small number of parts, it is a relatively simple change to perform the local deformation process to restore the original image effect or to cancel the deformation caused by the entire deformation process. Through the deformation caused by the integral deformation processing and the local at least partial restoration of the integral deformation processing, on one hand, the requirement on only local deformation processing can be met, and on the other hand, the integral deformation processing and the local callback or restoration processing are carried out, so that compared with the method that deformation processing is carried out on each local part in the second part one by one, the method has the characteristics of small data processing amount and high data processing effect on the whole.

In some embodiments, the first portion may be smaller in rate of change in the shape of the acquisition object than the second portion. For example, in the case of a human, the fatness and thinness of the human may be more apparent in the body, while the head is less apparent. Taking the human body as an example, the fatness and thinness of the human body may be more obvious in the waist, abdomen and/or buttocks, but may not be obvious in the limbs.

In order to realize the deformation of the portrait, for example, adjusting the corresponding thickness of the portrait, if the places needing to be adjusted are adjusted one by one, the calculated amount is very large, and the efficiency is low; therefore, in this embodiment, the whole image can be adjusted by using the whole deformation process; then the original appearance of the head is restored or the deformation of the part is offset, so that the independent deformation of the second part can be realized; through whole deformation processing, combine the first local deformation processing of first part again, when having satisfied different deformation demands, can also realize the technological effect that the deformation effect is high.

In some embodiments, as shown in fig. 2, the method further comprises:

step S130: and performing the first part second local deformation processing based on the original size of the first local.

The restoration of the first portion is achieved by the first local deformation process, but the user may also need to deform the first portion. For example, taking a portrait as an example, if the head portrait is the first part; the person is imaged as the second portion. If the first part and the second part are subjected to overall thickness adjustment through overall deformation processing, the first part is restored to the original state of the head portrait through the first local deformation processing, which is equivalent to offsetting the deformation of the first part by the overall deformation processing, and the deformation of the first part by the overall deformation processing is offset through the first local deformation processing, so that the distortion of the first part caused by the overall deformation processing is avoided. In this way, the second local deformation process can be further processed on the distorted first portion as small as possible, so as to improve the deformation effect of the first portion after passing through the second local deformation process.

Fig. 3A is an image to be processed in step S110, fig. 3B is an image subjected to an overall deformation process, and the target image shown in fig. 3B is reduced in overall weight compared to the image shown in fig. 3A. But after the head shown in fig. 3B is thinned, the head is visually elongated. Fig. 3C is a view of restoring the head shown in fig. 3C to the original size with respect to the head in fig. 3A by a first local deformation process.

For example, the first part is used as the avatar, the second part is used as the portrait except the avatar, the first part and the second part are simultaneously subjected to deformation processing such as slimming adjustment by the global deformation processing, and then the deformation of the avatar caused by the global deformation processing is restored by the first local deformation processing, so that the avatar part is restored to the original size, contour and the like by the first local deformation processing. The second local deformation process is performed on the avatar again, and for example, the head is scaled up or down. Here, the scaling may include: and carrying out enlargement or reduction on the head portrait by adopting an equal proportion deformation mode on the first direction and the second direction of the head portrait.

Taking the example of the deformed mesh being deformed, the grid points formed by intersecting the horizontal lines and the vertical lines of the deformed mesh may be moved in the horizontal direction or in the vertical direction. The overall deformation process may further move in one direction. For example, the overall deformation processing is to make the human image slim or plump in the lateral direction, and if the overall deformation processing is performed using the deformed mesh, the control point may move only in the lateral direction. However, the first part and the second part move together in the transverse direction, and the proportion of the head portrait and the person portrait can still be maintained to be disordered; the whole deformation treatment is used for slimming, so that the head bag with the original normal size has an elongation effect in the longitudinal direction. The avatar can be restored to the original size in the lateral direction by the first local deformation process in the lateral direction in this embodiment. If fine adjustment of the avatar is required, a second local adjustment of the avatar may be performed by scaling or the like in order to maintain the original ratio of the avatar in the lateral and longitudinal directions.

The deformed mesh is displayed in the images in fig. 4 and 5, the entire target object is covered with the deformed mesh in the left part of fig. 4 and 5, and the entire deformation process is performed on the target object by the deformed mesh.

In the middle portion of fig. 4, the display parameters of the deformed mesh of the head and the second part other than the body as the first part are different. The body is displayed in grey as a deformed grid of the second part, indicating that the deformed grid cannot be manipulated, thus shielding deformation of other parts outside the first part. The right part of fig. 4 is a schematic view of the first part after the first local deformation processing.

In the middle part of fig. 5, the deformed mesh is still covered as the first part, and the deformed mesh is not covered in other parts, so that the deformation of other parts except the first part can be shielded. The right part of fig. 5 is a schematic view of the first part after the first local deformation processing.

In the first local deformation process performed in fig. 3B, the deformed mesh may be overlaid only on the deformed first portion.

It is worth noting that: in some embodiments, the step S110 and the step S120 may be executed synchronously, or may be executed sequentially, for example, as shown in fig. 1, the step S110 is executed first, and after the overall deformation processing is executed, the first local deformation processing is executed. In other embodiments, the step S110 and the step S120 may be performed synchronously, for example, a first deformation network is used to perform an overall deformation process, a second deformation grid is used to perform a first local deformation process, the deformed first deformation grid and the second deformation network are finally fused to obtain a third deformation grid, and a pixel coordinate of the image and a pixel value are mapped based on the third deformation grid, so that the image processing efficiency is improved by parallel processing, and the pixel coordinate mapping and the pixel value determination are performed only once, which may reduce unnecessary computation; the processing load of the image device is reduced.

Similarly, step S130 is executed after step S120 is executed as shown in fig. 2, or may be executed in parallel with step S110 or step S120. For example, a second local deformation is performed using a fourth deformation grid; and combining the deformed first deformed grid, the deformed second deformed grid and the deformed fourth deformed grid, fusing to obtain a fifth deformed grid, and performing pixel coordinate mapping and pixel value determination on the image based on the fifth deformed grid.

In some embodiments, the method further comprises:

the step S130 may include:

In this embodiment, the first size parameter of the second portion of the target object after the overall deformation may include, but is not limited to: the height of the second portion, the width of the second portion. In this embodiment, the second portion is different from the first portion. For example, the part of the portrait comprised by the second portion is different from the part of the portrait comprised by the first portion.

The first dimensional parameter may include: various information describing the dimensions of the second portion in the first direction and/or the second direction. The second direction may be perpendicular to the first direction. For example, if the first direction is a transverse direction, the second direction is a longitudinal direction; if the first direction is longitudinal, the second direction is transverse.

The first dimensional parameter may include at least one of:

an average width of the second portion after the entire deformation in the first direction;

a minimum width of the entirely deformed second portion in the first direction;

a maximum width of the second portion in the first direction after the entire deformation;

the overall length of the second portion in the second direction after the overall deformation;

the length of the different parts of the second portion after the global deformation in said second direction.

In this embodiment, before the second local deformation process is performed, a first size parameter of the second portion after the overall deformation process is determined, and a first deformation parameter for the second local deformation of the first portion is determined according to the first size parameter. For example, through the whole-body slimming process, a fat-thin parameter of the imaging of the body in the portrait may be taken as the first size parameter. The size of the head portrait in the image needs to be matched with the size of the second local part through the second local processing, so that the first deformation parameter is determined by imaging the body after slimming in the embodiment, so that the size of the head portrait after the second local processing is proportional to the imaging of the body, and the imaging of the head and feet with light head and large feet or the imaging of the head and feet with small head and large feet is avoided; so as to improve the image quality after the deformation processing.

In the embodiment of the present invention, each of the dimensions referred to by the first dimension parameter may be a euclidean distance between pixel coordinates of the end portions of the corresponding portions, but is not limited to the euclidean distance.

In some embodiments, the step S130 may specifically include:

performing local deformation processing on the first part in a first direction;

The local deformation process in this embodiment may include a local deformation in a first direction and a local deformation in a second direction. In the second local deformation processing in this embodiment, the local deformation processing in the first direction and the local deformation processing in the second direction are both considered, and the deformation amount in the first direction is equal to the deformation amount in the second direction, so that the equal-proportion deformation in the two perpendicular directions of the first portion can be realized.

In some embodiments, the overall deformation process may be a deformation process in one direction, for example, a deformation process in a first direction, or a deformation process in a second direction. However, the first part needs to be deformed in two directions, so as to avoid the whole deformation process from deforming the first part in one direction in advance, the first part is restored to the original state of the first part through the first local deformation process, and then the first part is processed through the local deformation processes performed in two perpendicular directions, on one hand, the first part is prevented from introducing strange deformation to the maximum extent, so that the image after the deformation process is imaged naturally, on the other hand, the subsequent direct operation is equal to the deformation in two perpendicular directions through the deformation restoration of the first part of the first local deformation process, and the accuracy is higher and simpler and more convenient. For example, the local deformation process in the first direction and the second direction may be converted into a deformation process in an angular bisector direction of the first direction and the second direction, such that the respective local deformation processes in the first direction and the second direction are mentioned by a primary deformation process in which the deformation direction is a direction on the angular bisector, thereby simplifying the deformation process and increasing the deformation process rate.

In some embodiments, the performing the local deformation process on the first portion in the first direction includes: moving coordinates of grid points of the warped mesh of the first portion in the first direction; the performing local deformation processing on the first portion in a second direction includes: moving coordinates of grid points of the warped mesh of the first portion in the second direction; and the second center point of the deformed grid of the first part after the coordinate movement in the second direction is superposed with the first center point of the deformed grid of the first part after the coordinate movement in the first direction.

The grid points in the deformed grid are control points for deformation processing, and the coordinate change of the control points directly determines the conversion of the pixel coordinates of the pixels in the grid where the frame of control points are located. In this embodiment, when the local deformation processing in the first direction and the local deformation processing in the second direction are performed, they may be performed separately; in order to ensure that the last deformation amounts in the first direction and the second direction are consistent, the deformation amounts are determined by the coincidence of the center points in the first direction and the second direction after the movement.

In some embodiments, the method comprises:

the step S110 may include:

In some embodiments, the first portion may be a portion that the user needs to keep unchanged, and thus, in this embodiment, in order to make the size of the second portion after the overall deformation match the size of the first portion, the overall deformation parameter is determined according to the second size parameter of the first portion in this embodiment; the overall deformation parameter is used for carrying out overall deformation processing on the target object.

In some embodiments, the step S130 may include: and performing the first local deformation processing on the first part according to the overall deformation parameter, specifically, performing the first local deformation processing on the first part according to the deformation amount in the overall deformation parameter, so as to recover the deformation of the first part caused by the overall deformation processing by using the first local deformation processing. In this way, the overall deformation processing is used in the overall deformation processing and the first local deformation processing, so that the deformation parameters do not need to be determined again in the first local deformation processing, and the multiplexing of the parameters is realized; the processing of the image device is simplified.

In some embodiments, the method further comprises:

acquiring the pose of the first part in the image;

In the present embodiment, when determining the second size parameter, the pose of the first portion in the image is considered, and if the pose of the first portion in the image is different, the size of the first portion in the image is also different, so that the actual size parameter of the first portion needs to be determined in combination with the pose.

In some embodiments, to reduce the accuracy, the second size parameter is determined based on a size of the first portion in the image only when the first portion is posed in the image in the first preset pose. For example, taking a human image as an example, if the front face of the human face faces the front photograph acquired by the camera and containing the front face of the human face, that is, the pose of the first part in the image is the front pose, the second size parameter is determined only according to the size of the first part in the front pose in the image.

In some embodiments, if the posture of the first part in the image is a second preset posture, obtaining at least two candidate size parameters of the first part according to the association relationship between the second part of the target object and the first part;

The second preset posture is different from the first preset posture, and the second preset posture can be other than the first preset posture. Since the pose of the first portion in the image is the second preset pose, and the second size parameter may have an insufficient accuracy if the second size parameter is determined directly according to the size of the first portion in the image, in this embodiment, an association relationship between the second portion and the first portion is introduced to obtain the candidate size parameters of the first portion, for example, at least two candidate size parameters may be obtained, and one second size parameter is selected from the candidate size parameters. For example, taking a portrait as an example, a person may take a photograph in a different pose, and the second portion may include: a plurality of parts such as shoulders, waist, span, legs and the like, which have a proportion relation with the head part in the first part relative to a fixed range, so that one or more candidate size parameters can be estimated according to the proportion relation between one or more parts in the second part and the head part, and then a better candidate size parameter is selected as the second size parameter. For example, a person assumes different poses, but the relative positions between different parts and the camera are different. In this embodiment, an alternative size parameter converted from a proportional relationship between the front-collected part and the head may be selected as the second size parameter. The front side acquisition here may include: the predetermined surface of the acquired part is parallel or approximately parallel to the acquisition surface of the camera, where the approximately parallel may be that the minimum angle formed by the two surfaces is not greater than a preset angle, for example, 10 degrees or 5 degrees. For example, the predetermined surfaces corresponding to when the front capture head obtains the front capture avatar may be: the indication of the five sense organs. The predetermined surface corresponding to the waist being collected frontally may be the widest surface of the waist. Of course, the above is only an example, and the specific implementation manner is various and is not limited to any one of the above.

In some embodiments, the target object may be a human image; the first portion includes: head portrait; the second portion of the target object comprises: and the person image is other than the head image in the person image.

In some embodiments, the step S130 may include:

and if the image deformation mode is the joint deformation mode, performing the first part of second local deformation processing based on the original size of the first local.

If the image deformation mode is the independent deformation mode, the step S130 is not executed.

In this manner, the image device can automatically determine whether the second local deformation processing needs to be further performed according to the image deformation mode. If the image deformation mode is the independent mode, after the step S120 is executed, the present deformation processing is automatically ended, and the image generated after the second local deformation processing is directly output. If the image deformation mode is the joint deformation mode, after step S120 is completed, the image apparatus enters the processing ready state of step S130, and in this ready state, the second local deformation processing may be performed according to a third deformation instruction input by the user. Alternatively, if the image deformation processing model is the joint deformation mode, after step S130, the image device may determine to execute the second local deformation processing based on a standard size parameter or the like. The image subjected to the entire morphing map processing, the second partial morphing processing, and the third partial morphing processing is output after step S130.

The ready state of the second local deformation process may include:

the image device activates a first grid area in the deformed grid corresponding to the first part, and grid points in the first grid area can be adjusted, so that a user can conveniently and directly adjust coordinates of the grid points to realize second local deformation processing of the first part.

Meanwhile, the method further comprises the following steps:

in order to ensure that the second part is deformed again in the process of carrying out the second local deformation processing on the first part, the second grid area in the deformed grid corresponding to the second part is deactivated.

Deactivating the second grid area may include: hiding the second grid area and/or prohibiting a move operation of grid points in the second grid area.

In this way, after the image device enters the ready state of the second local deformation processing, at least the following display interfaces can be presented:

the first method comprises the following steps: the first part and the second part on the image are covered with the deformed grids displayed with the same display parameters, but only grid points in the first grid area are movable, and grid points in the second grid area are not movable;

and the second method comprises the following steps: covering a first part on the image with the deformed grid, and not covering a second part on the image with the deformed grid;

and the third is that: the warped mesh is overlaid on the image with the first display parameter on the first portion and the warped mesh is overlaid with the second display parameter on the second portion. The first display parameter is different from the first display parameter; the grid points in the warped grid displayed with the second display parameter may not be moved. The display effect of the deformed grids covered on the first part and the second part is different through the difference of the first display parameter and the second display parameter, for example, the display color is different, the line thickness of the deformed grids is different, or the forms of the grid points are different; some grid points are circular, some are rectangular, etc. Therefore, through the distinguishing of the first display parameter and the second display parameter, the user can know which grid points in the deformed grid can be moved conveniently.

In some embodiments, the method further comprises:

determining the image deformation mode.

There are several ways to determine the image deformation pattern, and several alternatives are provided below:

the image deformation mode is determined according to a selection instruction input by a user, for example, before the image deformation processing, several deformation processing options are displayed in a dialog box or the like, and the image deformation mode is determined according to the deformation processing option selected by the user. For example, taking a portrait as an example, if it is detected that the user selects slimming, it may be considered that the current image deformation mode may be the independent deformation mode. If the integral beautification processing is detected to be selected, the image deformation mode can be considered as the joint deformation mode.

In some embodiments, the joint deformation mode and the independent deformation mode may be switched during the deformation process. For example, the joint deformation mode is selected by default, and if the third deformation instruction input by the user is not detected within a specified time after the overall deformation processing and the first local deformation processing are completed, the image device automatically switches from the joint deformation mode to the independent deformation mode, and does not perform step S130 any more; and the selection of such independent deformation mode is also used for subsequent adjustment of the image; thereby again promoting the intelligence of the image device and the satisfaction degree of the user experience.

In some embodiments, the step S110 may include: and mapping the whole pixel coordinates of the target object by using a deformed grid and a first deformed difference algorithm.

The step S120 may include: local pixel coordinate mapping of the first part is carried out by utilizing a deformed grid and a second deformed difference algorithm, so that pixel coordinate mapping is carried out again on the pixel coordinate of the first part after the whole pixel coordinate mapping;

the first and second deformation difference algorithms may be collectively referred to as deformation difference algorithms. The grid points of the warped mesh will be subjected to pixel coordinate mapping, and the pixels associated with the grid points also need to be subjected to pixel coordinate mapping, but the association relationship between the coordinate mapping of different pixels and the coordinate mapping of grid points is determined by the warped difference algorithm. In this way, the smoothness between the pixels of the target pixel coordinates after the mapping of the pixels can be realized by the deformed difference algorithm.

In some embodiments, the first and second deformation difference algorithms may be the same or different.

In some embodiments, the pixel difference algorithm may be selected according to the number of pixel values or corresponding image region areas that need to be determined. For example, if the image area of the second portion is larger than the image area of the first portion, or the second portion includes more pixel values than the first portion, the deformed difference algorithm suitable for more pixels may be preferentially selected, for example, the first difference algorithm may be a bezier curve algorithm; the second difference algorithm may be a spline curve algorithm.

During the global deformation process and the first local deformation process or the local deformation process (e.g., the first local deformation process and/or the second local deformation process), not only the pixel coordinate mapping but also the determination of the pixel value corresponding to the target pixel coordinate after the pixel coordinate mapping to the target pixel coordinate is involved. For example, in the process of slimming, because the size of the target object is reduced, some pixels included in the target object may need to be removed, and pixels in other image areas except the target object are increased, at this time, the pixel values of the pixels are removed, and the pixel values of the increased pixels are what; may be determined using a pixel difference algorithm.

The step S110 may further include: and determining the target pixel value of the second part by using a pixel difference algorithm and the original pixel value of the second part. The original pixel value and the target pixel value may be collectively referred to as a pixel value, and the pixel value determines various parameters such as a display color and a display luminance of the corresponding pixel. The difference value processing of the pixel values is carried out through a pixel difference value algorithm, so that the pixel smoothing processing can be realized, and the phenomenon of poor image effect caused by the fact that adjacent pixel values cannot be smoothly transited due to pixel coordinate mapping and/or size change and the like is reduced.

The step S120 may include:

and determining a target pixel value of the first part by utilizing a pixel difference algorithm and the original pixel value of the first part.

In the course of performing the global deformation processing and the local deformation processing, in order to reduce the amount of calculation, determination of the target pixel value is performed only once for both the first part and the second part. For example, if the first part may undergo at least the global deformation process and the first local deformation process, the final determination of the target pixel value is finally performed by using the corresponding pixel difference algorithm after the final mapping of the pixel coordinates is completed by the deformation grid. For another example, in some embodiments, if the first portion may undergo the global deformation process, the first local deformation process, and the second local deformation process, and finally, after the final mapping of the pixel coordinates of the second local deformation process is completed, the target pixel value is finally determined by using the pixel difference algorithm. The method has the characteristics of less calculation amount and high efficiency compared with the method that the target pixel value is determined once every time the pixel coordinate transformation is performed.

Further, in some embodiments, the method may further comprise: the determination of the pixel values of other graphical objects in the image than the target object after the deformation process may also be performed by a pixel difference algorithm.

For example, for inserting a pixel between two original adjacent pixels, the pixel value of the inserted pixel may be determined according to the pixel values of the original adjacent two pixels, for example, taking the average value or the median value of the pixel values of the original adjacent two pixels, so as to ensure the smoothness of the image maturity value after inserting a new pixel.

As shown in fig. 6, the present embodiment provides an image processing apparatus including:

an integral deformation module 110, configured to perform integral deformation processing on a target object of an image;

a first local deformation module 120 configured to perform a first local deformation process on a first portion of the target object; wherein the first local deformation process is configured to at least partially counteract a deformation of the first portion by the global deformation process.

In some embodiments, the global deformation module 110 and the first local deformation module 120 may be program modules, and the program modules are capable of executing the first deformation process, the determination of the first deformation, and the second deformation process after being executed by a processor.

In other embodiments, the global deformation module 110 and the first local deformation module 120 may correspond to a combined module of hardware and software, for example, may correspond to a complex programmable device or a field programmable device.

In some embodiments the apparatus further comprises:

In other embodiments, the apparatus further comprises:

In some embodiments, the second local deformation module is specifically configured to perform local deformation processing on the first portion in a first direction; performing local deformation processing on the first part in a second direction, wherein the second direction is perpendicular to the second direction; the deformation ratio in the first direction is equal to the deformation ratio in the second direction.

In some embodiments, the second local deformation module is specifically configured to move coordinates of grid points of the warped mesh of the first portion in the first direction; moving coordinates of grid points of the warped mesh of the first portion in the second direction; and the second center point of the deformed grid of the first part after the coordinate movement in the second direction is superposed with the first center point of the deformed grid of the first part after the coordinate movement in the first direction.

In other embodiments, the apparatus comprises:

the integral deformation module 110 is configured to perform integral deformation processing on the target object according to the integral deformation parameter.

In still other embodiments, the apparatus further comprises:

In other embodiments, the third determining module is specifically configured to, if the posture of the first portion in the image is a second preset posture, obtain at least two candidate size parameters of the first portion according to an association relationship between a second portion of the target object and the first portion;

In other embodiments, the target object is a portrait;

the first portion includes: head portrait;

As shown in fig. 7, the present embodiment provides an image apparatus including:

a memory;

and the processor is connected with the memory and used for realizing the image processing method provided by one or more of the foregoing embodiments by executing the computer executable instructions on the memory, for example, one or more of the image processing methods shown in fig. 1 and fig. 2.

The memory can be various types of memories, such as random access memory, read only memory, flash memory, and the like. The memory may be used for information storage, e.g., storing computer-executable instructions, etc. The computer-executable instructions may be various program instructions, such as object program instructions and/or source program instructions, and the like.

The processor may be various types of processors, such as a central processing unit, a microprocessor, a digital signal processor, a programmable array, a digital signal processor, an application specific integrated circuit, or an image processor, among others.

The processor may be connected to the memory via a bus. The bus may be an integrated circuit bus or the like.

In some embodiments, the image device may further include: a communication interface, which may include: a network interface, e.g., a local area network interface, a transceiver antenna, etc. The communication interface is also connected with the processor and can be used for information transceiving.

In some embodiments, the electronic device also includes a human interaction interface, which may include various input and output devices, such as a keyboard, a touch screen, and the like, for example.

The present embodiments provide a computer storage medium having stored thereon computer-executable instructions; the computer-executable instructions, when executed, enable the image processing methods provided by one or more of the foregoing embodiments, for example, one or more of the image processing methods shown in fig. 1 and 2.

The computer storage medium may be various recording media including a recording function, for example, various storage media such as a CD, a floppy disk, a hard disk, a magnetic tape, an optical disk, a usb disk, or a removable hard disk. Optionally, the computer storage medium may be a non-transitory storage medium, and the computer storage medium may be readable by a processor, so that after the computer executable instructions stored in the computer storage mechanism are acquired and executed by the processor, the information processing method provided by any one of the foregoing technical solutions can be implemented, for example, the information processing method applied to the terminal device or the information processing method applied to the application server is executed.

The present embodiments also provide a computer program product comprising computer executable instructions; the computer-executable instructions, when executed, enable the image processing methods provided by one or more of the foregoing embodiments, for example, one or more of the image processing methods shown in fig. 1 and 2.

The computer program comprises a computer program tangibly embodied on a computer storage medium, the computer program comprising program code for executing the method illustrated in the flowchart, the program code may include instructions corresponding to the execution of the method steps provided by the embodiments of the present application.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, all functional units in the embodiments of the present application may be integrated into one processing module, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method, comprising:

carrying out integral deformation processing on a target object of the image;

2. The method of claim 1,

the method further comprises the following steps:

3. The method of claim 2,

the method further comprises the following steps:

4. The method of claim 2,

performing local deformation processing on the first part in a first direction;

5. The method of claim 4,

the performing local deformation processing on the first part in a first direction includes:

6. The method according to any one of claims 1 to 5, characterized in that it comprises:

7. The method of claim 6, further comprising:

acquiring the pose of the first part in the image;

8. An image processing apparatus characterized by comprising:

9. A computer storage medium having computer executable code stored thereon; the computer executable code, when executed, is capable of implementing the method as provided by any one of claims 1 to 7.

10. An image device, comprising:

a memory for storing information;

a processor coupled to the memory for enabling implementation of the method provided in any one of claims 1 to 7 by executing computer executable instructions stored on the memory.