WO2021109863A1

WO2021109863A1 - Photo processing method and photo processing apparatus

Info

Publication number: WO2021109863A1
Application number: PCT/CN2020/129181
Authority: WO
Inventors: 乔宇; 李英; 王锴; 彭小江
Original assignee: 深圳先进技术研究院
Priority date: 2019-12-04
Filing date: 2020-11-16
Publication date: 2021-06-10
Also published as: CN111062279A; CN111062279B

Abstract

A photo processing method and a photo processing apparatus, which are applicable to the technical field of image processing. The photo processing method comprises: obtaining a photo to be processed, and capturing a plurality of sub-images from the photo to be processed, the sub-images comprising at least one portrait (S201); respectively calculating a group cohesion degree corresponding to each sub-image, the group cohesion degree being used for representing a cohesion degree between portraits in the sub-image (S202); and determining a sub-image corresponding to the highest group cohesion degree as the processed photo (S203). By means of the method, the composition of a figure group photo can be effectively adjusted, and then the integrity and coordination of a figure group photo picture are improved.

Description

Photo processing method and photo processing device

Technical field

This application belongs to the field of image processing technology, and in particular relates to a photo processing method and a photo processing device.

Background technique

With the continuous improvement of image processing technology, the functions of photographing equipment have gradually become stronger, and users have higher and higher requirements for photos. Taking a group photo is a common shooting behavior. In a group photo, not only the expression of each character, but also the relative position between the characters and the relative proportion between the characters and the background, etc., are the composition of the picture. If there is a problem with the composition, it will affect the integrity and coordination of the photo frame.

In the prior art, the processing method for group photos of people is relatively simple. Generally, it only improves the sharpness of the photos and removes obstructions in the photos. The composition of the group photos is not adjusted, and the integrity of the photo frame cannot be guaranteed. And coordination.

technical problem

The embodiments of the present application provide a photo processing method and a photo processing device, which can solve the problem of poor picture integrity and coordination of existing group photos of people.

Technical solutions

In the first aspect, an embodiment of the present application provides a photo processing method, including:

Acquiring a photo to be processed, and intercepting multiple sub-images from the photo to be processed, the sub-images containing at least one portrait;

Separately calculating the group cohesion degree corresponding to each sub-image, where the group cohesion degree is used to characterize the degree of cohesion between the individual portraits in the sub-image;

Determine the sub-image corresponding to the highest group cohesion as the processed photo.

In a possible implementation manner of the first aspect, the intercepting multiple sub-images from the photo to be processed includes:

Determining the smallest rectangle containing all human figures in the photo to be processed, and intercepting the image corresponding to the smallest rectangle to obtain the first sub-image;

Determining the largest rectangle in the photo to be processed, and intercepting an image corresponding to the largest rectangle to obtain a second sub-image, wherein the largest rectangle and the smallest rectangle have the same center and aspect ratio;

N sub-images are captured based on the maximum rectangle and the minimum rectangle, the N sub-images and the minimum rectangle have the same center and aspect ratio, and any sub-image in the N sub-images The area is larger than the area of the smallest rectangle and smaller than the area of the largest rectangle;

Wherein, the multiple sub-images include the first sub-image, the second sub-image, and the N sub-images.

In a possible implementation manner of the first aspect, the determining the smallest rectangle containing all human figures in the to-be-processed photo includes:

Perform portrait recognition on the photo to be processed, and obtain the coordinates of each pixel corresponding to the recognized portrait;

According to the coordinates, respectively determine the edge position point corresponding to each side of the photo to be processed, wherein the edge position point corresponding to one side is the pixel with the shortest distance from the one side among all the pixels corresponding to the portrait point;

The minimum rectangle is determined according to the edge position points corresponding to each side of the photo to be processed.

In a possible implementation manner of the first aspect, the capturing N sub-images based on the largest rectangle and the smallest rectangle includes:

Obtain N preset ratios, and scale the smallest rectangles according to each preset ratio to obtain N middle rectangles. The middle rectangle and the smallest rectangle have the same center, and the middle rectangle has the same center. The area is smaller than the area of the largest rectangle;

The images corresponding to each middle rectangle are respectively intercepted to obtain N sub-images.

One possible implementation of the first aspect of the image are calculated for each sub-group corresponding to the degree of aggregation, comprising:

For each sub-image, calculate the overall scene cohesion, face cohesion, and body cohesion corresponding to the sub-image, where the overall scene cohesion is used to characterize the relationship between the portrait and the background in the sub-image. Cohesion, where the face cohesion is used to characterize the facial expression of each person in the sub-image; the body cohesion is used to characterize the body posture of each person in the sub-image;

The group cohesion degree corresponding to the sub-image is calculated according to the overall scene cohesion degree, the face cohesion degree and the body cohesion degree.

In a possible implementation of the first aspect, the separately calculating the overall scene cohesion, face cohesion, and body cohesion corresponding to the sub-images includes:

Acquiring a trained neural network, the neural network including three sub-networks, and the three sub-networks are respectively used to calculate the overall scene cohesion, the face cohesion, and the body cohesion;

The sub-image is input into the neural network for processing, and the overall scene cohesion, face cohesion, and body cohesion corresponding to the sub-image are obtained.

In a possible implementation of the first aspect, the calculation of the group cohesion corresponding to the sub-image according to the overall scene cohesion, face cohesion, and body cohesion includes:

The overall scene cohesion, the face cohesion, and the body cohesion are weighted and summed according to preset weights to obtain the group cohesion.

In the second aspect, an embodiment of the present application provides a photo processing device, including:

An acquiring unit, configured to acquire a photo to be processed, and to intercept multiple sub-images from the photo to be processed, the sub-images containing at least one portrait;

A calculation unit, configured to separately calculate a group cohesion degree corresponding to each sub-image, where the group cohesion degree is used to characterize the degree of cohesion between the individual portraits in the sub-image;

The processing unit is used to determine the sub-image corresponding to the highest group cohesion, which is the processed photo.

In a third aspect, an embodiment of the present application provides a photo processing device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes The computer program implements the photo processing method according to any one of the above-mentioned first aspects.

In the fourth aspect, an embodiment of the present application provides a computer-readable storage medium, and an embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and is characterized in that the When the computer program is executed by the processor, the photo processing method according to any one of the above-mentioned first aspects is realized.

In a fifth aspect, the embodiments of the present application provide a computer program product, which when the computer program product runs on a terminal device, causes the terminal device to execute the photo processing method described in any one of the above-mentioned first aspects.

It is understandable that, for the beneficial effects of the second aspect to the fifth aspect described above, reference may be made to the relevant description in the first aspect described above, and details are not repeated here.

Beneficial effect

Compared with the prior art, the embodiments of this application have the following beneficial effects:

In the embodiment of the application, by acquiring the photos to be processed, multiple sub-images are intercepted from the photos to be processed. The sub-images contain at least one portrait, and these sub-images are used as the photos to be selected; and then the corresponding sub-images are calculated separately. Group cohesion, the group cohesion is used to characterize the degree of cohesion between the individual portraits in the sub-image, and this is used as an index to measure the quality of the sub-image; finally the sub-image corresponding to the highest group cohesion is determined, which is the post-processing Photo. Through the above method, from the existing photos to be processed, the group cohesion is used as an index to determine the processed photos, which can effectively adjust the composition of the group photo, thereby improving the integrity and coordination of the picture of the photo group.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only of the present application. For some embodiments, for those of ordinary skill in the art, other drawings may be obtained based on these drawings without creative labor.

FIG. 1 is a schematic diagram of a photo processing system provided by an embodiment of the present application;

FIG. 2 is a schematic flowchart of a photo processing method provided by an embodiment of the present application;

FIG. 3 is a schematic flowchart of a method for capturing sub-images according to an embodiment of the present application;

FIG. 4 is a schematic diagram of the smallest rectangle provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of the largest rectangle and the smallest rectangle provided by an embodiment of the present application;

Fig. 6 is a schematic diagram of a middle rectangle provided by an embodiment of the present application;

FIG. 7 is a structural block diagram of a photo processing device provided by an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a photo processing device provided by an embodiment of the present application.

Embodiments of the present invention

In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are proposed for a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from obstructing the description of this application.

It should be understood that when used in the specification and appended claims of this application, the term "comprising" indicates the existence of the described features, wholes, steps, operations, elements and/or components, but does not exclude one or more other The existence or addition of features, wholes, steps, operations, elements, components, and/or collections thereof.

In addition, in the description of the specification of this application and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

Reference to "one embodiment" or "some embodiments" described in the specification of this application means that one or more embodiments of this application include a specific feature, structure, or characteristic described in combination with the embodiment. Therefore, the sentences "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification are not necessarily All refer to the same embodiment, but mean "one or more but not all embodiments" unless it is specifically emphasized otherwise.

First, an application scenario of the photo processing method provided in the embodiment of the present application is introduced. Refer to Fig. 1, which is a schematic diagram of a photo processing system provided by an embodiment of this application. As shown in the figure, the photo processing system may include a photographing device 101 and a terminal device 102. Among them, the photographing device may be a camera, a video camera, a mobile phone with a photographing function, and so on. The terminal device can be a mobile phone, a computer, etc. The photographing device and the terminal device can be connected in a wired or wireless manner. The photographing device sends the photographed photos to the terminal device, and the terminal device processes the received photos using the photo processing method provided in this embodiment of the application, and displays the processed photos to the user, or returns the processed photos To the camera, the camera displays the processed photos to the user.

Of course, in practical applications, the terminal device 102 may be integrated with the photographing device 101. In this way, the terminal device not only has a photographing function, but also has a photo processing capability.

FIG. 2 shows a schematic flowchart of a photo processing method provided by an embodiment of the present application. As an example and not a limitation, the method may include the following steps:

S201: Obtain a photo to be processed, and intercept multiple sub-images from the photo to be processed, where the sub-images include at least one portrait.

In practical applications, the photos to be processed are usually photos of people, that is, the photos include multiple portraits. Take sub-images from the photos to be processed. The sub-images contain one or more portraits, preferably all portraits. Use multiple sub-images as the candidate images.

For the process of intercepting multiple sub-images from the photos to be processed, please refer to the description in the embodiment of FIG. 3, which will not be repeated here.

S202: Calculate the group cohesion degree corresponding to each sub-image, where the group cohesion degree is used to represent the degree of cohesion between the individual portraits in the sub-image.

Group cohesion refers to the degree to which group members are attracted to each other and are willing to stay in the group. It is a resultant force to maintain the effectiveness of group behavior. In the embodiments of the present application, the degree of group cohesion is used as a measurement index to characterize the consistency and cohesion degree of various characters in the photo. The higher the cohesion of the group, the higher the quality of the photos of the people.

In an embodiment, calculating the group cohesion corresponding to each sub-image in step S202 may include the following steps:

S21: For each sub-image, respectively calculate the overall scene cohesion, face cohesion, and body cohesion corresponding to the sub-image.

Among them, the overall scene cohesion is calculated based on the whole of the sub-image, and is used to characterize the cohesion between the characters and the background in the sub-image. Face cohesion is calculated based on the facial image in the sub-image, and is used to characterize the facial expressions of each person in the sub-image. The body cohesion is calculated based on the body image of the person in the sub-image, and is used to characterize the body posture of each person in the sub-image. Optionally, step S21 may include:

S211: Obtain a trained neural network, where the neural network includes three sub-networks, and the three sub-networks are respectively used to calculate the overall scene cohesion, the face cohesion, and the body cohesion.

S212: Input the sub-image into the neural network for processing, and obtain the overall scene cohesion, face cohesion, and body cohesion corresponding to the sub-image.

In practical applications, the neural network needs to be trained in advance.

Optionally, for the sub-network used to calculate the overall scene cohesion and the sub-network of the body cohesion, SE-NET (Squeeze-and-Excitation network) The network structure is pre-trained with the ImageNet data set, and trained and tested with the GAF-Cohesion database. In the training process, you can also add emoticons to assist in supervised learning. The loss function of this sub-network can adopt cross entropy, mean square error, emotional rank loss (Rank Loss) function and hourglass loss (Hourglass Loss) function.

Optionally, for the sub-network used to calculate the degree of face cohesion, the ResNet network structure can be used, the PERPlus data set is used for pre-training, and the GAF-Cohesion database is used for training and testing. The loss function of this sub-network can adopt cross entropy, mean square error, Rank Loss and Hourglass Loss functions.

Using the trained neural network to calculate the overall scene cohesion, face cohesion, and body cohesion can improve calculation efficiency while ensuring calculation accuracy.

S22: Calculate the group cohesion degree corresponding to the sub-image according to the overall scene cohesion degree, the face cohesion degree and the body cohesion degree.

Optionally, step S22 may include:

Exemplarily, the weight value of the overall scene cohesion h1 is set to w1, the weight value of h2 is set to the face cohesion number w2, and the weight value of the body cohesion h3 is set to w3. Then the group cohesion is H= h1×w1+ h2×w2+ h3×w3.

Combining the above three cohesion degrees to calculate the final group cohesion, not only the relationship between the sub-image characters and the background can be considered, but also the facial expressions and body postures of the characters in the sub-images. The obtained group cohesion can be More comprehensively characterize the coordination and completeness of sub-images.

S203: Determine the sub-image corresponding to the highest group cohesion as a processed photo.

From the sub-images to be selected in S201, the sub-image with the highest degree of group cohesion is selected as the processed photo.

The embodiment of this application obtains photos to be processed, intercepts multiple sub-images from the photos to be processed, and uses these sub-images as photos to be selected; then calculates the group cohesion degree corresponding to each sub-image, and uses the group cohesion degree to represent The consistency and cohesion between the various characters in the sub-images are used as an index to measure the quality of the sub-images; the sub-image corresponding to the highest group cohesion is finally determined as the processed photo. Through the above method, from the existing photos to be processed, the group cohesion is used as an index to determine the processed photos, which can effectively improve the processing effect of the group photos of people, and further improve the quality of the photos of the people group photos.

Refer to FIG. 3, which is a schematic flowchart of a method for capturing sub-images provided by an embodiment of this application. As shown in FIG. 3, in the above step S201, intercepting multiple sub-images from the to-be-processed photo may include the following steps:

S301: Determine the smallest rectangle containing all human figures in the photo to be processed, and intercept the image corresponding to the smallest rectangle to obtain a first sub-image.

In the embodiment of the present application, the portrait in the photo to be processed includes a facial image and a body image.

Optionally, in step S301, determining the smallest rectangle containing all human figures in the to-be-processed photo may include the following steps:

S3011: Perform portrait recognition on the photo to be processed, and obtain the coordinates of each pixel point corresponding to the recognized portrait.

Among them, face recognition includes two parts: face recognition and body detection.

Exemplarily, face recognition can use a multi-task convolutional neural network (Multi-task convolutional neural network). neural network, MTCNN) implementation. The neural network combines face region detection and face key point detection together, and the output result contains both the face detection result and the key points of the detected face.

Body detection can be implemented using the open source human gesture recognition project OpenPose. This project can realize the posture estimation of human body movement, joint movement, etc. The output result includes the positioning of each joint point of the human body.

In this embodiment, firstly, the image to be processed is recognized to obtain the recognition result, and the recognition result may include the contour of the recognized portrait. Then obtain the coordinates of the pixels covered by the contour of the portrait in the photo to be processed.

S3012: According to the coordinates, respectively determine the edge position point corresponding to each side of the photo to be processed, wherein the edge position point corresponding to one side is the shortest distance from the one side among all the pixel points corresponding to the portrait Of pixels.

In practical applications, the shape of the photo is generally rectangular, that is, it has 4 sides. For each side, you can calculate the distance from each pixel corresponding to the identified portrait to this side (that is, calculate the distance from the point to the line) according to the coordinates, and use the pixel corresponding to the minimum distance as the side corresponding Edge position point. Through the above method, 4 edges correspond to 4 edge position points.

S3013: Determine the minimum rectangle according to edge position points corresponding to each side of the photo to be processed.

For each edge location point, draw a straight line parallel to the edge corresponding to the edge location point through the edge location point. The figure enclosed by the straight line passing through each edge position point is recorded as the smallest rectangle.

Exemplarily, refer to FIG. 4, which is a schematic diagram of the smallest rectangle provided in an embodiment of this application. As shown in Figure 4, among all the pixels corresponding to the recognized portrait, pixel A is the pixel closest to the side a of the photo to be processed, and pixel B is the pixel closest to the side b of the photo to be processed Point, pixel point C is the pixel point closest to the side c of the photo to be processed, and pixel point D is the pixel point closest to the side d of the photo to be processed. Therefore, pixel points A, B, C, and D are sides respectively The edge position points corresponding to a, b, c, and d. Make a line parallel to side a through point A, make a line parallel to side b through point B, make a line parallel to side c through point C, and make a line parallel to side d through point D. The rectangle enclosed by 4 straight lines is the smallest rectangle.

S302: Determine the largest rectangle in the photo to be processed, and intercept the image corresponding to the largest rectangle to obtain a second sub-image.

Wherein, the largest rectangle and the smallest rectangle have the same center and aspect ratio.

In other words, the largest rectangle is the largest rectangle with the same center and the same aspect ratio as the smallest rectangle in the photo to be processed, and the largest rectangle is actually a rectangle obtained by scaling up the smallest rectangle. Moreover, the largest rectangle contains the smallest rectangle, and the sides of the largest rectangle do not intersect with the sides of the smallest rectangle. Refer to FIG. 5, which is a schematic diagram of the largest rectangle and the smallest rectangle provided in an embodiment of this application. As shown in Fig. 5, the smallest rectangle and the largest rectangle have the same center O, and the diagonal of the largest rectangle passes through the apex of the smallest rectangle.

S303: Capture N sub-images based on the maximum rectangle and the minimum rectangle.

Wherein, the N sub-images and the smallest rectangle have the same center and aspect ratio, and the area of any one of the N sub-images is larger than the area of the smallest rectangle and smaller than the area of the largest rectangle.

In the embodiment of the present application, the multiple sub-images include a first sub-image, a second sub-image, and N sub-images. Optionally, step S303, capturing N sub-images based on the maximum rectangle and the minimum rectangle, may include the following steps:

S3031. Obtain N preset ratios, and scale the smallest rectangles respectively according to each preset ratio to obtain N middle rectangles, where the middle rectangle and the smallest rectangle have the same center, and the middle rectangle has the same center as the smallest rectangle. The area of the rectangle is smaller than the area of the largest rectangle.

S3032: Separate the images corresponding to each middle rectangle to obtain N sub-images.

In practical applications, if the preset ratio is set, the corresponding N is determined. Exemplarily, suppose that the preset ratios are 1.1, 1.2, and 1.3 respectively, and the corresponding N=3. At this time, the smallest rectangle is enlarged to 1.1 times, 1.2 times, and 1.3 times, respectively, to obtain 3 middle rectangles, and 3 corresponding sub-images are captured.

Optionally, the value of N can also be set first, and then the smallest rectangle is scaled up according to the value of N and according to certain rules.

Exemplarily, refer to FIG. 6, which is a schematic diagram of the middle rectangle provided in this embodiment of the present application. As shown in Figure 6, first set N=2. Then a vertex P of the smallest rectangle and a vertex Q corresponding to the vertex of the largest rectangle are connected to form a line segment PQ, and the line segment PQ is divided into 3 (N+1) equal parts to obtain two equal points M1 and M2. Take M1 and M2 as the vertices, and scale up the smallest rectangle to obtain the middle rectangle 1 and the middle rectangle 2.

In the above example, the smallest rectangle is scaled up according to the rule of taking the equal division points of the line segment PQ. In practical applications, the non-equal points of the line segment PQ can also be taken, which is not limited here.

In the embodiment of this application, the first sub-image is obtained by determining the smallest rectangle containing all human figures in the photo to be processed, and intercepting the image corresponding to the smallest rectangle; then determining the largest rectangle in the photo to be processed, and intercepting the image corresponding to the largest rectangle to obtain the second Zhang sub-images; finally, N sub-images are captured based on the maximum rectangle and the minimum rectangle. Through the above method, multiple sub-images to be selected can be obtained, and it can be ensured that each sub-image contains all human portraits. Selecting and processing photos from these sub-images effectively guarantees the quality of the processed photos.

It should be understood that the size of the sequence number of each step in the foregoing embodiment does not mean the order of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

Corresponding to the photo processing method described in the above embodiment, FIG. 7 shows a structural block diagram of a photo processing device provided in an embodiment of the present application. For ease of description, only parts related to the embodiment of the present application are shown.

Referring to Figure 7, the device includes:

The acquiring unit 71 is configured to acquire a photo to be processed, and to intercept multiple sub-images from the photo to be processed, the sub-images containing at least one portrait.

The calculation unit 72 is configured to calculate the group cohesion degree corresponding to each sub-image, and the group cohesion degree is used to characterize the degree of cohesion between the individual portraits in the sub-image.

The processing unit 73 is used to determine the sub-image corresponding to the highest group cohesion, which is the processed photo.

Optionally, the obtaining unit 71 includes:

The first determining module is configured to determine the smallest rectangle containing all human figures in the photo to be processed, and intercept the image corresponding to the smallest rectangle to obtain the first sub-image.

The second determining module is used to determine the largest rectangle in the photo to be processed, and to intercept the image corresponding to the largest rectangle to obtain a second sub-image, wherein the largest rectangle and the smallest rectangle have the same center and Aspect ratio.

The interception module is configured to intercept N sub-images based on the largest rectangle and the smallest rectangle, where the N sub-images and the smallest rectangle have the same center and aspect ratio, and any of the N sub-images The area of a sub-image is larger than the area of the smallest rectangle and smaller than the area of the largest rectangle;

Optionally, the first determining module includes:

The recognition sub-module is used to recognize the portrait of the photo to be processed and obtain the coordinates of each pixel corresponding to the recognized portrait.

The point determination sub-module is used to determine the edge position points corresponding to each side of the photo to be processed according to the coordinates, wherein the edge position points corresponding to one side are all pixels corresponding to the portrait. Describe the pixel with the shortest side distance.

The rectangle determining sub-module is configured to determine the minimum rectangle according to the edge position points corresponding to each side of the photo to be processed.

Optionally, the interception module includes:

The scaling sub-module is used to obtain N preset ratios, and scale the smallest rectangles according to each preset ratio to obtain N middle rectangles, the middle rectangles and the smallest rectangles have the same center And the area of the middle rectangle is smaller than the area of the largest rectangle.

The interception sub-module is used to separately intercept the image corresponding to each middle rectangle to obtain N sub-images.

Optionally, the calculation unit 72 includes:

The first calculation sub-module is used to calculate the overall scene cohesion, face cohesion, and body cohesion corresponding to each sub-image, wherein the overall scene cohesion is used to characterize the sub-image. The degree of cohesion between the portrait and the background in the image, the degree of facial cohesion is used to characterize the facial expressions of each portrait in the sub-image; the degree of body cohesion is used to characterize the body posture of each portrait in the sub-image.

The second calculation sub-module is used to calculate the group cohesion degree corresponding to the sub-image according to the overall scene cohesion degree, face cohesion degree and body cohesion degree.

Optionally, the first calculation sub-module is also used to obtain a trained neural network, the neural network includes three sub-networks, and the three sub-networks are respectively used to calculate the overall scene cohesion, the face cohesion, and The body cohesion; the sub-image is input into the neural network for processing, and the overall scene cohesion, face cohesion, and body cohesion corresponding to the sub-image are obtained.

Optionally, the second calculation submodule is further configured to perform a weighted summation of the overall scene cohesion, the face cohesion, and the body cohesion according to preset weights to obtain the group cohesion.

It should be noted that the information interaction and execution process between the above-mentioned devices/units are based on the same concept as the method embodiment of this application, and its specific functions and technical effects can be found in the method embodiment section. I won't repeat it here.

In addition, the photo processing device shown in FIG. 7 can be a software unit, a hardware unit, or a combination of software and hardware built into an existing terminal device, or it can be integrated into the terminal device as an independent pendant, or Exist as an independent terminal device.

Those skilled in the art can clearly understand that for the convenience and conciseness of description, only the division of the above functional units and modules is used as an example. In practical applications, the above functions can be allocated to different functional units and modules as required. Module completion, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist alone physically, or two or more units can be integrated into one unit. The above-mentioned integrated units can be hardware-based Formal realization can also be realized in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of this application. For the specific working process of the units and modules in the foregoing system, reference may be made to the corresponding process in the foregoing method embodiment, which will not be repeated here.

FIG. 8 is a schematic structural diagram of a photo processing device provided by an embodiment of the application. As shown in FIG. 8, the photo processing device 8 of this embodiment includes: at least one processor 80 (only one is shown in FIG. 8), a processor, a memory 81, and a memory 81 that is stored in the memory 81 and can be stored in the at least one processor. The computer program 82 running on the processor 80 implements the steps in any of the foregoing photo processing method embodiments when the processor 80 executes the computer program 82.

The photo processing device may be a mobile phone, a desktop computer, a notebook, a palmtop computer and other equipment with a shooting function. The photo processing device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that FIG. 8 is only an example of the photo processing device 8 and does not constitute a limitation on the photo processing device 8. It may include more or less components than those shown in the figure, or combine certain components, or different components. The components of, for example, can also include input and output devices, network access devices, and so on.

The so-called processor 80 may be a central processing unit (Central Processing Unit, CPU), and the processor 80 may also be other general-purpose processors or digital signal processors (Digital Signal Processors). Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

The memory 81 may be an internal storage unit of the photo processing device 8 in some embodiments, such as a hard disk or a memory of the photo processing device 8. In other embodiments, the memory 81 may also be an external storage device of the photo processing device 8, such as a plug-in hard disk or a smart memory card (Smart Media Card, SMC) equipped on the photo processing device 8. Secure Digital (SD) card, flash card (Flash Card), etc. Further, the memory 81 may also include both an internal storage unit of the photo processing apparatus 8 and an external storage device. The memory 81 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 81 can also be used to temporarily store data that has been output or will be output.

The embodiments of the present application also provide a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in each of the foregoing method embodiments can be realized.

The embodiments of the present application provide a computer program product. When the computer program product runs on a mobile terminal, the steps in the foregoing method embodiments can be realized when the mobile terminal is executed.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the implementation of all or part of the processes in the above-mentioned embodiment methods in this application can be accomplished by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by the processor, the steps of the foregoing method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may include at least: any entity or device capable of carrying computer program code to the photo processing device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM) , Random Access Memory), electrical carrier signals, telecommunications signals, and software distribution media. For example, U disk, mobile hard disk, floppy disk or CD-ROM, etc. In some jurisdictions, in accordance with legislation and patent practices, computer-readable media cannot be electrical carrier signals and telecommunication signals.

In the above-mentioned embodiments, the description of each embodiment has its own focus. For parts that are not described in detail or recorded in an embodiment, reference may be made to related descriptions of other embodiments.

A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

In the embodiments provided in this application, it should be understood that the disclosed apparatus/network equipment and method may be implemented in other ways. For example, the device/network device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units. Or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, a person of ordinary skill in the art should understand that it can still implement the foregoing The technical solutions recorded in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in Within the scope of protection of this application.

Claims

A photo processing method, characterized in that it comprises:

Acquiring a photo to be processed, and intercepting multiple sub-images from the photo to be processed, the sub-images containing at least one portrait;

Separately calculating the group cohesion degree corresponding to each sub-image, where the group cohesion degree is used to characterize the degree of cohesion between the individual portraits in the sub-image;

Determine the sub-image corresponding to the highest group cohesion as the processed photo.
8. The photo processing method of claim 1, wherein the intercepting multiple sub-images from the photo to be processed comprises:

Determining the smallest rectangle containing all human figures in the photo to be processed, and intercepting the image corresponding to the smallest rectangle to obtain the first sub-image;

Determining the largest rectangle in the photo to be processed, and intercepting an image corresponding to the largest rectangle to obtain a second sub-image, wherein the largest rectangle and the smallest rectangle have the same center and aspect ratio;

N sub-images are captured based on the maximum rectangle and the minimum rectangle, the N sub-images and the minimum rectangle have the same center and aspect ratio, and any sub-image in the N sub-images The area is larger than the area of the smallest rectangle and smaller than the area of the largest rectangle;

Wherein, the multiple sub-images include the first sub-image, the second sub-image, and the N sub-images.
3. The photo processing method according to claim 2, wherein the determining the smallest rectangle containing all human portraits in the to-be-processed photo comprises:

Perform portrait recognition on the photo to be processed, and obtain the coordinates of each pixel corresponding to the recognized portrait;

According to the coordinates, respectively determine the edge position point corresponding to each side of the photo to be processed, wherein the edge position point corresponding to one side is the pixel with the shortest distance from the one side among all the pixels corresponding to the portrait point;

The minimum rectangle is determined according to the edge position points corresponding to each side of the photo to be processed.
3. The photo processing method according to claim 2, wherein the capturing N sub-images based on the largest rectangle and the smallest rectangle comprises:

Obtain N preset ratios, and scale the smallest rectangles according to each preset ratio to obtain N middle rectangles. The middle rectangle and the smallest rectangle have the same center, and the middle rectangle has the same center. The area is smaller than the area of the largest rectangle;

The images corresponding to each middle rectangle are respectively intercepted to obtain N sub-images.
The picture processing method as claimed in claim 1, wherein the sub-images are calculated for each group corresponding to the degree of aggregation, comprising:

For each sub-image, calculate the overall scene cohesion, face cohesion, and body cohesion corresponding to the sub-image, where the overall scene cohesion is used to characterize the relationship between the portrait and the background in the sub-image. Cohesion, where the face cohesion is used to characterize the facial expression of each person in the sub-image; the body cohesion is used to characterize the body posture of each person in the sub-image;

The group cohesion degree corresponding to the sub-image is calculated according to the overall scene cohesion degree, the face cohesion degree and the body cohesion degree.
5. The photo processing method of claim 5, wherein the separately calculating the overall scene cohesion, face cohesion, and body cohesion corresponding to the sub-images comprises:

Acquiring a trained neural network, the neural network including three sub-networks, and the three sub-networks are respectively used to calculate the overall scene cohesion, the face cohesion, and the body cohesion;

The sub-image is input into the neural network for processing, and the overall scene cohesion, face cohesion, and body cohesion corresponding to the sub-image are obtained.
5. The photo processing method of claim 5, wherein the calculating the group cohesion corresponding to the sub-image according to the overall scene cohesion, face cohesion, and body cohesion includes:

The overall scene cohesion, the face cohesion, and the body cohesion are weighted and summed according to preset weights to obtain the group cohesion.
A photo processing device, characterized in that it comprises:

An acquiring unit, configured to acquire a photo to be processed, and to intercept multiple sub-images from the photo to be processed, the sub-images containing at least one portrait;

A calculation unit, configured to separately calculate a group cohesion degree corresponding to each sub-image, where the group cohesion degree is used to characterize the degree of cohesion between the individual portraits in the sub-image;

The processing unit is used to determine the sub-image corresponding to the highest group cohesion, which is the processed photo.
A photo processing device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program as claimed in claim 1. To the method of any one of 7.
A computer-readable storage medium storing a computer program, wherein the computer program implements the method according to any one of claims 1 to 7 when the computer program is executed by a processor.