WO2023010661A1

WO2023010661A1 - Image cropping method and related product

Info

Publication number: WO2023010661A1
Application number: PCT/CN2021/119420
Authority: WO
Inventors: 蒋彬; 宋利伟; 殷晨晖; 周奥; 杨天明
Original assignee: 展讯通信（上海）有限公司
Priority date: 2021-08-04
Filing date: 2021-09-18
Publication date: 2023-02-09
Also published as: CN113628229B; CN113628229A

Abstract

An image cropping method and a related product. The image cropping method comprises: acquiring first face area information, the first face area information representing a first face area in an original image; acquiring second face area information on the basis of a preset proportion coefficient and the first face area information, the second face area information representing a second face area obtained after the first face area is adjusted according to the preset proportion coefficient; and cropping the original image on the basis of the boundaries of the second face area and the original image to obtain a first image. By means of the method, an area having serious distortion in an original image can be cropped accurately, thereby avoiding excessive cropping or insufficient cropping.

Description

Image cropping method and related products

[Corrected 15.10.2021 under Rule 91]
This application claims the priority of the Chinese patent application with the application number 202110890240.3 and the application title "Image Cropping Method and Related Products" filed with the China Patent Office on August 04, 2021, the entire contents of which are incorporated by reference in this application.

technical field

The present application relates to the technical field of image processing, in particular to an image cropping method and related products.

Background technique

Lenses currently on the market can be roughly divided into two types: wide-angle lenses and non-wide-angle lenses. Images captured by products with non-wide-angle lenses, such as mobile phone lenses, have less distortion. The distortion of images taken by products with wide-angle lenses or ultra-wide-angle lenses is usually very large, and it is almost impossible to see without distortion correction. There are two main types of image distortion: radial distortion and tangential distortion. Radial distortion: The distortion at the center is the smallest, and the distortion increases with the increase of the radius. Radial distortion can be divided into pincushion distortion and barrel distortion. Tangential distortion is caused by the fact that the lens itself is not parallel to the camera sensor plane (imaging plane) or the image plane. This situation is mostly caused by the installation deviation of the lens pasted on the lens module.

In practical applications, it is usually necessary to correct the phenomenon of portrait distortion in the image (which may be referred to as distortion correction processing). At present, the phenomenon of portrait distortion in the image can be corrected through different image processing techniques such as projection transformation. While correcting the phenomenon of portrait distortion in the image, the image content around the portrait will also be affected to a certain extent, such as the bending of straight lines. After correcting the phenomenon of portrait distortion in the image, it is generally necessary to further crop the corrected image to remove part of the image content in the corrected image that is affected by the correction process. The distortion-corrected image is more natural after being cropped. At present, there is a problem of cropping too much or not cropping enough in the way of cropping and distortion correcting the processed image. How to avoid excessive cropping or insufficient cropping is a problem that needs to be studied by the target.

Contents of the invention

The embodiment of the present application discloses an image cropping method and related products.

In the first aspect, an embodiment of the present application provides an image cropping method, the method comprising: acquiring first face area information; the first face area information characterizes the first face area in the original image; based on preset Scale factor and the first face area information to obtain second face area information; the second face area information represents the second face adjusted by the first face area according to the preset scale coefficient area: based on the boundary between the second human face area and the original image, crop the original image to obtain a first image.

In the embodiment of the present application, based on the boundary between the second face area and the original image, the original image is cropped to obtain the first image; the area with severe distortion in the original image can be accurately cropped to avoid excessive cropping or insufficient cropping .

In a possible implementation manner, the cropping the original image based on the boundary between the second face area and the original image to obtain the first image includes: When the number of pixels whose boundary exceeds the first boundary of the original image is greater than a first threshold, use the first cropping boundary as a new first boundary of the original image to crop the original image to obtain the first image; The first cropping boundary is parallel to the first boundary of the original image, and the number of pixels between the first cropping boundary and the first boundary of the original image is equal to the first boundary of the second human face area The difference between the number of pixels beyond the first boundary of the original image and the first threshold. The number of pixels between the first cropping boundary and the first boundary of the original image refers to the number of pixels corresponding to the distance between the first cropping boundary and the first boundary of the original image. The first boundary is any one of left boundary, right boundary, upper boundary and lower boundary.

In this implementation manner, the original image is cropped using the first cropping boundary as a new first boundary of the original image to obtain the first image. The first clipping boundary is determined by the difference between the number of pixels that the first boundary of the second human face region exceeds the first boundary of the original image and the first threshold, and the first cropping boundary takes into account the size of the first human face region and The distance between the first boundary of the first human face area and the first boundary of the original image can be used to crop the edge of the image in a reasonable proportion.

In a possible implementation manner, the cropping the original image based on the boundary between the second face area and the original image to obtain the first image includes: When the distance between the boundary and the second boundary of the original image is less than a second threshold and the second face area is located in the original image, the second cropping boundary is used as the new second boundary of the original image cropping the original image to obtain the first image; the second cropping boundary is a second boundary of the second face area. The fact that the second face area is located in the original image means that the second face area is included in the original image. The second boundary is any one of left boundary, right boundary, upper boundary and lower boundary.

The closer the image is to the edge, the more serious the distortion will be. In the original image, the area between the second boundary of the second human face area and the second boundary of the original image can be understood as an area with severe distortion, and other areas are areas with light distortion.

In this implementation manner, the original image is cropped using the second cropping boundary as a new second boundary of the original image to obtain the first image. Taking the second cropping boundary as the original image, the new second boundary cropping the original image takes into account the size of the first human face area and the distance between the second boundary of the first human face area and the second boundary of the original image, which can be The edges of the image are cropped with reasonable proportions.

In a possible implementation manner, the first face area information includes: the vertex coordinates of the first face area and the length and width of the first face area; For the first face area information, obtaining the second face area information includes: taking the product of the length and width of the first face area and the preset proportional coefficient as the length of the second face area and width, and determine the vertex coordinates of the second face area according to the vertex coordinates, length and width of the first face area.

In this implementation manner, the second face region information can be acquired quickly and accurately.

In a possible implementation manner, the coordinates of the first vertex of the second face area are the same as the coordinates of the first vertex of the first face area. The first vertex is any one of an upper left vertex, a lower left vertex, an upper right vertex, and a lower right vertex. The first vertex is any vertex on the third boundary of the second human face area, and the distance between the third boundary of the first human face area and the third boundary of the original image is greater than other boundaries of the first human face area The distance from the corresponding border of the original image.

In a possible implementation manner, the coordinates of the center point of the second face area are the same as the coordinates of the center point of the first face area.

In a possible implementation manner, after the original image is cropped based on the boundary of the second human face area and the original image to obtain the first image, the method further includes: Width, height and the ratio of width to height are scaled and interpolated to the first image to obtain a second image; the width, height and ratio of width to height of the second image are respectively the same as the width, height and ratio of the original image The ratio of width to height is the same.

In this implementation, the first image is scaled and interpolated according to the width, height and ratio of width and height of the original image to obtain the second image; the width, height and width of the original image can be obtained from the cropped first image The second image is the same with high proportions.

In a possible implementation manner, the method further includes: performing distortion correction processing on the input image to obtain the original image.

In a second aspect, an embodiment of the present application provides an image processing device, including: an acquisition unit, configured to first face area information; the first face area information represents a first face area in an original image; the The acquisition unit is further configured to acquire second face area information based on a preset scale factor and the first face area information; the second face area information indicates that the first face area is in accordance with the preset The second human face area after the scale factor has been adjusted; an image cropping unit, configured to crop the original image based on the boundary between the second human face area and the original image to obtain the first image.

In a possible implementation manner, the image cropping unit is specifically configured to: when the number of pixels in which the first boundary of the second face area exceeds the first boundary of the original image is greater than a first threshold , using the first cropping boundary as the new first boundary of the original image to crop the original image to obtain the first image; the first cropping boundary is parallel to the first boundary of the original image, and the first The number of pixels between the cropping boundary and the first boundary of the original image is equal to the difference between the number of pixels where the first boundary of the second face area exceeds the first boundary of the original image and the first threshold .

In a possible implementation manner, the image cropping unit is specifically configured to make the distance between the second boundary of the first face area and the second boundary of the original image smaller than a second threshold and the In the case where the second human face area is located in the original image, the original image is cropped with the second crop boundary as the new second boundary of the original image to obtain the first image; the second crop boundary is the Describe the second boundary of the second face area.

In a possible implementation manner, the first face area information includes: the vertex coordinates of the first face area and the length and width of the first face area; Taking the product of the length and width of the first human face area and the preset proportional coefficient as the length and width of the second human face area, and according to the vertex coordinates, length and Width determines the vertex coordinates of the second face area.

In a possible implementation manner, the coordinates of the first vertex of the second face area are the same as the coordinates of the first vertex of the first face area.

In a possible implementation manner, the device further includes: a processing unit configured to perform scaling and interpolation on the first image according to the width, height, and ratio of width to height of the original image to obtain a second image; The width, height and ratio of width to height of the second image are respectively the same as the width, height and ratio of width to height of the original image.

In a possible implementation manner, the processing unit is further configured to perform distortion correction processing on the input image to obtain the original image.

Regarding the technical effects brought about by the second aspect or various optional implementation manners, reference may be made to the introduction to the technical effects of the first aspect or corresponding implementation manners.

In a third aspect, an embodiment of the present application provides an image processing device, the image processing device includes: a processor and a memory, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, The processor is made to execute the method according to the above first aspect and any possible implementation manner.

In a fourth aspect, an embodiment of the present application provides a chip, where the chip includes a data interface and a processor, where the processor is configured to execute the method in the first aspect or any possible implementation manner of the first aspect.

In the fifth aspect, the embodiment of the present application provides a computer-readable storage medium, the computer storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the above-mentioned first step. One aspect and any optional implementation method of the first aspect above.

In a sixth aspect, an embodiment of the present application provides a computer program product, the computer program product includes program instructions, and when the program instructions are executed by a processor, the processor executes the above-mentioned first aspect and any optional method of implementation.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiment of the present application or the background art, the following will describe the drawings that need to be used in the embodiment of the present application or the background art.

FIG. 1 is a flow chart of an image cropping method provided in an embodiment of the present application;

FIG. 2 is a schematic diagram of an example of a first face area and a second face area provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of another example of a first human face area and a second human face area provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of an example of cropping an original image provided in an embodiment of the present application;

FIG. 5 is a schematic diagram of another example of cropping an original image provided by an embodiment of the present application;

FIG. 6 is a flow chart of another image cropping method provided in the embodiment of the present application;

FIG. 7 is a flowchart of an image distortion correction processing method provided by an embodiment of the present application;

FIG. 8 is a schematic structural diagram of an image processing device provided by an embodiment of the present application;

FIG. 9 is a schematic structural diagram of a server provided by an embodiment of the present application;

FIG. 10 is a schematic structural diagram of a terminal device provided in an embodiment of the present application.

Detailed ways

The terms "first" and "second" in the specification, claims and drawings of the present application are only used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally It also includes other steps or units inherent to these processes, methods, products, or devices.

The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. As used in the specification and appended claims of this application, the singular expressions "a", "an", "above", "aforementioned", "the" and "this" are intended to include Plural expressions, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in this application refers to and includes any and all possible combinations of one or more of the listed items. For example, "A and/or B" can mean: there are only A, only B, and both A and B, where A and B can be singular or plural. The term "plurality" as used in this application means two or more.

In order to solve the problem of excessive cropping or insufficient cropping of the image processed by distortion correction. The present application provides an image cropping method that can accurately crop the edge region with severe distortion in the image, and can better solve the problem of excessive cropping or insufficient cropping of the distortion-corrected image. The image cropping method provided in the embodiment of the present application can be applied to image distortion correction processing scenarios. The following briefly introduces the applicable scenarios of the image cropping method provided by the embodiment of the application.

Scenario 1: The user inputs the image to be processed (that is, the image to be processed by distortion correction) to an image processing device (for example, a desktop computer, a notebook computer, a personal computer, etc.); the image processing device performs distortion correction processing on the image to be processed, The image after distortion correction is obtained. In scenario 1, after the image processing device performs distortion correction processing on the image to be processed, the image cropping method provided in this application may be used to crop the image after distortion correction processing.

Scenario 2: The user sends the image to be processed (that is, the image to be processed for distortion correction) to the image processing device through a terminal device (such as a desktop computer, a notebook computer, a personal computer, a mobile phone, etc.); the image processing device performs processing on the image to be processed Distortion correction processing to obtain a distortion-corrected image; the image processing device sends the distortion-corrected image to the terminal device. In scenario 2, after the image processing device performs distortion correction processing on the image to be processed, the image cropping method provided in the present application may be used to crop the distortion-corrected image.

In the above scenario, by implementing the image cropping method provided in the embodiment of the present application, the edge region with severe distortion in the image can be cropped out more accurately.

The image cropping method provided by the embodiment of the present application will be described below with reference to the accompanying drawings.

FIG. 1 is a flow chart of an image cropping method provided by an embodiment of the present application. As shown in Figure 1, the method includes:

101. An image processing apparatus acquires first face region information.

The image processing device may be a terminal device capable of image processing such as a desktop computer, a notebook computer, or a personal computer, or may be a server, such as a cloud server.

The first face area information represents the first face area (a rectangular area) in the original image. In some embodiments, the original image is a distortion-corrected image. In a possible implementation manner, the first face region information includes: vertex coordinates of the first face region and length and width of the first face region. For example, the information of the first human face area includes: the initial coordinates (x, y) of the first human face area, the width and height (w, h) of the first human face area, and the initial coordinates are the The coordinates of the lower left vertex of the face region. For another example, for example, the first human face area information includes: the initial coordinates (x, y) of the first human face area, the width and height (w, h) of the first human face area, and the initial coordinates are the The coordinates of the upper right vertex of the first face area.

In some embodiments, before performing step 101, the image processing device may perform face detection on the original image to obtain the first face area. In practical applications, the image processing device may perform face detection on the original image to obtain one (ie, the first face area) or more than one face area. In some embodiments, before performing step 101, the image processing device may perform distortion correction processing on the input image to obtain the original image.

102. Acquire second face area information based on a preset scale factor and first face area information.

The second face area information represents the second face area adjusted from the first face area according to a preset scale factor.

In a possible implementation, the above-mentioned first human face area information includes: the vertex coordinates of the above-mentioned first human face area and the length and width of the above-mentioned first human face area; the implementation of step 102 is as follows: The product of the length and width of the face area and the above-mentioned preset proportional coefficient is used as the length and width of the second face area, and the vertex coordinates, length and width of the first face area are used to determine the size of the second face area. Vertex coordinates. In some embodiments, the coordinates of the first vertex of the second face area are the same as the coordinates of the first vertex of the first face area. The first vertex is any one of an upper left vertex, a lower left vertex, an upper right vertex, and a lower right vertex. In some embodiments, the first vertex is any vertex on the third boundary of the second human face area, and the distance between the third boundary of the first human face area and the third boundary of the original image is greater than that of the first human face The distance between the other boundaries of the region and the corresponding boundaries of the original image. For example, the first vertex is the lower right vertex or lower left vertex of the first human face area, and the distance between the upper boundary of the first human face area and the upper boundary of the original image is the distance between the upper boundary of the first human face area and the original image. The distance between the upper border, the distance between the lower border of the first face area and the lower border of the original image, the distance between the left border of the first face area and the left border of the original image, and the distance between the first face area The largest of the distances between the right border of and the right border of the original image. Fig. 2 is a schematic diagram of an example of a first human face area and a second human face area provided by an embodiment of the present application. In FIG. 2 , the solid-line rectangular frame 201 represents the first human face area, and the dotted-line rectangular frame 202 represents the second human face area. The lower left vertex of the first human face area is the same as the lower left vertex of the second human face area. In some embodiments, the coordinates of the center point of the second human face area are the same as the coordinates of the center point of the first human face area. FIG. 3 is a schematic diagram of another example of a first human face area and a second human face area provided by an embodiment of the present application. In Fig. 3, the solid line rectangular frame 301 represents the first human face region, the dotted line rectangular frame 302 represents the second human face region, the coordinates of the center point of the first human face region and the coordinates of the center point of the second human face region same.

In this implementation manner, the range of the first human face area is adjusted through the preset proportional coefficient k, so as to achieve a reasonable range of the human face area (that is, the range of the second human face area). In view of the scale transformation of the distance between the face and the camera (the face area closer to the camera occupies more pixels, and the face farther away from the camera occupies less pixels), the adjusted face area (corresponding to the second The size of the face area) should be a function of the combination of the size (width or height) of the current face area (corresponding to the first face area) and the preset proportional coefficient k.

103. Based on the boundary between the second face area and the original image, crop the original image to obtain the first image.

In some embodiments, the image processing device can determine whether the first human face region triggers the Crop the original image.

A possible implementation of step 103 is as follows: when the number of pixels of the first border of the second human face area beyond the first border of the original image is greater than the first threshold, the first clipping border is used as the new border of the original image. The above original image is cropped by the first boundary to obtain the above first image. The first boundary is any one of left boundary, right boundary, upper boundary and lower boundary. The above-mentioned first clipping boundary is parallel to the above-mentioned first boundary of the original image. The number of pixels between the first cropping boundary and the first boundary of the original image is equal to the difference between the number of pixels where the first boundary of the second human face region exceeds the first boundary of the original image and the first threshold. The number of pixels of the first boundary of the second human face area beyond the first boundary of the original image is greater than the first threshold is the condition for triggering the first boundary of the original image, that is, determining the new first boundary of the original image and following the new The first boundary condition for clipping the original image. That is to say, if the number of pixels of the first boundary of the second face area beyond the first boundary of the original image is less than or equal to the first threshold, it is not necessary to determine a new first boundary of the original image.

FIG. 4 is a schematic diagram of an example of cropping an original image provided by an embodiment of the present application. As shown in Figure 4, delta_u represents the difference between the upper boundary of the face area 1 (corresponding to the second face area) beyond the upper boundary of the original image (corresponding to the number of pixels), crop_u_thr represents the upper boundary of the face area 1 How many pixels exceed the upper boundary of the original image is the threshold for cropping, and c_u represents the number of pixels whose upper boundary of the face area 1 exceeds the upper boundary of the original image. c_u=delta_u-crop_u_thr, c_u represents the number of pixels to be cropped on the upper boundary. Similarly, delta_d represents the difference (corresponding to the number of pixels) that the lower boundary of the face area 2 (corresponding to the second face area) exceeds the lower boundary of the original image, and crop_d_thr indicates that the lower boundary of the face area 2 exceeds the original image. The number of pixels in the lower boundary of the image is the threshold for cropping, and c_d indicates the number of pixels in which the lower boundary of the face area 2 exceeds the lower boundary of the original image. Similarly, delta_l represents the difference (corresponding to the number of pixels) that the left boundary of the face area 3 (corresponding to the second face area) exceeds the left boundary of the original image, and crop_l_thr indicates that the left boundary of the face area 3 exceeds the original image. The number of pixels on the left border of the image is the threshold for cropping, and c_l represents the number of pixels that the left border of the face area 3 exceeds the left border of the original image. Similarly, delta_r indicates the difference (corresponding to the number of pixels) that the right boundary of the face area 4 (corresponding to the second face area) exceeds the right boundary of the original image, and crop_r_thr indicates that the right boundary of the face area 4 exceeds the original image The number of pixels on the right border of the image is the threshold for cropping, and c_r represents the number of pixels that the right border of the face area 4 exceeds the right border of the original image.

A possible implementation of step 103 is as follows: when the distance between the second boundary of the first human face area and the second boundary of the original image is less than a second threshold and the second human face area is located in the original image Next, use the second cropping boundary as a new second boundary of the original image to crop the above-mentioned original image to obtain the above-mentioned first image. The above-mentioned second clipping boundary is the second boundary of the above-mentioned second human face area. The aforementioned second human face area located in the aforementioned original image means that the aforementioned second human face area is included in the aforementioned original image. The second boundary is any one of left boundary, right boundary, upper boundary and lower boundary. The distance between the second boundary of the first human face area and the second boundary of the above-mentioned original image is less than the second threshold and the above-mentioned second human face area is located in the above-mentioned original image is the condition for triggering the cropping of the second boundary of the original image, that is, determining The new second boundary of the original image and the condition of cropping the original image according to the new second boundary. It should be understood that if the distance between the second boundary of the first human face area and the second boundary of the above-mentioned original image is not less than the second threshold or part of the above-mentioned second human face area is not included in the above-mentioned original image, then it is not necessary to Determine the new second boundary of the original image.

FIG. 5 is a schematic diagram of another example of cropping an original image provided by an embodiment of the present application. As shown in Figure 5, delta_2 represents the distance between the second boundary of the first face region and the second boundary of the original image, crop_2_thr represents the distance between the second boundary of the first face region and the second boundary of the original image The number of pixels of is less than the threshold for clipping, and 501 represents the second clipping boundary. The closer the image is to the edge, the more serious the distortion will be. In the original image, the area between the second boundary of the second human face area and the second boundary of the original image can be understood as an area with severe distortion, and other areas are areas with light distortion. In this implementation manner, the original image is cropped using the second cropping boundary as a new second boundary of the original image to obtain the first image. Taking the second cropping boundary as the original image, the new second boundary cropping the original image takes into account the size of the first human face area and the distance between the second boundary of the first human face area and the second boundary of the original image, which can be The edges of the image are cropped with reasonable proportions.

The method flow in FIG. 1 only shows how to crop the original image according to the first face area. It should be understood that the image processing device may crop the original image according to other face regions in a similar manner. In practical applications, the image processing device may crop the original image one or more times, for example, once for two or more boundaries.

FIG. 6 is a flow chart of another image cropping method provided by the embodiment of the present application. The method flow in FIG. 6 is a possible implementation of the method flow in FIG. 1 . As shown in Figure 6, the method includes:

601. The image processing apparatus acquires information about one or more face regions.

The one or more pieces of face area information acquired by the image processing device may include the above-mentioned first face area information. In some embodiments, the image processing device may perform face detection on the original image to obtain one or more face area information. Each face area information may include the starting coordinates of a face area (a rectangular area) and the length and width of the face area.

602. The image processing device sets a preset proportional coefficient.

Step 602 is optional, but not necessary. In some embodiments, the preset proportional coefficient may be preset, and the user does not need to set the preset proportional coefficient, that is, the preset preset proportional coefficient is adopted. In some embodiments, the user can set the preset proportional coefficient according to actual needs. The preset scale factor may be referred to as a face frame expansion factor.

603. The image processing device acquires one or more pieces of extended face area information according to a preset scale factor and one or more pieces of face area information.

The second face area information is an extended face area information, and the first face area information is a face area information. For example, each extended face area information may include the starting coordinates of an extended face area (a rectangular area) and the length and width of the extended face area. Step 102 in FIG. 1 describes a manner of acquiring the above-mentioned second face area information according to the preset scale factor and the above-mentioned first face area information. For the implementation manner of step 603, reference may be made to the implementation manner of step 102.

604. The image processing apparatus determines whether a condition for triggering cropping of an original image is met according to one or more pieces of extended face area information.

A possible implementation manner of step 604 may be to respectively judge whether the conditions for triggering the cropping of the boundary (edge) of the original image are met. Referring to Fig. 4, judging whether the condition for triggering the upper boundary of cropping the original image can be: determining whether the upper boundary of at least one extended human face area is determined in one or more extended human face areas indicated by one or more extended human face area information. The number of pixels whose boundary exceeds the upper boundary of the original image is greater than crop_u_thr; if so, the condition for triggering the cropping of the upper boundary of the original image is met. In the same way, judging whether the condition for triggering the lower boundary of cropping the original image can be: determining whether the lower boundary of at least one extended human face area in one or more extended human face areas indicated by one or more extended human face area information The number of pixels beyond the lower boundary of the original image is greater than crop_d_thr; if so, the condition for triggering the cropping of the lower boundary of the original image is met. Similarly, judging whether the condition for triggering the left boundary of the cropped original image can be: determining whether the left boundary of at least one extended human face region is indicated by one or more extended human face regions indicated by one or more extended human face region information The number of pixels beyond the left border of the original image is greater than crop_l_thr; if so, the condition for triggering the cropping of the left border of the original image is met. Similarly, judging whether the condition for triggering the cropping of the right boundary of the original image can be: determining whether the right boundary of at least one extended human face region is indicated by one or more extended human face regions indicated by one or more extended human face region information The number of pixels beyond the right boundary of the original image is greater than crop_r_thr; if so, the condition for triggering the cropping of the right boundary of the original image is met.

605. The image processing apparatus determines clipping coordinates corresponding to one or more boundaries to be clipped.

The one or more to-be-cropped boundaries may be boundaries that meet the conditions for triggering the cropping of the original image. For example, if the image processing device determines that the conditions for triggering the cropping of the upper boundary and the lower boundary of the original image are met, then the upper boundary and the lower boundary of the original image are boundaries to be cropped. In some embodiments, after the image processing device determines that the condition for triggering the cropping of the original image is met, the cropping coordinates corresponding to the boundaries to be cropped can be respectively determined. Determining the cropping coordinates corresponding to a boundary to be cropped by the image processing device may be: determining the number of pixel crops corresponding to the boundary to be cropped, for example c_u=delta_u-crop_u_thr; parallel to the boundary to be cropped and distanced from the boundary to be cropped by the pixel The coordinates on the straight line of the number of clippings are used as the clipping coordinates corresponding to the boundary to be clipped (corresponding to the first clipping boundary). It should be understood that the image processing device can determine the clipping coordinates corresponding to any boundary to be clipped in a similar manner. For example, a boundary to be cropped of the original image is the left boundary, and the number of pixels to be cropped corresponding to the boundary to be cropped is c_l=delta_l-crop_l_thr, which will be parallel to the boundary to be cropped and at a distance c_l (for example, 10 pixels) on the straight line as the clipping coordinates corresponding to the border to be cropped. For another example, a boundary to be cropped of the original image is the lower boundary, and the number of pixels to be cropped corresponding to the boundary to be cropped is c_d=delta_d-crop_d_thr, which will be parallel to the boundary to be cropped and at a distance c_d from the boundary to be cropped (for example 10 pixels) on a straight line as the clipping coordinates corresponding to the boundary to be clipped.

606. The image processing device crops the original image according to the clipping coordinates corresponding to one or more boundaries to be cropped, to obtain the first image.

The clipping coordinates corresponding to any boundary to be clipped may be understood as coordinates of a new boundary corresponding to the boundary to be clipped. For example, a boundary to be cropped of the original image is the left boundary, and the cropping coordinates corresponding to the boundary to be cropped are the coordinates of the new left boundary. In some embodiments, the image processing device may crop the original image according to the cropping coordinates corresponding to each boundary to be cropped.

607. The image processing apparatus scales the first image to the same size as the original image.

In some embodiments, the image processing device may perform scaling and interpolation on the cropped original image (corresponding to the first image) according to the width, height, and width-to-height ratio of the original image to obtain the second image; The width, height, and ratio of width to height are the same as those of the above original image, respectively.

In the embodiment of the present application, considering the size of each face area and the distance between the boundaries of each face area and each boundary of the original image, the edge of the image can be cropped in a reasonable proportion.

FIG. 7 is a flowchart of an image distortion correction processing method provided by an embodiment of the present application. As shown in Figure 7, the method includes:

701. The image processing apparatus acquires an input image.

702. Divide the input image into grid points, and acquire the original coordinates of each grid point in the input image.

A possible implementation of step 702 is as follows: select the number of grid points in the horizontal direction and the number of grid points in the vertical direction; The pixel interval between adjacent grid points and the pixel interval between adjacent grid points in the vertical direction; according to the pixel interval between adjacent grid points in the horizontal direction and the pixel interval between adjacent grid points in the vertical direction , divide the input image into grid points; obtain the original coordinates of each grid point in the input image.

703. Perform spherical projection transformation on the input image, and collect grid point coordinates of the input image after spherical projection.

In this application, grid point coordinates refer to coordinates of grid points.

704. Perform portrait segmentation processing on the input image, and acquire portrait region information after the portrait segmentation processing.

The portrait area information (corresponding to the portrait segmentation result) indicates the portrait area (one or more) obtained by subjecting the input image to the portrait segmentation process.

705. Perform face detection on the input image to acquire face region information.

The face area information (corresponding to the face detection result) indicates the face area(s) obtained by performing face detection on the input image.

706. Perform intersection processing on the portrait area and the face area to obtain the portrait and face area in the input image.

707. Collect coordinate information of grid points whose original coordinates fall within the area of the face of the portrait after spherical projection.

708. Calculate weight factors in the face area of the portrait.

The image processing device may calculate the weighting factor in the portrait and face area in any manner, which is not limited in this embodiment of the present application.

709. Calculate the radial weight factor of the input image.

While correcting the distortion of the image, it will have certain side effects on the shape of the content in the image. For example, after the distortion correction process is performed on the human face, abnormal deformation will occur, and the original real shape of the original human face will be lost, which will have a great impact on the beauty of the human face. In the middle part of the image, since the original distortion is relatively light, it has little influence on the deformation of the image content such as the face. At the edge of the photo taken by the wide-angle lens, due to the strong distortion correction, the image content such as the face has a strong deformation effect. Therefore, it is necessary to consider the position difference of the face in the image to correct the face deformation. To calculate the face deformation correction strength, the sigmoid function distribution can be used. The face distortion correction in the center of the image is weak. The deformation correction is stronger at the edge of the image. The image processing device may input the radial weight factor of the image in any manner, which is not limited in this embodiment of the present application.

710. Calculate a coefficient matrix of optimized grid points according to the weight factors in the portrait and face area and the radial weight factors of the input image.

711. Solve the linear equation constructed by using the coefficient matrix to obtain the coordinate information of the optimized grid points.

712. Perform point-by-point interpolation calculation for each pixel in the input image according to the coordinate information of the optimized grid points.

713. Crop and scale the obtained image after point-by-point interpolation calculation for each pixel.

The image processing device may use the image cropping method provided in this application to crop and scale the image obtained after point-by-point interpolation calculation of each pixel.

In the embodiment of the present application, the distorted image can be effectively corrected, and the side effects on the shape of the image content can be minimized while correcting the distortion of the image.

FIG. 8 is a schematic structural diagram of an image processing device provided by an embodiment of the present application. As shown in Figure 8, the image processing device includes:

The acquisition unit 801 is used for the first face area information; the first face area information represents the first face area in the original image, and the original image is a distortion-corrected image;

The acquisition unit 801 is further configured to acquire second face area information based on the preset scale factor and the first face area information; the second face area information represents that the first face area is adjusted according to the preset scale coefficient The second face area of ;

An image cropping unit 802, configured to crop the original image based on the boundary between the second face area and the original image to obtain a first image.

In a possible implementation manner, the image cropping unit 802 is specifically configured to, when the number of pixels beyond the first boundary of the above-mentioned second human face area beyond the first boundary of the above-mentioned original image is greater than the first threshold, the first A cropping boundary is used as the new first boundary of the original image to crop the above-mentioned original image to obtain the above-mentioned first image; the above-mentioned first cropping boundary is parallel to the first boundary of the above-mentioned original image, and the above-mentioned first cropping boundary is parallel to the first boundary of the above-mentioned original image The number of pixels between the boundaries is equal to the difference between the number of pixels where the first boundary of the second human face area exceeds the first boundary of the original image and the first threshold.

In a possible implementation manner, the image cropping unit 802 is specifically configured to make the distance between the second boundary of the first human face area and the second boundary of the original image smaller than a second threshold and the second human face In the case that the area is located in the above-mentioned original image, the above-mentioned original image is cropped with the second clipping boundary as the new second boundary of the above-mentioned original image to obtain the above-mentioned first image; the above-mentioned second clipping boundary is the second boundary of the above-mentioned second human face area .

In a possible implementation manner, the above-mentioned first face area information includes: the vertex coordinates of the above-mentioned first face area and the length and width of the above-mentioned first face area; The product of the length and width of the human face area and the above-mentioned preset proportional coefficient is used as the length and width of the second human face area, and the above-mentioned second human face area is determined according to the vertex coordinates, length and width of the first human face area vertex coordinates.

In a possible implementation manner, the above-mentioned apparatus further includes: a processing unit 803, configured to perform scaling and interpolation on the above-mentioned first image according to the width, height, and ratio of width to height of the above-mentioned original image to obtain a second image; The width, height and ratio of width to height of the second image are respectively the same as the width, height and ratio of width to height of the original image.

In a possible implementation manner, the above-mentioned processing unit 803 is further configured to perform distortion correction processing on the input image to obtain the above-mentioned original image.

FIG. 9 is a schematic structural diagram of a server provided by an embodiment of the present application. The server 900 may have relatively large differences due to different configurations or performances, and may include one or more CPU922 (for example, one or more processors) and Storage 932, one or more storage media 930 (such as one or more mass storage devices) for storing application programs 942 or data 944 . Wherein, the memory 932 and the storage medium 930 may be temporary storage or persistent storage. The program stored in the storage medium 930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 922 may be configured to communicate with the storage medium 930 , and execute a series of instruction operations in the storage medium 930 on the server 900 . The server 900 can execute the image cropping method provided in this application.

The server 900 can also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input and output interfaces 958, and/or, one or more operating systems 941, such as Windows Server™, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

The steps performed by the image processing apparatus in the above embodiments may be based on the server structure shown in FIG. 9 . For example, the central processing unit 922 can realize the functions of the acquiring unit 801, the image cropping unit 802, and the processing unit 803 in FIG. 8 .

FIG. 10 is a schematic structural diagram of a terminal device provided in an embodiment of the present application. As shown in FIG. 10, the terminal device 100 includes a processor 1001, a memory 1002, a communication interface 1003, and an input and output device 1004; the processor 1001, the memory 1002, and the communication interface 1003 are connected to each other through a bus. The terminal device in FIG. 10 may be the image processing apparatus in the foregoing embodiments.

Memory 1002 includes, but is not limited to, random access memory (random access memory, RAM), read-only memory (read-only memory, ROM), erasable programmable read-only memory (erasable programmable read only memory, EPROM), or portable Read-only memory (compact disc read-only memory, CDROM), the memory 1002 is used for related instructions and data. The communication interface 1003 is used to receive and send data. The input and output devices 1004 may include input devices such as keyboards, mice, and touch screens, and output devices such as monitors and screens. The user can input the image to be processed through the input device.

The processor 1001 may be one or more central processing units (central processing unit, CPU). In the case where the processor 1001 is a CPU, the CPU may be a single-core CPU or a multi-core CPU. The steps performed by the image processing apparatus in the foregoing embodiments may be based on the structure of the terminal device shown in FIG. 10 . Specifically, the processor 1001 may implement the functions of the acquiring unit 801, the image cropping unit 802, and the processing unit 803 in FIG. 8 .

In an embodiment of the present application, a computer-readable storage medium is provided, the above-mentioned computer-readable storage medium stores a computer program, and when the above-mentioned computer program is executed by a processor, the image cropping method provided in the above-mentioned embodiment is implemented.

An embodiment of the present application provides a computer program product containing instructions, which when run on a computer, causes the computer to execute the image cropping method provided in the foregoing embodiments.

The above is only a specific embodiment of the application, but the scope of protection of the application is not limited thereto. Any person familiar with the technical field can easily think of various equivalents within the scope of the technology disclosed in the application. Modifications or replacements, these modifications or replacements shall be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

An image cropping method, characterized in that the method comprises:

Obtaining the first face area information; the first face area information characterizes the first face area in the original image;

Based on the preset proportional coefficient and the first human face region information, the second human face region information is obtained; the second human face region information represents the first human face region adjusted according to the preset proportional coefficient. Two face area;

Based on the boundary between the second face area and the original image, crop the original image to obtain a first image.
The method according to claim 1, wherein the cutting the original image based on the boundary of the second human face area and the original image to obtain the first image comprises:

When the number of pixels of the first boundary of the second human face area beyond the first boundary of the original image is greater than the first threshold, the first cropping boundary is used as the new first boundary of the original image to crop the first boundary. The original image is obtained to obtain the first image; the first cropping boundary is parallel to the first boundary of the original image, and the number of pixels between the first cropping boundary and the first boundary of the original image is equal to A difference between the number of pixels that the first boundary of the second face area exceeds the first boundary of the original image and the first threshold.
The method according to claim 1, wherein the cutting the original image based on the boundary of the second human face area and the original image to obtain the first image comprises:

When the distance between the second boundary of the first human face area and the second boundary of the original image is less than a second threshold and the second human face area is located in the original image, the second cropping The boundary is used as a new second boundary of the original image to crop the original image to obtain the first image; the second cropping boundary is a second boundary of the second face area.
The method according to any one of claims 1 to 3, wherein the first face area information includes: the vertex coordinates of the first face area and the length and width of the first face area ; The acquisition of the second face area information based on the preset scale coefficient and the first face area information includes:

Taking the product of the length and width of the first human face area and the preset proportional coefficient as the length and width of the second human face area, and according to the vertex coordinates, length and Width determines the vertex coordinates of the second face area.
The method according to any one of claims 1 to 4, characterized in that, after cutting the original image based on the boundary of the second human face area and the original image to obtain the first image, the method Also includes:

Scale and interpolate the first image according to the width, height and ratio of width to height of the original image to obtain a second image; the width, height and ratio of width to height of the second image are respectively the same as the original The image has the same width, height, and width-to-height ratio.
An image processing device, characterized in that it comprises:

The acquisition unit is used for the first face area information; the first face area information represents the first face area in the original image;

The acquisition unit is further configured to acquire second face area information based on a preset scale factor and the first face area information; the second face area information represents the first face area according to the The second face area adjusted by the preset scale factor;

An image cropping unit, configured to crop the original image based on the boundary of the second human face area and the original image to obtain a first image.
The device according to claim 6, characterized in that,

The image cropping unit is specifically configured to use the first cropping boundary as the first boundary when the number of pixels beyond the first boundary of the original image is greater than a first threshold. The new first boundary of the original image crops the original image to obtain the first image; the first cropping boundary is parallel to the first boundary of the original image, and the first cropping boundary is parallel to the first boundary of the original image The number of pixels between a boundary is equal to the difference between the number of pixels where the first boundary of the second face area exceeds the first boundary of the original image and the first threshold.
The device according to claim 6, characterized in that,

The image cropping unit is specifically configured to: when the distance between the second boundary of the first human face area and the second boundary of the original image is less than a second threshold and the second human face area is located in the original In the case of an image, use the second cropping boundary as the new second boundary of the original image to crop the original image to obtain the first image; the second cropping boundary is the second edge of the second human face area. boundary.
The device according to any one of claims 6 to 8, wherein the information of the first human face area includes: the coordinates of vertices of the first human face area and the length and width of the first human face area ;

The obtaining unit is specifically configured to use the product of the length and width of the first human face area and the preset proportional coefficient as the length and width of the second human face area, and according to the first human face area The vertex coordinates, length and width of the face area determine the vertex coordinates of the second human face area.
The device according to any one of claims 6 to 9, wherein the device further comprises:

A zoom interpolation unit, configured to perform zoom interpolation on the first image according to the width, height, and ratio of width to height of the original image to obtain a second image; the width, height, and ratio of width to height of the second image The ratios are the same as the width, height and width-to-height ratio of the original image, respectively.
A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes The method according to any one of claims 1 to 5.
An electronic device, characterized by comprising a memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, so that the processor performs any of claims 1 to 5. one of the methods described.