CN110610171A - Image processing method and apparatus, electronic device, computer-readable storage medium - Google Patents

Abstract

The present application relates to an image processing method and apparatus, an electronic device, and a computer-readable storage medium. The method includes acquiring an image to be recognized; detecting whether there is a human face in the image to be recognized; when there is a human face in the to-be-recognized image, acquiring a candidate area containing a human portrait according to the human face; the human portrait includes the face; input the candidate area into a portrait segmentation network to obtain a portrait area, and use the portrait area as the main area of the image to be recognized; when there is no face in the image to be recognized, the image to be recognized The image is input into the subject recognition network to obtain the subject area of the image to be recognized. The above-mentioned method and apparatus, electronic device, and computer-readable storage medium can improve the accuracy of subject identification.

Description

Image processing method and apparatus, electronic device, computer-readable storage medium

technical field

The present application relates to the field of imaging technologies, and in particular, to an image processing method, apparatus, electronic device, and computer-readable storage medium.

Background technique

With the development of imaging technology, people are more and more accustomed to shooting images or videos through image acquisition devices such as cameras on electronic devices to record various information. After an electronic device acquires an image, it is often necessary to perform subject recognition on the image to identify the subject, so that a clearer image of the subject can be acquired. However, the traditional subject recognition technology often regards the most conspicuous area as the subject when recognizing the portrait, and cannot accurately identify the portrait, and there is a problem of inaccurate image processing.

SUMMARY OF THE INVENTION

Embodiments of the present application provide an image processing method, apparatus, electronic device, and computer-readable storage medium, which can improve the accuracy of subject identification.

An image processing method, comprising:

Get the image to be recognized;

Detecting whether there is a human face in the to-be-recognized image;

When there is a human face in the to-be-recognized image, obtain a candidate area containing a portrait according to the human face; the human portrait includes the human face;

Inputting the candidate region into a portrait segmentation network to obtain a portrait region, and using the portrait region as the subject region of the image to be recognized;

When there is no face in the to-be-recognized image, the to-be-recognized image is input into a subject recognition network to obtain the subject area of the to-be-recognized image.

An image processing device, comprising:

an image acquisition module, used to acquire an image to be recognized;

a face detection module for detecting whether there is a face in the to-be-recognized image;

a candidate region acquisition module, configured to acquire a candidate region containing a portrait according to the human face when there is a human face in the to-be-recognized image; the human portrait includes the human face;

a portrait segmentation module, configured to input the candidate region into a portrait segmentation network to obtain a portrait region, and use the portrait region as the main region of the image to be recognized;

The subject recognition module is configured to input the to-be-recognized image into a subject recognition network when there is no human face in the to-be-recognized image to obtain the subject area of the to-be-recognized image.

An electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the above-mentioned image processing method.

A computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of the above-mentioned method.

The above-mentioned image processing method and device, electronic equipment, and computer-readable storage medium, when detecting that there is a human face in the image to be recognized, obtain a candidate region containing the portrait according to the human face, input the candidate region into the portrait segmentation network, obtain the portrait region, and divide the The portrait area is used as the subject area of the image to be recognized; when it is detected that there is no human face in the image to be recognized, the image to be recognized is input into the subject recognition network to obtain the subject area of the image to be recognized. By designing a two-way network to obtain the subject area of the image to be recognized, that is, when there is no face in the image to be recognized, the subject area is obtained through the subject recognition network; when there is a face in the image to be recognized, it is determined from the image to be recognized. Candidate images containing portraits can be obtained through the portrait segmentation network to obtain more accurate portrait regions as subject regions.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1 is a schematic diagram of an image processing circuit in one embodiment;

2 is a flowchart of an image processing method in one embodiment;

3 is a flowchart of an image processing method in another embodiment;

4 is a flowchart of steps to obtain candidate regions in one embodiment;

5 is a flowchart of steps in one embodiment to determine the angle of a human face;

6 is a schematic diagram of image processing in another embodiment;

7 is a structural block diagram of an image processing apparatus in one embodiment;

FIG. 8 is a schematic diagram of the internal structure of an electronic device in one embodiment.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, a first portrait area may be referred to as a second portrait area, and similarly, a second portrait area may be referred to as a first portrait area, without departing from the scope of this application. Both the first portrait area and the second portrait area are portrait areas, but they are not the same portrait area.

Embodiments of the present application provide an electronic device. The above electronic device includes an image processing circuit, and the image processing circuit may be implemented by hardware and/or software components, and may include various processing units that define an ISP (Image Signal Processing, image signal processing) pipeline. FIG. 1 is a schematic diagram of an image processing circuit in one embodiment. As shown in FIG. 1 , for the convenience of description, only various aspects of the image processing technology related to the embodiments of the present application are shown.

As shown in FIG. 1 , the image processing circuit includes an ISP processor 140 and a control logic 150 . Image data captured by imaging device 110 is first processed by ISP processor 140 , which analyzes the image data to capture image statistics that can be used to determine and/or control one or more parameters of imaging device 110 . Imaging device 110 may include a camera having one or more lenses 112 and an image sensor 114 . Image sensor 114 may include an array of color filters (eg, Bayer filters), image sensor 114 may obtain light intensity and wavelength information captured with each imaging pixel of image sensor 114 and provide a set of raw materials that may be processed by ISP processor 140 . image data. The attitude sensor 120 (eg, three-axis gyroscope, Hall sensor, accelerometer) may provide the acquired image processing parameters (eg, anti-shake parameters) to the ISP processor 140 based on the interface type of the attitude sensor 120 . The attitude sensor 120 interface may utilize a SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above interfaces.

Additionally, the image sensor 114 may also send raw image data to the gesture sensor 120 , the sensor 120 may provide the raw image data to the ISP processor 140 based on the gesture sensor 120 interface type, or the gesture sensor 120 may store the raw image data to the image memory 130 middle.

The ISP processor 140 processes raw image data pixel by pixel in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 140 may perform one or more image processing operations on the raw image data, collecting statistical information about the image data. Among them, the image processing operations can be performed with the same or different bit depth precision.

The ISP processor 140 may also receive image data from the image memory 130 . For example, the gesture sensor 120 interface sends the raw image data to the image memory 130, and the raw image data in the image memory 130 is provided to the ISP processor 140 for processing. The image memory 130 may be a part of a memory device, a storage device, or an independent dedicated memory in an electronic device, and may include a DMA (Direct Memory Access, direct memory access) feature.

When receiving raw image data from the image sensor 114 interface or from the attitude sensor 120 interface or from the image memory 130, the ISP processor 140 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 130 for additional processing before being displayed. The ISP processor 140 receives processed data from the image memory 130 and performs image data processing in the original domain and in the RGB and YCbCr color spaces on the processed data. The image data processed by the ISP processor 140 may be output to the display 160 for viewing by a user and/or further processed by a graphics engine or a GPU (Graphics Processing Unit, graphics processor). In addition, the output of the ISP processor 140 may also be sent to the image memory 130 , and the display 160 may read image data from the image memory 130 . In one embodiment, image memory 130 may be configured to implement one or more frame buffers.

Statistics determined by the ISP processor 140 may be sent to the control logic 150 unit. For example, the statistics may include gyroscope vibration frequency, auto exposure, auto white balance, auto focus, flicker detection, black level compensation, lens 112 shading correction, etc. image sensor 114 statistics. Control logic 150 may include a processor and/or microcontroller executing one or more routines (eg, firmware) that may determine control parameters and ISP processing of imaging device 110 based on received statistics control parameters of the controller 140. For example, control parameters of imaging device 110 may include attitude sensor 120 control parameters (eg, gain, integration time for exposure control, stabilization parameters, etc.), camera flash control parameters, camera stabilization displacement parameters, lens 112 control parameters (eg, focus or focal length for zooming), or a combination of these parameters. ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (eg, during RGB processing), and lens 112 shading correction parameters.

In one embodiment, the to-be-recognized image is acquired through the lens 112 and the image sensor 114 in the imaging device (camera) 110 , and the to-be-recognized image is sent to the ISP processor 140 . After the ISP processor 140 receives the to-be-recognized image, it detects whether there is a human face in the to-be-recognized image; when there is a human face in the to-be-recognized image, obtains a candidate area containing the human portrait according to the human face; the human portrait includes human faces; Portrait segmentation, that is, input the candidate region into the portrait segmentation network to obtain the portrait region, and use the portrait region as the main region of the image to be recognized.

When there is no face in the to-be-recognized image, the ISP processor 140 performs subject recognition on the to-be-recognized image, that is, the to-be-recognized image is input into the subject recognition network to obtain the subject area of the to-be-recognized image.

In one embodiment, the identified body regions may be sent to the control logic 150 . After receiving the subject area, the control logic 150 controls the lens 112 in the imaging device 110 to move, so as to focus on the subject corresponding to the subject area and obtain a clearer image of the subject.

FIG. 2 is a flowchart of an image processing method in one embodiment. As shown in FIG. 2 , the image processing method includes steps 202 to 210 .

Step 202, acquiring an image to be recognized.

The image to be recognized refers to an image for recognizing the subject area. By recognizing the to-be-recognized image, the subject in the to-be-recognized image can be obtained. The image to be recognized may be one of RGB (Red, Green, Blue) images, grayscale images, and the like. RGB images can be captured by a color camera. Grayscale images can be captured with a black and white camera. The to-be-recognized image may be stored locally by the electronic device, may also be stored by other devices, may also be stored from the network, or may be captured by the electronic device in real time, which is not limited thereto.

Specifically, the ISP processor or the central processing unit of the electronic device may acquire the image to be recognized from a local or other device or network, or obtain the image to be recognized by photographing a scene with a camera.

Step 204, detecting whether there is a human face in the image to be recognized.

Generally, a human face includes features such as eyes, nose, mouth, ears, eyebrows, etc., and there is a corresponding positional relationship between each feature. For example, the left eye and the right eye are symmetrical, the mouth is left-right symmetrical, the nose is in the middle, and the ears are in the middle. The sides of the face, the eyebrows are above the eyes, etc. The face recognition platform detects whether the image to be recognized contains features such as eyes, nose, mouth, ears, eyebrows, etc. and the positional relationship of each feature, so as to determine whether there is a face in the image to be recognized. In the face recognition platform, a large amount of face information is pre-stored, and the face detection through the face recognition platform can reduce extra overhead and save computer resources.

Step 206 , when there is a human face in the image to be recognized, obtain a candidate area containing the human portrait according to the human face; the human portrait includes human faces.

A portrait refers to an area that contains a human face, that is, a portrait can include parts such as the face, neck, arms, and legs. The candidate area refers to the area that contains the portrait.

It can be understood that the face is generally at the top of the portrait, and the region containing the portrait can be selected downward as a candidate region according to the position of the face. The candidate area can be obtained by box selection, that is, the candidate area is a rectangular area. The candidate area may also be a circular area, a square area, a triangular area, etc., which is not limited thereto.

In one embodiment, when there is a human face in the image to be recognized, a candidate region containing the human portrait is acquired according to the human face. In another embodiment, when there are at least two human faces in the image to be recognized, one human face is determined from the at least two human faces, and a candidate region containing the human portrait is obtained according to the determined human face. Among them, to determine a face from at least two faces, the face with the largest area can be determined by comparing the area of each face; it can also be determined by comparing the position of each face in the to-be-recognized image to determine the closest face. The face in the center of the image to be recognized.

In one embodiment, when there is a face in the image to be recognized, the position of each face is obtained, and when the position of each face is not within the preset range of the image to be recognized, the image to be recognized is input into the subject recognition network. , the step of obtaining the subject area of the image to be recognized.

It can be understood that, when the user is taking a landscape photo, some tourists may be captured in the edge of the photo, and the subject captured by the user is the landscape, not the tourists. Therefore, when there are faces in the to-be-recognized image, but the position of each face is within the preset range of the to-be-recognized image, the step of inputting the to-be-recognized image into the subject recognition network to obtain the subject area of the to-be-recognized image is performed, that is, the to-be-recognized image is obtained. The image is used for subject recognition as an image without a human face. Wherein, the preset range may be the central area of the image to be recognized.

In one embodiment, when there is a face in the image to be recognized, the area of each face is obtained, and when the area of each face is smaller than the area threshold, the image to be recognized is input into the subject recognition network to obtain the subject of the image to be recognized. Area steps.

It is understandable that when the user is taking a landscape photo or other objects, there may be some tourists or other people in the background of the photo, then the subject of the user's photo is the scenery or other objects, not the tourists or other people in the background. . Therefore, when there is a face in the image to be recognized, but the area of each face is smaller than the area threshold, it means that the face is not the face of the subject to be photographed by the user, and the image to be recognized is input into the subject recognition network to obtain the image to be recognized. The step of the subject area of the image is to perform subject recognition on the image to be recognized as an image without a human face.

Step 208 , input the candidate region into the portrait segmentation network to obtain the portrait region, and use the portrait region as the main region of the image to be recognized.

The portrait segmentation network refers to the network used to segment out the portrait area. The segmented portrait region may include all parts of the person, or part of the person. For example, the portrait area may include all parts such as the face, neck, hands, feet, and upper torso; the portrait area may also include only the face, hands, and upper torso.

The candidate area contains a portrait. After the candidate area is input into the portrait segmentation network, the portrait area can be obtained, and the portrait area is used as the main area of the image to be recognized.

The traditional subject recognition technology usually inputs the image to be recognized with a face into the subject recognition network, and the identified portrait area has the problem of inaccurate recognition, and the entire portrait may not be recognized, or the portrait and the area around the portrait are used as In the subject area, there is a problem of inaccurate subject recognition.

In this application, when there is a human face in the image to be recognized, the portrait region can be obtained more accurately through the portrait segmentation network, and the portrait region can be used as the subject area of the image to be recognized, so as to more accurately identify the subject of the image to be recognized. area.

Step 210, when there is no face in the image to be recognized, input the image to be recognized into the subject recognition network to obtain the subject area of the image to be recognized.

Among them, salient object detection refers to automatically processing the region of interest and selectively ignoring the uninteresting region when facing a scene. The region of interest is called the subject region. Among them, the subject refers to various objects, such as flowers, cats, dogs, cows, blue sky, white clouds, backgrounds, etc.

When there is no face in the image to be recognized, the image to be recognized is input into the subject recognition network, and the subject area of the image is recognized through the subject recognition network.

In one embodiment, the subject recognition network performs subject recognition on the image to be recognized to obtain the subject area of the image to be recognized, including: Step 1, generating a center weight map corresponding to the image to be recognized, wherein the weight value represented by the center weight map It gradually decreases from the center to the edge; step 2, the image to be recognized and the center weight map are input into the subject recognition model, and the confidence map of the subject area is obtained, wherein the subject recognition model is based on the image to be recognized and the center weight of the same scene in advance. The model obtained by training the image and the corresponding marked subject mask image; step 3, determining the subject area in the image to be recognized according to the subject area confidence map.

Step 1: Generate a center weight map corresponding to the image to be processed, wherein the weight value represented by the center weight map gradually decreases from the center to the edge.

The central weight map refers to a map used to record the weight values of each pixel in the image to be recognized. The weight value recorded in the center weight map gradually decreases from the center to the four sides, that is, the center weight is the largest, and the weight gradually decreases toward the four sides. The weight value from the image center pixel point to the image edge pixel point of the image to be recognized is gradually reduced by the center weight map.

The ISP processor or the central processing unit can generate the corresponding center weight map according to the size of the image to be recognized. The weight value represented by the center weight map gradually decreases from the center to the four sides. The center weight map can be generated by using a Gaussian function, or using a first-order equation, or a second-order equation. The Gaussian function may be a two-dimensional Gaussian function.

Step 2, input the to-be-recognized image and the center weight map into the subject recognition model, and obtain the subject area confidence map, wherein the subject recognition model is based on the image to be recognized, the depth map, the center weight map and the corresponding image of the same scene in advance. The model obtained by training the annotated subject mask map.

Among them, the subject recognition model is obtained by collecting a large amount of training data in advance, and inputting the training data into the subject recognition model including the initial network weights for training. Each set of training data includes the to-be-recognized image corresponding to the same scene, the center weight map, and the labeled subject mask map. Among them, the image to be recognized and the center weight map are used as the input of the trained subject recognition model, and the marked subject mask map is used as the ground truth that the trained subject recognition model expects to output. The subject mask map is an image filter template used to identify the subject in the image, which can block other parts of the image and filter out the subject in the image. The subject recognition model can be trained to identify and detect various subjects, such as people, flowers, cats, dogs, backgrounds, etc.

Specifically, the ISP processor or the central processing unit can input the to-be-recognized image and the center weight map into the subject recognition model, and perform detection to obtain the subject area confidence map. The confidence map of the subject area is used to record the probability of which identifiable subject the subject belongs to. For example, the probability of a pixel belonging to a person is 0.8, the probability of a flower is 0.1, and the probability of the background is 0.1.

Step 3: Determine the subject area in the to-be-recognized image according to the subject area confidence map.

Among them, the subject refers to various objects, such as people, flowers, cats, dogs, cows, blue sky, white clouds, background, etc. The subject area refers to the required subject, which can be selected as required.

Specifically, the ISP processor or the central processing unit may select the highest or second highest confidence level as the subject in the image to be recognized according to the subject area confidence map, and if there is one subject, the subject will be used as the subject area; if there are multiple subjects , you can select one or more subjects as the subject area as required.

In the image processing method in this embodiment, after acquiring the image to be recognized and generating the center weight map corresponding to the image to be recognized, the image to be recognized and the center weight map are input into the corresponding subject recognition model for detection, and the confidence of the subject area can be obtained. According to the confidence map of the subject area, the subject area in the image to be recognized can be determined. Using the center weight map can make the object in the center of the image easier to detect. Using the trained image to be recognized, the center weight map and the subject mask The subject recognition model obtained by training such as film image can more accurately identify the subject area in the image to be recognized.

In the above-mentioned subject recognition method, when it is detected that there is a human face in the to-be-recognized image, a candidate area containing the portrait is obtained according to the human face, the candidate area is input into the portrait segmentation network to obtain the portrait area, and the portrait area is used as the subject area of the to-be-recognized image; when When it is detected that there is no human face in the to-be-recognized image, the to-be-recognized image is input into the subject recognition network to obtain the subject area of the to-be-recognized image. By designing a two-way network to obtain the subject area of the image to be recognized, that is, when there is no face in the image to be recognized, the subject area is obtained through the subject recognition network; when there is a face in the image to be recognized, it is determined from the image to be recognized. Candidate images containing portraits can be obtained through the portrait segmentation network to obtain more accurate portrait regions as subject regions.

In one embodiment, when there is a human face in the to-be-recognized image, acquiring a candidate area containing the human face according to the human face includes: when it is detected that there are at least two human faces in the to-be-recognized image, separately acquiring the human face area containing the human face ; a face area contains one face; respectively obtain the areas of at least two face areas; compare the areas of at least two face areas, and obtain the face area with the largest area as the first face area; according to the first face area The region acquires the first candidate region containing the portrait. Input the candidate area into the portrait segmentation network to obtain the portrait area, and use the portrait area as the main area of the image to be recognized, including: inputting the first candidate area into the portrait segmentation network to obtain the first portrait area, and using the first portrait area as the to-be-recognized image area. Identify the subject area of the image.

A face area refers to an area containing a human face. The face area can be obtained by frame selection, that is, the face area is a rectangular area. The face area may also be a circular area, a square area, a triangular area, etc., which is not limited thereto. The first face area refers to the face area with the largest area. The first candidate region refers to a candidate region containing a portrait corresponding to the first face region.

It can be understood that, when the area of the face area in the image to be processed is larger, it means that the face is closer to the camera, and the object closer to the camera is the subject that the user wants to photograph. Therefore, the areas of at least two face regions can be obtained respectively; the areas of the at least two face regions are compared, and the face region with the largest area is obtained as the first face region; according to

The first face region acquires a first candidate region containing a portrait. Generally, the face in the face area with the largest area is closest to the camera.

In this embodiment, when it is detected that there are at least two faces in the to-be-recognized image, face areas containing the faces are obtained respectively; areas of at least two face areas are obtained respectively, and the areas of the respective face areas are compared , obtain the face area with the largest area as the first face area, obtain the first candidate area containing the portrait according to the first face area, improve the accuracy of the obtained first face area and the first candidate area, and input The portrait segmentation network can obtain a more accurate first portrait area, and use the first portrait area as the subject area, thereby improving the accuracy of subject recognition. A face is determined from multiple faces, and finally a portrait corresponding to the face is obtained, which avoids the problem of identifying multiple subjects in a multi-face scene, and improves the unity of focusing.

In one embodiment, the face region with the next largest area may also be acquired as the first face region, but not limited to this.

In one embodiment, when there is a human face in the to-be-recognized image, acquiring a candidate area containing the human face according to the human face includes: when it is detected that there are at least two human faces in the to-be-recognized image, separately acquiring the human face area containing the human face A human face area includes a human face; Obtain the position information of at least two human face areas respectively; Obtain the nearest human face area from the center of the image to be recognized according to the position information of at least two human face areas as the second human face area; According to The second face region acquires a second candidate region containing a portrait. Input the candidate area into the portrait segmentation network to obtain the portrait area, and use the portrait area as the main area of the image to be recognized, including: inputting the second candidate area into the portrait segmentation network to obtain the second portrait area, and using the second portrait area as the image to be recognized main area.

A face area refers to an area containing a human face. The face area can be obtained by frame selection, that is, the face area is a rectangular area. The face area may also be a circular area, a square area, a triangular area, etc., which is not limited thereto. The position information of the face area may be represented by the coordinates of the center of the face area, or may be represented by the coordinates of any point in the face area, which is not limited thereto.

The second face area refers to the face area closest to the center of the image to be recognized. The second candidate region refers to a candidate region containing a portrait corresponding to the second face region.

Specifically, the position information of the center of the image to be recognized can be obtained in advance, the position information of at least two face regions is compared with the position information of the center of the image to be recognized, and the center of the image to be recognized can be obtained according to the comparison result. face area.

For example, the position information of the center of the image to be recognized is (50, 50), the position information of the face area A is represented by (40, 30), the position information of the face area B is represented by (20, 50), and the face area The position information of C is (40,40), therefore, the following formula can be used to calculate the distance between each face region and the center of the image to be recognized: Among them, S is the distance between the face area and the center of the image to be recognized, a ₁ and b ₁ are the abscissa and ordinate of one point, respectively, and a ₂ and b ₂ are the abscissa and ordinate of another point, respectively.

Therefore, the distance between the face area A and the center of the image to be recognized is The distance between the face area B and the center of the image to be recognized is The distance between the face area C and the center of the image to be recognized is but The face area C is the closest to the center of the image to be recognized, and the face area C is used as the second face area.

In this embodiment, when it is detected that there are at least two faces in the to-be-recognized image, face regions containing human faces are obtained respectively, and position information of at least two face regions is obtained respectively; The face area is used as the second face area, and the second candidate area containing the portrait can be obtained according to the second face area, and the more accurate second face area and the second candidate area can be obtained, so that the second candidate area can be input into the portrait By segmenting the network, a more accurate second portrait area can be obtained as the subject area. A face is determined from multiple faces, and finally a portrait corresponding to the face is obtained, which avoids the problem of identifying multiple subjects in a multi-face scene, and improves the unity of focusing.

In one embodiment, when it is detected that there are at least two human faces in the image to be recognized, face regions containing human faces are obtained respectively; one face region includes one human face; the areas of at least two face regions are obtained respectively; Obtain candidate face regions whose area is greater than the area threshold in at least two face regions; obtain the position information of each candidate face region respectively; obtain the candidate face region closest to the center of the image to be recognized according to the position information of each candidate face region As the target face area; obtain the candidate area containing the portrait according to the target face area.

Specifically, the area of each face area can be obtained separately, and the candidate face area with the area larger than the area threshold can be obtained, that is, the face area with a smaller area is screened out, and the processing of the face area with a smaller area is avoided, and the improvement of Efficiency of subject recognition.

Obtain the position information of each candidate face region, and obtain the candidate face region closest to the center of the image to be recognized as the target face region, which can obtain a more accurate face region; obtain the candidate region containing the portrait according to the target face region , more accurate candidate regions can be obtained.

In one embodiment, as shown in FIG. 3 , the image to be recognized 302 is acquired; the image to be recognized 302 is subjected to face detection to determine whether there is a human face in the image to be recognized 302 , namely step 304 ; when the image to be recognized 302 does not exist In the case of a human face, the to-be-recognized image 302 is input into the subject recognition network 306 to obtain a subject area 308 of the to-be-recognized image 302 .

When there is a human face in the image to be recognized 302, a human face area 310 containing the human face can be acquired. When there are at least two human faces in the to-be-recognized image 302, face regions including human faces may be acquired respectively, and a human face region 310 may be determined from the at least two human face regions.

In one embodiment, the areas of at least two face regions may be obtained respectively, and the face region with the largest area may be obtained as the first face region, that is, the face region 310 . In another embodiment, the position information of at least two face regions can also be obtained separately, and the face region closest to the center of the image to be recognized is obtained as the second face region, that is, the face region 310 .

A candidate region 310 containing a portrait is obtained according to the face region 310 ; the candidate region 312 is input into the portrait segmentation network 314 to obtain a portrait region, and the portrait region is used as the main region 308 of the image to be recognized 302 .

In one embodiment, as shown in FIG. 4 , a candidate region containing a portrait is obtained according to a human face, including:

Step 402, acquiring a face area including a face.

Step 404, acquiring at least two feature points in the face region.

The feature point refers to the point where the gray value of the image changes drastically or the point with large curvature on the edge of the image (that is, the intersection of two edges). In the face area, the feature points can be eyes, nose, mouth corners, eyebrows, moles, etc.

Step 406: Determine the angle of the human face according to the at least two feature points.

The angle of the face refers to the angle at which the face is tilted. The angle of the face can be tilted to the left, tilted to the right, deflected to the rear left, deflected to the rear right, and so on. For example, the angle of the face can be tilted 20 degrees to the left, 10 degrees to the right, and so on.

It can be understood that when the face is tilted, the feature points in the face region also change the relative positional relationship accordingly. For example, when the face is tilted to the left, a line is connected between two feature points in the face region, that is, the left-eye feature point and the right-eye feature point, and the line is also tilted to the left. For another example, when the face is tilted to the right, two feature points in the face region, that is, the feature point of the left mouth corner and the feature point of the left mouth corner are connected, and the line is also tilted to the right. For another example, when the face is deflected to the rear left, the length of the line connecting the feature point of the left eye and the feature point of the right eye becomes shorter, and the feature point of the nose tip is located to the left of the face area.

Step 408: Obtain a candidate area containing the portrait according to the angle of the human face; the difference between the angle of the candidate area and the angle of the human face is within a preset range.

Generally, when the body is in a tilted state, the human face will be tilted accordingly. Therefore, a candidate region containing the portrait can be obtained according to the angle of the human face, and the difference between the angle of the candidate region and the angle of the human face is within a preset range. The preset range can be set according to user needs. For example, the preset range may be between 5 degrees and 10 degrees.

For example, when the angle of the human face is inclined to the left by 20 degrees, the angle at which the candidate region containing the human portrait is obtained according to the angle of the human face may be 20 degrees, 25 degrees, or the like. A more accurate candidate region can be obtained from the angle of the face.

In this embodiment, a face region containing a human face is acquired, at least two feature points in the face region are acquired, an angle of the human face is determined according to the at least two feature points, and a more accurate candidate can be acquired according to the angle of the human face area.

In one embodiment, as shown in FIG. 5 , the angle of the human face is determined according to at least two feature points, including:

Step 502: Connect at least two feature points to obtain each connection line.

After acquiring at least two feature points in the face region, the at least two feature points are connected. For example, when the feature points are left-eye feature points and right-eye feature points, connect the left-eye feature points and right-eye feature points to obtain a connecting line; when the feature points are left-mouth feature points and right-mouth feature points, connect the left-mouth feature points Connect the feature points and the feature points of the right corner of the mouth to obtain the connecting line; when the feature points are the nose tip feature point, the left eye feature point and the right eye feature point, connect the left eye feature point and the right eye feature point to obtain the connecting line, and then Connect the center of the connecting line to the nose tip feature point to get another connecting line.

Step 504: Acquire the angle of each connection line, and determine the angle of the face based on the angle of each connection line.

The angle of the connecting line can be used to represent the angle of the face. For example, when the angle of the connecting line between the left-eye feature point and the right-eye feature point is deflected to the left by 20 degrees, the angle of the face can be deflected to the left by 20 degrees; If the angle is deflected to the right by 10 degrees, the angle of the face can be deflected to the right by 10 degrees.

When there is a connecting line, the angle of the connecting line can be regarded as the angle of the face. When there are at least two connecting lines, the angle of the human face is determined based on the angles of the at least two connecting lines. For example, when there are two connecting lines, one connecting line is the connecting line between the left eye feature point and the right eye feature point, and the other connecting line is the connecting line between the left mouth corner feature point and the right mouth corner feature point, you can connect the two The angle of the line is averaged, and the average value is taken as the angle of the face.

For another example, when there are two connecting lines, the first connecting line is the connecting line between the left eye feature point and the right eye feature point, and the second connecting line is obtained by connecting the center of the first connecting line with the nose tip feature point, Then the angle of the second connecting line can represent the angle of the face.

In this embodiment, at least two feature points are connected to obtain each connection line, the angle of each connection line is obtained, and a more accurate face angle can be determined based on the angle of each connection line.

In one embodiment, connecting at least two feature points to obtain each connection line includes: when the at least two feature points include a left-eye feature point and a right-eye feature point, connecting the left-eye feature point and the right-eye feature point Point to connect to get the first connection line. Obtaining the angle of each connection, and determining the angle of the face based on the angle of each connection line includes: obtaining the angle of the first connection line, and determining the angle of the face based on the angle of the first connection line.

The first connecting line refers to the connecting line between the left-eye feature point and the right-eye feature point.

It can be understood that, when the human face in the image to be processed is in an oblique state, the line connecting the left eye and the right eye on the human face is also in an oblique state. Therefore, the left-eye feature point and the right-eye feature point can be connected to obtain the first connecting line, the angle of the first connecting line can be obtained, and the angle of the face can be determined based on the angle of the first connecting line.

Further, the angle of the human face is determined based on the angle of the first connecting line; the difference between the angle of the human face and the angle of the first connecting line is within a preset range.

The preset range can be set according to user needs. In one embodiment, the angle of the first connection line may be used as the angle of the human face. In another embodiment, the angle of the human face may also be determined based on the angle of the first connecting line, the angle of the human face is different from the angle of the first connecting line, and the difference between the angle of the human face and the angle of the first connecting line within the preset range. For example, the angle of the first connection line is 10 degrees to the left, and the determined angle of the face may be 8 degrees to the left.

In this embodiment, the first connecting line is obtained by connecting the left-eye feature point and the right-eye feature point, the angle of the first connecting line is obtained, and based on the angle of the first connecting line, a more accurate angle of the face can be determined .

In one embodiment, connecting at least two feature points to obtain each connection line includes: when the at least two feature points include a left mouth corner feature point and a right mouth corner feature point, connecting the left mouth corner feature point and the right mouth corner feature point Point to connect to get the second connecting line. Acquiring the angle of each connection and determining the angle of the face based on the angle of each connection line includes: acquiring the angle of the second connection line, and determining the angle of the face based on the angle of the second connection line.

The second connecting line refers to the connecting line between the feature points of the left corner of the mouth and the feature points of the right corner of the mouth.

It can be understood that when the human face in the image to be processed is in a tilted state, the line connecting the left corner of the mouth and the right corner of the mouth on the human face is also in a tilted state. Therefore, a second connecting line can be obtained by connecting the feature points of the left corner of the mouth and the feature points of the right corner of the mouth, the angle of the second connecting line is obtained, and the angle of the face is determined based on the angle of the second connecting line.

Further, the angle of the human face is determined based on the angle of the second connecting line; the difference between the angle of the human face and the angle of the second connecting line is within a preset range.

The preset range can be set according to user needs. In one embodiment, the angle of the second connection line may be used as the angle of the human face. In another embodiment, the angle of the human face may also be determined based on the angle of the second connecting line, the angle of the human face is different from the angle of the second connecting line, and the difference between the angle of the human face and the angle of the second connecting line within the preset range. For example, the angle of the second connection line is 8 degrees to the left, and the determined angle of the face can be 9 degrees to the left.

In this embodiment, a second connecting line is obtained by connecting the feature point of the left corner of the mouth and the feature point of the right corner of the mouth, the angle of the second connecting line is obtained, and based on the angle of the second connecting line, a more accurate angle of the face can be determined .

In one embodiment, the above method further includes: when the at least two feature points include a left-eye feature point, a right-eye feature point, a left-mouth corner feature point, and a right-mouth corner feature point, comparing the left-eye feature point and the right-eye feature point The first connecting line is obtained by connecting, and the second connecting line is obtained by connecting the feature point of the left corner of the mouth and the feature point of the right corner of the mouth; the angle of the first connecting line and the angle of the second connecting line are obtained respectively, and the angle of the first connecting line and the angle of the second connecting line are obtained. The angle of the second connecting line determines the angle of the face.

In one embodiment, the angle of the first connection line and the angle of the second connection line may be averaged, and the average value may be used as the angle of the human face. In another embodiment, the first weighting factor of the angle of the first connecting line and the second weighting factor of the angle of the second connecting line may also be obtained, according to the angle of the first connecting line, the first weighting factor, the second weighting factor A weighted average is obtained from the angle of the connecting line and the second weighting factor, and the weighted average is taken as the angle of the face.

In this embodiment, the first connecting line and the second connecting line are obtained based on the left eye feature point, the right eye feature point, the left mouth corner feature point and the right mouth corner feature point, and the angle of the first connecting line and the second connecting line are obtained according to the angle of the first connecting line and the second connecting line. The angle can get a more accurate angle of the face.

In one embodiment, the above method further includes: when the at least two feature points further include a nose tip feature point, acquiring position information of the nose tip feature point in the face region. The determining the angle of the face based on the angle of the first connecting line includes: determining the angle of the face based on the angle of the first connecting line and the position information of the nose tip feature point in the face region.

The position information of the nose tip feature point in the face region may be represented by coordinates, and may also be represented by the distance between the nose tip feature point and the center of the face region, which is not limited thereto.

Generally, the tip of the nose is located in the central area of the face. By obtaining the position information of the nose tip feature point in the face area, the front and rear orientation of the face can be known. For example, when the human face is turned to the left and rear, the nose tip is to the left in the human face area; when the human face is turned to the right rear, the nose tip is to the right in the human face area. Based on the angle of the first connecting line and the position information of the nose tip feature point in the face region, the angle of the face can be determined more accurately.

In another embodiment, the above method further includes: when the at least two feature points further include a nose tip feature point, acquiring position information of the nose tip feature point in the face region. The determining the angle of the face based on the angle of the second connecting line includes: determining the angle of the face based on the angle of the second connecting line and the position information of the nose tip feature point in the face region.

For example, when the human face is turned to the left and rear, the nose tip is to the left in the human face area; when the human face is turned to the right rear, the nose tip is to the right in the human face area. Based on the angle of the second connecting line and the position information of the nose tip feature point in the face region, the angle of the face can be more accurately determined.

In one embodiment, when the at least two feature points further include the nose tip feature point, the position information of the nose tip feature point in the face region is obtained; the angle of the first connecting line and the angle of the second connecting line are obtained respectively, and based on The angle of the first connecting line, the angle of the second connecting line and the position information of the nose tip feature point in the face region determine the angle of the face.

For example, if the angle of the first connection line is 8 degrees to the left, the angle of the second connection line is 10 degrees to the left, and the nose tip feature point is located to the left in the face area, then based on the angle of the first connection line and the The angle of the two connecting lines can be averaged, and the face is deflected to the left by 9 degrees; the feature point of the nose tip is located on the left side of the face area, indicating that the face is deflected to the left and rear; therefore, the angle of the face is a left deflection 9 degrees and turn left rear.

In this embodiment, the angle of the human face can be more accurately determined by combining the left eye feature point, the right eye feature point, the left mouth corner feature point, the right mouth corner feature point and the nose tip feature point.

In one embodiment, the above method further includes: acquiring the area of the face region. Obtaining a candidate region containing a portrait according to the angle of the face includes: obtaining a candidate region containing the portrait according to the angle of the face and the area of the face region; the area of the candidate region is positively correlated with the area of the face region.

It can be understood that when the area of the face area is larger, it means that the face is closer to the camera, the area of the portrait including the face is also larger, and the candidate area including the portrait is also larger. Therefore, the area of the candidate region is positively correlated with the area of the face region. According to the angle of the face and the area of the face region, a more accurate candidate region can be obtained.

In one embodiment, as shown in FIG. 6 , the to-be-recognized image 602 is acquired; the to-be-recognized image 602 is subjected to face detection, and two faces are detected. Obtain the face regions containing the faces respectively, and obtain two face regions. The position information of the two face regions can be obtained respectively, and the face region closest to the center of the image to be recognized, that is, the face region 604 can be obtained.

Obtain 5 feature points from the face area 604, which are respectively the feature point of the left eye, the feature point of the right eye, the feature point of the tip of the nose, the feature point of the left mouth corner and the feature point of the right mouth corner; connect the feature points of the left eye and the right eye Obtain the first connecting line, connect the feature points of the left corner of the mouth and the feature points of the right mouth corner to obtain the second connecting line; obtain the angle of the first connecting line and the angle of the second connecting line respectively; based on the angle of the first connecting line and the second connecting line The angle of the connecting line is to connect the nose tip feature points with the first connecting line and the second connecting line respectively to obtain the third connecting line.

For example, the average value of the angle of the first connecting line and the angle of the second connecting line can be obtained, and the straight line corresponding to the average value is perpendicular to the third connecting line passing through the nose tip feature point.

The angle of the third connecting line is obtained; the angle of the face is determined based on the angle of the third connecting line; the candidate area 606 containing the portrait is obtained according to the angle of the face. Wherein, the difference between the angle of the candidate area and the angle of the face is within a preset range. By inputting the candidate region 606 into the portrait segmentation network, a portrait region 608 can be obtained, and the portrait region 608 is used as the main region of the image to be recognized.

It should be understood that although the steps in the flowcharts of FIGS. 2 , 4 and 5 are sequentially displayed in accordance with the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in FIG. 2 , FIG. 4 and FIG. 5 may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. The order of execution of the sub-steps or phases is also not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or phases of the other steps.

FIG. 7 is a structural block diagram of an image processing apparatus according to an embodiment. As shown in FIG. 7, an image processing apparatus 700 is provided, including: an image acquisition module 702, a face detection module 704, a candidate region acquisition module 706, a portrait segmentation module 708, and a subject recognition module 710, wherein:

An image acquisition module 702, configured to acquire an image to be recognized.

The face detection module 704 is used for detecting whether there is a face in the image to be recognized.

The candidate region obtaining module 706 is configured to obtain a candidate region including a portrait according to the human face when there is a human face in the image to be recognized; the human portrait includes a human face.

The portrait segmentation module 708 is configured to input the candidate region into the portrait segmentation network to obtain the portrait region, and use the portrait region as the main region of the image to be recognized.

The subject recognition module 710 is configured to input the to-be-recognized image into the subject recognition network when there is no human face in the to-be-recognized image to obtain the subject area of the to-be-recognized image.

The above-mentioned image processing device, when detecting that there is a human face in the to-be-recognized image, obtains a candidate area containing the portrait according to the human face, inputs the candidate area into the portrait segmentation network, obtains the portrait area, and uses the portrait area as the main area of the image to be identified; when When it is detected that there is no human face in the to-be-recognized image, the to-be-recognized image is input into the subject recognition network to obtain the subject area of the to-be-recognized image. By designing a two-way network to obtain the subject area of the image to be recognized, that is, when there is no face in the image to be recognized, the subject area is obtained through the subject recognition network; when there is a face in the image to be recognized, it is determined from the image to be recognized. Candidate images containing portraits can be obtained through the portrait segmentation network to obtain more accurate portrait regions as subject regions.

In one embodiment, the above-mentioned candidate region obtaining module 706 is further configured to obtain face regions containing human faces when it is detected that there are at least two faces in the image to be identified; one face region includes one face; Areas of at least two face regions; compare the areas of the at least two face regions, and obtain the face region with the largest area as the first face region; obtain a first candidate region containing a portrait according to the first face region. Input the candidate area into the portrait segmentation network to obtain the portrait area, and use the portrait area as the main area of the image to be recognized, including: inputting the first candidate area into the portrait segmentation network to obtain the first portrait area, and using the first portrait area as the to-be-recognized image area. Identify the subject area of the image.

In one embodiment, the above-mentioned candidate region obtaining module 706 is further configured to obtain face regions containing human faces when it is detected that there are at least two faces in the image to be identified; one face region includes one face; position information of at least two face regions; obtain the face region closest to the center of the image to be recognized as a second face region according to the position information of at least two face regions; obtain a second candidate containing a portrait according to the second face region area. Input the candidate area into the portrait segmentation network to obtain the portrait area, and use the portrait area as the main area of the image to be recognized, including: inputting the second candidate area into the portrait segmentation network to obtain the second portrait area, and using the second portrait area as the image to be recognized main area.

In one embodiment, the above-mentioned candidate region obtaining module 706 is further configured to obtain a face region containing a human face; obtain at least two feature points in the face region; determine the angle of the face according to the at least two feature points; The angle of the face obtains a candidate area containing the portrait; the difference between the angle of the candidate area and the angle of the face is within a preset range.

In one embodiment, the above-mentioned candidate region obtaining module 706 is further configured to connect at least two feature points to obtain each connecting line; obtain the angle of each connecting line, and determine the face value based on the angle of each connecting line angle.

In one embodiment, the above-mentioned candidate region obtaining module 706 is further configured to connect the left-eye feature point and the right-eye feature point to obtain a first connecting line when the at least two feature points include a left-eye feature point and a right-eye feature point . Obtaining the angle of each connection, and determining the angle of the face based on the angle of each connection line includes: obtaining the angle of the first connection line, and determining the angle of the face based on the angle of the first connection line.

In one embodiment, the candidate region obtaining module 706 is further configured to connect the left mouth corner feature point and the right mouth corner feature point to obtain the second connecting line when the at least two feature points include the left mouth corner feature point and the right mouth corner feature point . Acquiring the angle of each connection and determining the angle of the face based on the angle of each connection line includes: acquiring the angle of the second connection line, and determining the angle of the face based on the angle of the second connection line.

In one embodiment, the above-mentioned subject identification apparatus 700 further includes a position information acquisition module, configured to acquire position information of the nose tip feature point in the face region when the at least two feature points further include the nose tip feature point. Determining the angle of the face based on the angle of the first connecting line includes: determining the angle of the face based on the angle of the first connecting line and the position information of the nose tip feature point in the face region. Or determining the angle of the face based on the angle of the second connecting line includes: determining the angle of the face based on the second connecting line and the position information of the nose tip feature point in the face region.

In one embodiment, the above-mentioned image processing apparatus 700 further includes an area acquisition module for acquiring the area of the face region. Obtaining a candidate region containing a portrait according to the angle of the face includes: obtaining a candidate region containing the portrait according to the angle of the face and the area of the face region; the area of the candidate region is positively correlated with the area of the face region.

The division of each module in the above image processing apparatus is only for illustration. In other embodiments, the image processing apparatus may be divided into different modules as required to complete all or part of the functions of the above image processing apparatus.

FIG. 8 is a schematic diagram of the internal structure of an electronic device in one embodiment. As shown in FIG. 8, the electronic device includes a processor and a memory connected by a system bus. Among them, the processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory may include non-volatile storage media and internal memory. The nonvolatile storage medium stores an operating system and a computer program. The computer program can be executed by the processor to implement an image processing method provided by the following embodiments. Internal memory provides a cached execution environment for operating system computer programs in non-volatile storage media. The electronic device may be a mobile phone, a tablet computer, a personal digital assistant or a wearable device, and the like.

The implementation of each module in the image processing apparatus provided in the embodiments of the present application may be in the form of a computer program. The computer program can be run on a terminal or server. The program modules constituted by the computer program can be stored in the memory of the terminal or the server. When the computer program is executed by the processor, the steps of the methods described in the embodiments of the present application are implemented.

The embodiment of the present application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, when executed by one or more processors, cause the processors to perform the steps of an image processing method.

A computer program product containing instructions, when run on a computer, causes the computer to perform an image processing method.

Any reference to a memory, storage, database, or other medium as used in embodiments of the present application may include non-volatile and/or volatile memory. Suitable nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Memory Bus (Rambus) Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM).

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent of the present application. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims (11)

1. An image processing method, comprising:

acquiring an image to be identified;

detecting whether a human face exists in the image to be recognized;

when a face exists in the image to be recognized, acquiring a candidate region containing a portrait according to the face; the portrait includes the face;

inputting the candidate area into a portrait segmentation network to obtain a portrait area, and taking the portrait area as a main area of the image to be identified;

and when the face does not exist in the image to be recognized, inputting the image to be recognized into a main body recognition network to obtain a main body area of the image to be recognized.

2. The method according to claim 1, wherein when a face exists in the image to be recognized, acquiring a candidate region containing the face according to the face comprises:

when detecting that at least two faces exist in the image to be recognized, respectively acquiring face regions containing the faces;

respectively acquiring the areas of at least two face regions;

comparing the areas of at least two face regions to obtain a face region with the largest area as a first face region;

acquiring a first candidate region containing a portrait according to the first face region;

inputting the candidate region into a portrait segmentation network to obtain a portrait region, and using the portrait region as a main region of the image to be recognized, including:

and inputting the first candidate region into a portrait segmentation network to obtain a first portrait region, and taking the first portrait region as a main region of the image to be identified.

3. The method according to claim 1, wherein when a face exists in the image to be recognized, acquiring a candidate region containing the face according to the face comprises:

when detecting that at least two faces exist in the image to be recognized, respectively acquiring face regions containing the faces; a face region contains a face;

respectively acquiring the position information of at least two face areas;

acquiring a face area closest to the center of the image to be recognized according to the position information of at least two face areas as a second face area;

acquiring a second candidate region containing a portrait according to the second face region;

inputting the candidate region into a portrait segmentation network to obtain a portrait region, and using the portrait region as a main region of the image to be recognized, including:

and inputting the second candidate area into a portrait segmentation network to obtain a second portrait area, and taking the second portrait area as a main area of the image to be identified.

4. The method of claim 1, wherein obtaining candidate regions containing human images from the human face comprises:

acquiring a face area containing the face;

acquiring at least two feature points in the face region;

determining the angle of the face according to the at least two feature points;

acquiring a candidate region containing a portrait according to the angle of the face; and the difference value between the angle of the candidate region and the angle of the human face is within a preset range.

5. The method of claim 4, wherein determining the angle of the face from the at least two feature points comprises:

connecting the at least two characteristic points to obtain each connecting line;

and acquiring the angle of each connecting line, and determining the angle of the face based on the angle of each connecting line.

6. The method of claim 5, wherein said connecting the at least two feature points to obtain each connecting line comprises:

when the at least two feature points comprise a left-eye feature point and a right-eye feature point, connecting the left-eye feature point and the right-eye feature point to obtain a first connecting line; or

When the at least two feature points comprise a left mouth corner feature point and a right mouth corner feature point, connecting the left mouth corner feature point and the right mouth corner feature point to obtain a second connecting line;

the obtaining the angles of the connections and determining the angles of the human face based on the angles of the connections include:

acquiring the angle of the first connecting line, and determining the angle of the face based on the angle of the first connecting line; or

And acquiring the angle of the second connecting line, and determining the angle of the face based on the angle of the second connecting line.

7. The method of claim 6, further comprising:

when the at least two feature points further comprise nose tip feature points, acquiring position information of the nose tip feature points in the face region;

the determining the angle of the human face based on the angle of the first connecting line includes: determining the angle of the face based on the angle of the first connecting line and the position information of the nose tip characteristic point in the face region; or

The determining the angle of the face based on the angle of the second connecting line includes: and determining the angle of the face based on the second connecting line and the position information of the nose tip characteristic point in the face region.

8. The method of claim 4, further comprising:

acquiring the area of the face region;

the obtaining of the candidate region containing the portrait according to the angle of the face includes:

acquiring a candidate region containing a portrait according to the angle of the face and the area of the face region; the area of the candidate region is positively correlated with the area of the face region.

9. An image processing apparatus characterized by comprising:

the image acquisition module is used for acquiring an image to be identified;

the face detection module is used for detecting whether a face exists in the image to be identified;

the candidate region acquisition module is used for acquiring a candidate region containing a portrait according to the face when the face exists in the image to be identified; the portrait includes the face;

the portrait segmentation module is used for inputting the candidate region into a portrait segmentation network to obtain a portrait region, and the portrait region is used as a main region of the image to be identified;

and the main body identification module is used for inputting the image to be identified into a main body identification network when the human face does not exist in the image to be identified, so as to obtain a main body area of the image to be identified.

10. An electronic device comprising a memory and a processor, the memory having stored therein a computer program that, when executed by the processor, causes the processor to perform the steps of the image processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 8.