CN111275045A - Method and device for identifying image subject, electronic equipment and medium - Google Patents

Method and device for identifying image subject, electronic equipment and medium Download PDF

Info

Publication number
CN111275045A
CN111275045A CN202010131197.8A CN202010131197A CN111275045A CN 111275045 A CN111275045 A CN 111275045A CN 202010131197 A CN202010131197 A CN 202010131197A CN 111275045 A CN111275045 A CN 111275045A
Authority
CN
China
Prior art keywords
main body
target image
difference degree
difference
degree
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010131197.8A
Other languages
Chinese (zh)
Inventor
贾玉虎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to CN202010131197.8A priority Critical patent/CN111275045A/en
Publication of CN111275045A publication Critical patent/CN111275045A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/26Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/44Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components

Abstract

The application provides a method, a device, an electronic device and a medium for identifying a subject of an image, wherein the method comprises the following steps: by acquiring a target image; determining a first difference degree of values of all the pixel points in the central area and a second difference degree of values of all the pixel points in the edge area; determining a maximum of the first degree of difference and the second degree of difference; if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area; if the maximum value is the first difference degree, performing general main body segmentation on the main body in the target image to obtain a second main body area; focusing according to the first body region or the second body region. Therefore, the main body segmentation is carried out by adopting a corresponding segmentation method through determining the region corresponding to the maximum value of the difference degree of the values of all the pixel points, the detection rate of the edge main body is improved, and the problem of focusing failure caused by low detection rate of the edge main body in the existing detection method is solved.

Description

Method and device for identifying image subject, electronic equipment and medium
Technical Field
The present application relates to the field of image recognition technologies, and in particular, to a method and an apparatus for recognizing a subject of an image, an electronic device, and a medium.
Background
With the development of terminal devices, more and more users are used to shoot images or videos through imaging devices such as cameras on electronic devices. After the electronic device acquires the image, the image is often subjected to subject detection to detect the subject, so that a clearer image of the subject can be acquired.
However, when the subject in the captured image is not in the center position but in the edge area, the conventional subject detection method has a poor detection effect, and a large number of edge subjects are missed, thereby causing a problem of failure in auto-focusing.
Disclosure of Invention
The present application is directed to solving, at least to some extent, one of the technical problems in the related art.
An embodiment of a first aspect of the present application provides a method for identifying a subject of an image, including:
acquiring a target image; wherein the target image includes a central region and an edge region surrounding the central region;
determining a first difference degree of values of all the pixels in the central area and a second difference degree of values of all the pixels in the edge area;
determining a maximum of the first degree of difference and the second degree of difference;
if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area;
if the maximum value is the first difference degree, performing general main body segmentation on a main body in the target image to obtain a second main body area;
focusing according to the first body region or the second body region.
As a first possible implementation manner of the embodiment of the present application, before determining a maximum value of the first difference degree and the second difference degree, the method further includes:
comparing the first difference degree and the second difference degree with a set threshold value respectively;
and determining that the main body exists in the target image according to the fact that at least one of the first difference degree and the second difference degree is larger than a set threshold value.
As a second possible implementation manner of the embodiment of the present application, after comparing the first difference degree and the second difference degree with a set threshold, the method further includes:
and according to the fact that the first difference degree and the second difference degree are both smaller than the set threshold value, determining that no main body exists in the target image, and focusing to the center of the target image.
As a third possible implementation manner of the embodiment of the present application, the determining a first difference degree of values of each pixel point in the central area and a second difference degree of values of each pixel point in the edge area includes:
calculating gradient values of all pixel points in the central region and the edge region;
taking the sum of the gradient values of all the pixel points in the central area as the first difference degree;
and taking the sum of the gradient values of all the pixel points in the edge area as the second difference degree.
As a fourth possible implementation manner of the embodiment of the present application, the gradient value is used to indicate a difference between a corresponding pixel point and a neighboring pixel point.
As a fifth possible implementation manner of the embodiment of the present application, the acquiring a target image includes:
collecting a preview image;
dividing the preview image into a plurality of regions;
and extracting the region with the maximum value difference degree of each pixel point and the adjacent region thereof from the plurality of regions of the preview image to be used as the target image.
As a sixth possible implementation manner of the embodiment of the present application, before focusing according to the first main body region or the second main body region, the method further includes:
and if the target image is identified to contain a plurality of first main body areas or a plurality of second main body areas, selecting the main body with the largest connected domain from the plurality of first main body areas or the plurality of second main body areas for focusing.
According to the method for identifying the main body of the image, the target image is obtained; wherein the target image comprises a central area and an edge area surrounding the central area; determining a first difference degree of values of all the pixel points in the central area and a second difference degree of values of all the pixel points in the edge area; determining a maximum of the first degree of difference and the second degree of difference; if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area; if the maximum value is the first difference degree, performing general main body segmentation on the main body in the target image to obtain a second main body area; focusing according to the first body region or the second body region. Therefore, the areas corresponding to the maximum value of the difference degree of the values of the pixel points are determined from the central area and the edge area of the image, so that the main body is segmented by adopting the corresponding segmentation method, the detection rate of the edge main body is greatly improved, the problem of focusing failure caused by low detection rate of the edge main body in the existing detection method is solved, and the user experience is improved.
The embodiment of the second aspect of the present application provides an apparatus for identifying a subject of an image, including:
the acquisition module is used for acquiring a target image; wherein the target image includes a central region and an edge region surrounding the central region;
the first determining module is used for determining a first difference degree of values of all the pixels in the central area and a second difference degree of values of all the pixels in the edge area;
a second determining module for determining a maximum of the first degree of difference and the second degree of difference;
a first identification module, configured to perform edge segmentation identification on the main body in the target image to obtain a first main body region if the maximum value is the second difference degree;
a second identification module, configured to perform general subject segmentation on the subject in the target image to obtain a second subject region if the maximum value is the first difference degree;
the focusing module is used for focusing according to the first main body area or the second main body area.
The subject recognition device of the image of the embodiment of the application acquires the target image; wherein the target image comprises a central area and an edge area surrounding the central area; determining a first difference degree of values of all the pixel points in the central area and a second difference degree of values of all the pixel points in the edge area; determining a maximum of the first degree of difference and the second degree of difference; if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area; if the maximum value is the first difference degree, performing general main body segmentation on the main body in the target image to obtain a second main body area; focusing according to the first body region or the second body region. Therefore, the areas corresponding to the maximum value of the difference degree of the values of the pixel points are determined from the central area and the edge area of the image, so that the main body is segmented by adopting the corresponding segmentation method, the detection rate of the edge main body is greatly improved, the problem of focusing failure caused by low detection rate of the edge main body in the existing detection method is solved, and the user experience is improved.
An embodiment of a third aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the subject identification method according to the embodiment of the first aspect.
A fourth aspect of the present application is directed to a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the subject identification method according to the first aspect.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flowchart of a method for identifying a subject of an image according to an embodiment of the present disclosure;
fig. 2 is an exemplary diagram of a method for identifying a subject of an image according to an embodiment of the present application;
fig. 3 is an exemplary diagram of dividing an image into a plurality of regions according to an embodiment of the present application;
FIG. 4 is a diagram of a framework example of a general subject segmentation model provided in an embodiment of the present application;
fig. 5 is a schematic flowchart of another method for identifying a subject of an image according to an embodiment of the present application;
fig. 6 is a schematic flowchart of a method for identifying a subject of another image according to an embodiment of the present application;
fig. 7 is a schematic flowchart of a further method for identifying a subject of an image according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an image subject identification apparatus according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
In the existing edge subject detection method, a target segmentation or target detection method is usually adopted to detect an edge subject, and a segmentation result or a detection result is sent to automatic focusing to assist focusing.
The existing main body detection method based on segmentation includes a segmentation method based on graph theory, a segmentation method based on clustering, a segmentation method based on semantics and the like.
In the graph theory-based segmentation method, an image is generally divided into a plurality of subgraphs, the divided subgraphs keep the maximum similarity inside, and the similarities among the subgraphs keep the minimum, such as normalizeddut, GraphCut, and the like. The segmentation method based on clustering generally initializes a rough cluster, uses an iteration mode to cluster pixel points with similar characteristics to the same superpixel, and iterates until convergence, thereby obtaining the final image segmentation result, such as k-means, SLIC and the like. A segmentation method based on semantics generally adopts a convolutional neural network to perform softmax cross entropy classification on each pixel point in an image, so as to realize segmentation of targets, such as FCN, deep Lab series and the like.
The target detection method may be classified into a conventional target detection method and a deep learning-based target detection method.
The traditional target detection method adopts an integral map characteristic + AdaBoost method to detect a main body. There are methods of extracting HOG features from a subject target candidate region and performing decision making in conjunction with an SVM classifier or DPM, and the like. Target detection methods based on deep learning, such as One-stage (networks of the YOLO and SSD series): and directly regressing the position of the target frame, namely directly converting the problem of target frame positioning into a regression problem without generating a candidate frame. Two-stage (FasterRCNN series of networks): and recommending the candidate area by using the RPN network.
However, when a subject in an image is detected, when the subject is not in the center position and is in the edge area, the conventional subject detection method has a poor detection effect, and a large number of edge subjects are missed to be detected, thereby causing a technical problem of focusing failure.
In order to solve the technical problems, the application provides a subject identification method of an image, which comprises the steps of obtaining a target image; determining a first difference degree of values of all the pixel points in the central area and a second difference degree of values of all the pixel points in the edge area; if the first difference degree is smaller than the second difference degree, adopting an edge main body segmentation model to identify a main body in the target image; the edge main body segmentation model is obtained by training a training sample with a main body positioned at the edge of an image; if the first difference degree is larger than or equal to the second difference degree, adopting a general main body segmentation model to identify a main body in the target image; the general subject segmentation model is obtained by training a training sample with a subject in the center of an image and a training sample with a subject in the edge of the image.
A subject recognition method, apparatus, electronic device, and medium of an image according to an embodiment of the present application are described below with reference to the drawings.
Fig. 1 is a schematic flowchart of a method for identifying a subject of an image according to an embodiment of the present disclosure.
The embodiment of the present application is exemplified in that the method for identifying the subject of the image is configured in the subject identification apparatus of the image, and the subject identification apparatus of the image can be applied to any electronic device, so that the electronic device can execute the function of identifying the subject of the image.
The electronic device may be a Personal Computer (PC), a cloud device, a mobile device, and the like, and the mobile device may be a hardware device having various operating systems and imaging devices, such as a mobile phone, a tablet Computer, a Personal digital assistant, a wearable device, and an in-vehicle device.
As shown in fig. 1, the method for recognizing a subject of an image includes the steps of:
step 101, acquiring a target image; wherein the target image includes a central region and an edge region surrounding the central region.
In the embodiment of the application, the target image may be a preview image displayed on a photographing interface of the electronic device, or may be a partial image area in the preview image.
In the embodiment of the application, in the process of acquiring the image by the imaging device, the preview interface can be displayed according to the shooting operation of the user so as to display the image on the preview interface of the imaging device, and the preview image acquired by the imaging device is acquired, so that the user can clearly see the imaging effect of each frame of image in the process of image processing.
It should be noted that, when the image sensors of the imaging devices are different, the acquired target images are also different. For example, when the image sensor is an RGB sensor, the acquired target image is an RGB image; when the image sensor is a depth sensor, the acquired target image is a depth image, and so on. The target image in the embodiment of the present application is not limited to the RGB image and the depth image, and may be another type of image.
In this embodiment of the application, in order to reduce the calculation amount of subject identification of subsequent images, after the target image is acquired, the target image may be reduced to a smaller size, such as 224 × 224, or may be of other sizes, which is not limited herein.
Step 102, determining a first difference degree of values of all the pixel points in the central area and a second difference degree of values of all the pixel points in the edge area.
In the embodiment of the application, after the target image including the central area and the edge area surrounding the central area is obtained, a first difference degree of values of all the pixel points in the central area and a second difference degree of values of all the pixel points in the edge area are further determined.
As a possible implementation manner, gradient values are calculated for each pixel point in a central region and each edge region of the target image, and the sum of the gradient values of each pixel point in the central region is used as a first difference degree; and taking the sum of the gradient values of all the pixel points in the edge area as a second difference degree.
And the gradient value is used for indicating the difference between the corresponding pixel point and the neighborhood pixel point.
In this embodiment, the gradient value of the pixel point may be a gray difference between each pixel point and an adjacent pixel point. Alternatively, gradient operators, such as Roberts operator (Roberts), Sobel operator, Prewitt operator, Laplacian operator (Laplacian), Log operator, and the like, may be used to calculate gradient values of each pixel point in the central region and the edge region of the target image.
For example, the Sobel operator is one of the most important operators in pixel image edge detection, and plays a significant role in information technology fields such as machine learning, digital media, computer vision, and the like. Technically, it is a discrete first order difference operator used to calculate the approximation of the first order gradient of the image intensity function. Using this operator at any point in the image will produce the corresponding gradient vector or its normal vector.
As an example, as shown in fig. 2, for the pixel point a, the Sobel operator is used to first calculate Gx and Gy, and modulo sum of Gx and Gy is performed to obtain a gradient value of the pixel point a, or a value obtained by calculating a square sum of Gx and Gy and then squaring is used as the gradient value of the pixel point a.
Step 103, determining the maximum value of the first difference degree and the second difference degree.
In the embodiment of the application, after the first difference degree of each pixel value in the central area of the target image and the second difference degree of each pixel value in the edge area are determined, the maximum value of the first difference degree and the second difference degree can be determined.
For example, the maximum value may be a first difference degree of values of each pixel point in the central area, and may also be a second difference degree of values of each pixel point in the edge area.
And 104, if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area.
The subject in the target image is not limited to a person, a landscape, an object, and the like.
In the embodiment of the application, after determining the first difference degree of each pixel value in the central area of the target image and the second difference degree of each pixel value in the edge area, and determining the maximum value of the first difference degree and the second difference degree, the main body in the target image can be segmented by adopting a corresponding main body segmentation method according to the area with the maximum difference degree.
In a possible case, the maximum value is determined as the second difference degree, which indicates that an edge subject exists in the target image, and the subject in the target image is subjected to edge segmentation to obtain a first subject region. As a possible implementation manner, an edge body segmentation model may be used to perform edge segmentation on the target body, so as to obtain the first body region. The edge subject segmentation model is obtained by training a training sample with a subject at the edge of an image. The edge main body segmentation model trained by the training sample has high detection rate of the edge main body in the actual edge main body scene.
In the embodiment of the application, when the current shooting scene is judged to be the edge main body scene according to the difference degree of the values of the pixel points in the central area and the edge area in the target image, the target image can be further cut to cut out the edge main body area and the adjacent area, and then the edge main body area is sent to the edge main body segmentation model to be segmented. Thus, by reducing the edge body segmentation region of the target image, the segmentation accuracy of the edge body can be further improved.
The following illustrates how to crop the target image to crop the edge body region and the adjacent region, and the target image in fig. 3 is taken as an example to describe in detail.
In fig. 3, the target image is uniformly divided into several regions, as shown in fig. 3, the target image is uniformly divided into 9 regions, and the numbers 1, 2, 3, 4, 5, 6, 7, 8, and 9 are marked. Calculating gradient values of all pixel points in each divided region, and taking the sum of the gradient values of all the pixel points in each region as the difference degree of the region, and recording the difference degree as sum 1. And (5) obtaining the maximum value from sum1 to sum9, namely sum _ max.
When sum _ max is sum1, i.e., when the edge body is in region 1, cutting regions 1, 2, 4, 5;
when sum _ max is sum2, i.e., when the edge body is in region 2, cutting 1, 2, 3, 4, 5, 6 regions;
when sum _ max is sum3, i.e., when the edge body is in region 3, cutting 2, 3, 5, 6 regions;
when sum _ max is sum4, i.e., when the edge body is in region 4, cutting 1, 2, 4, 5, 7, 8 regions;
when sum _ max is sum6, i.e., when the edge body is in region 6, cutting 2, 3, 5, 6, 8, 9 regions;
when sum _ max is sum7, i.e., when the edge body is in region 7, cutting 4, 5, 7, 8 regions;
when sum _ max is sum8, i.e., when the edge body is in region 8, cutting out regions 4, 5, 6, 7, 8, 9;
when sum _ max is sum9, that is, when the edge body is in the region 9, the regions 5, 6, 8, and 9 are cut.
Therefore, the target image is cut according to the sum of the gradient values of all the pixel points of all the areas of the target image, so that the main body in the cut area is subjected to edge segmentation, and the edge main body segmentation precision is improved.
And 105, if the maximum value is the first difference degree, performing general subject segmentation on the subject in the target image to obtain a second subject region. In another possible case, the maximum value is determined as the first degree of difference, and the subject in the target image is determined to be at the center of the image. In this case, the subject in the target image is segmented using a general subject segmentation method to obtain the second subject region.
As a possible implementation manner, the maximum value is determined as the first difference degree, and a general subject segmentation model may be used to segment the subject in the target image to obtain the second subject region. The general subject segmentation model is obtained by training a training sample with a subject in the center of an image and a training sample with a subject in the edge of the image. Therefore, the main body in the target image is segmented through the trained general main body segmentation model, and the detection rate of the image main body is improved.
It should be noted that the target segmentation algorithm adopted by the general subject segmentation model may be, but is not limited to, a deplab series segmentation algorithm, U-Net, FCN, and the like. The object segmentation algorithm generally includes an Encoder (Encoder) feature encoding module and a Decoder (Decoder) object template generation module.
As an example, as shown in fig. 4, fig. 4 is a diagram illustrating a framework of a general subject segmentation model according to an embodiment of the present application.
The input is a target image, and when the size of the target image is larger, the reduced target image can be used as the input; the Encoder module can be composed of basic neural network models such as MobileNet, ShuffleNet, ResNet and the like; the Decoder module may consist of deconvolution or interpolated upsampling; jump connection means that shallow features are fused to deep features to increase generalization capability of the algorithm model; the model output is a segmented target binary mask, i.e. the subject identified by the generic subject segmentation model from the target image.
It should be noted that, the steps 104 and 105 are not performed sequentially, but only the step 103 or only the step 104 is determined to be performed according to the maximum value of the first difference degree and the second difference degree.
And 106, focusing according to the first main body area or the second main body area.
And in a possible case, determining the maximum value as the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area, and then focusing according to the first main body area.
In another possible case, the maximum value is determined as the first difference degree, the main body in the target image is subjected to edge segmentation to obtain a second main body area, and then focusing is performed according to the second main body area.
According to the method for identifying the main body of the image, the target image is obtained; wherein the target image comprises a central area and an edge area surrounding the central area; determining a first difference degree of values of all the pixel points in the central area and a second difference degree of values of all the pixel points in the edge area; determining a maximum of the first degree of difference and the second degree of difference; if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area; if the maximum value is the first difference degree, performing general main body segmentation on the main body in the target image to obtain a second main body area; focusing according to the first body region or the second body region. Therefore, the areas corresponding to the maximum value of the difference degree of the values of the pixel points are determined from the central area and the edge area of the image, so that the main body is segmented by adopting the corresponding segmentation method, the detection rate of the edge main body is greatly improved, the problem of focusing failure caused by low detection rate of the edge main body in the existing detection method is solved, and the user experience is improved.
On the basis of the above embodiment, in step 101, when the target image is obtained, a preview image may be further acquired, and the preview image is divided into a plurality of regions, so as to extract a region with the largest value difference of each pixel point and an adjacent region thereof as the target image. The above process is described in detail with reference to fig. 5, and fig. 5 is a flowchart illustrating another method for identifying a subject of an image according to an embodiment of the present application.
As shown in fig. 5, the step 101 may further include the following steps:
step 201, collecting a preview image.
The preview image is an image displayed on a photographing interface of the imaging device.
In the embodiment of the application, in the process of acquiring the image by the imaging device, the preview interface can be displayed according to the shooting operation of the user, so that the image is displayed on the preview interface of the imaging device, and the preview image acquired by the imaging device is acquired.
Step 202, the preview image is divided into a plurality of areas.
In the embodiment of the application, after the preview image acquired by the imaging device is acquired, the preview image can be divided into a plurality of areas.
As an example, as shown in fig. 3, the preview image is uniformly divided into several regions in fig. 3, e.g., the target image is uniformly divided into 9 regions, and each region is numbered 1, 2, 3, 4, 5, 6, 7, 8, 9.
Step 203, extracting the region with the maximum value difference degree of each pixel point and the adjacent region thereof from the plurality of regions of the preview image as the target image.
In the embodiment of the application, after the preview image is divided into a plurality of regions, gradient values are calculated for all pixel points in the central region and the edge region of the preview image, and then the sum of the gradient values of all the pixel points in all the regions is used as the value difference degree of all the pixel points in the corresponding regions.
And further, extracting the region with the maximum value difference degree of each pixel point and the adjacent region thereof from the plurality of regions of the preview image to be used as a target image.
Continuing with the preview image example in fig. 3, after the gradient values of the pixels in the respective regions of the preview image are calculated, the sum of the gradient values of the pixels in the respective regions is used as the value difference degree of the pixels in the region. Assuming that the value difference degree of each pixel point of the region 1 is maximum, the region 1 and adjacent regions 2, 4 and 5 thereof can be used as target images.
According to the image subject identification method, the preview image is collected, the preview image is divided into a plurality of areas, and the area with the largest value difference degree of each pixel point and the adjacent area of the area are extracted from the plurality of areas of the preview image and serve as the target image. Therefore, the target image is extracted from the preview image according to the value difference degree of each pixel point of the plurality of regions of the acquired preview image, so that the calculated amount is reduced, and the detection rate of the image main body identification is improved.
On the basis of the above embodiment, before identifying the subject in the target image, it may be further determined whether the subject exists in the target image according to the first difference degree of the central region and the second difference degree of the edge region, so as to determine whether to segment the target image by using a subject segmentation method. The above process is described in detail with reference to fig. 6, and fig. 6 is a flowchart illustrating a method for identifying a subject of another image according to an embodiment of the present application.
As shown in fig. 6, the method for identifying a subject of an image may further include the following steps:
step 301, comparing the first difference degree and the second difference degree with a set threshold respectively.
The threshold is set to be the minimum value of the sum of the gradient values of all the pixel points when the main body exists in the preset area.
In the embodiment of the application, after the first difference degree of each pixel value in the central area and the second difference degree of each pixel value in the edge area are determined, the first difference degree and the second difference degree are respectively compared with the set threshold value, so that whether a main body exists in the target image or not is determined according to the comparison result.
Step 302, determining that the main body exists in the target image according to the fact that at least one of the first difference degree and the second difference degree is larger than a set threshold value.
In one possible case, the first degree of difference and the second degree of difference are respectively compared with a set threshold value, and at least one of the first degree of difference and the second degree of difference is determined to be greater than the set threshold value. In which case it is determined that a subject is present in the target image.
It can be understood that when a subject exists in the target image, an area with a large value difference degree of each pixel point exists in the target image, and therefore, when at least one of the first difference degree and the second difference degree is greater than a set threshold value, the subject exists in the target image.
In the embodiment of the application, after the main body exists in the target image, the maximum value of the first difference degree and the second difference degree is determined to determine that the main body in the target image is an edge main body or a center main body, and then the main body in the target image is segmented by adopting a corresponding main body segmentation method.
In the embodiment of the present application, a subject in a target image is segmented, and it may be recognized that the target image includes a plurality of first subject regions or a plurality of second subject regions. In this case, the subject focus having the largest connected domain may be selected from among the plurality of first subject regions or from among the plurality of second subject regions.
Step 303, determining that no main body exists in the target image according to the fact that the first difference degree and the second difference degree are both smaller than the set threshold, and focusing to the center of the target image.
In one possible case, the first degree of difference and the second degree of difference are compared with a set threshold value, respectively, and it is determined that both of the first degree of difference and the second degree of difference are smaller than the set threshold value. In this case, if it is determined that the subject does not exist in the target image, the subject is focused on the center of the target image.
It should be noted that, the steps 302 and 303 are not executed sequentially, but only the step 302 or only the step 303 is determined to be executed according to the first difference degree and the second difference degree and the magnitude of the set threshold value.
According to the method for identifying the main body of the image, the first difference degree and the second difference degree are respectively compared with the set threshold, the main body in the target image is determined to exist according to the fact that at least one of the first difference degree and the second difference degree is larger than the set threshold, and the main body does not exist in the target image according to the fact that the first difference degree and the second difference degree are smaller than the set threshold, and then the main body is focused to the center of the target image. Therefore, whether a main body exists in the target image or not is determined according to the first difference degree of each pixel value in the central area and the second difference degree of each pixel value in the marginal area and the size relation of the set threshold, so that when the main body does not exist in the target image, the main body is directly focused to the center of the target image, the image processing time is shortened, and the shooting experience of a user is improved.
On the basis of the above embodiments, referring to fig. 7, fig. 7 is a schematic flowchart of a further method for identifying a subject of an image according to an embodiment of the present application.
As shown in fig. 7, the method for recognizing a subject of an image may include the steps of:
step 401, acquiring a target image; wherein the target image includes a central region and an edge region surrounding the central region.
Step 402, determining a first difference degree of values of each pixel point in the central area and a second difference degree of values of each pixel point in the edge area.
In step 403, the maximum value of the first degree of difference and the second degree of difference is determined.
In the embodiment of the present application, the implementation process of step 401 to step 403 refers to the implementation process of step 101 to step 103 in the above embodiment, and is not described herein again.
In step 404, it is determined whether the maximum value is less than a set threshold.
In the embodiment of the application, after the maximum value of the first difference degree and the second difference degree is determined, the maximum value is compared with the set threshold value to judge whether the maximum value is smaller than the set threshold value.
Step 405, determining that the maximum value is smaller than a set threshold value, determining that no main body exists in the target image, and focusing to the center of the target image.
In the embodiment of the present application, it is determined that the maximum value of the first difference degree and the second difference degree is smaller than the set threshold, that is, both the first difference degree and the second difference degree are smaller than the set threshold. In this case, it can be determined that the subject is not present in the target image, and the subject is focused on the center of the target image.
Step 406, determining that the maximum value is greater than the set threshold, and determining whether the maximum value is the first difference degree.
In the embodiment of the application, it is determined that the maximum value of the first difference degree and the second difference degree is greater than a set threshold, and further, whether the maximum value is the first difference degree of values of each pixel point in the central area is judged.
Step 407, if the maximum value is determined to be the first difference degree, performing general subject segmentation on the subject in the target image to obtain a second subject region.
In a possible case, the maximum value is determined to be the first difference degree of the values of each pixel point in the central area, which is referred to the implementation process of step 105 in the foregoing embodiment and is not described herein again.
And step 408, if the maximum value is determined not to be the first difference degree, performing edge segmentation on the main body in the target image to obtain a first main body region.
In a possible case, it is determined that the maximum value is not the first difference degree of the values of the pixels in the central area, and the maximum value is the second difference degree of the values of the pixels in the edge area, refer to the implementation process of step 104 in the above embodiment, which is not described herein again.
Step 409, focusing according to the first body region or the second body region.
According to the image subject identification method, whether a subject exists in each region is judged according to the comparison between the difference degree of each pixel value in each region of the target image and the set threshold, after the edge subject or the central subject exists in the region is determined, the subject in the target image is segmented by adopting a corresponding subject segmentation method, and therefore the detection rate of the edge subject is improved.
In order to implement the above embodiments, the present application also provides a subject recognition apparatus for an image.
Fig. 8 is a schematic structural diagram of an image subject identification apparatus according to an embodiment of the present application.
As shown in fig. 8, the subject recognition apparatus 500 for the image may include: an acquisition module 510, a first determination module 520, a second determination module 530, a first recognition module 540, a second recognition module 550, and a focus module 560.
The acquiring module 510 is configured to acquire a target image; wherein the target image includes a central region and an edge region surrounding the central region.
The first determining module 520 is configured to determine a first difference degree of values of each pixel point in the central area and a second difference degree of values of each pixel point in the edge area.
A second determining module 530, configured to determine a maximum value of the first difference degree and the second difference degree.
The first identifying module 540 is configured to, if the maximum value is the second difference degree, perform edge segmentation identification on the main body in the target image to obtain a first main body region.
The second identifying module 550 is configured to perform general subject segmentation on the subject in the target image to obtain a second subject region if the maximum value is the first difference degree.
A focusing module 560 for focusing according to the first body region or the second body region.
As a possible case, the apparatus 500 for recognizing a subject of an image may further include:
and the comparison module is used for comparing the first difference degree and the second difference degree with a set threshold respectively.
And the determining module is used for determining that the main body exists in the target image according to the fact that at least one of the first difference degree and the second difference degree is larger than a set threshold value.
As another possible case, the apparatus 500 for identifying a subject of an image may further include:
and the focusing module is used for determining that no main body exists in the target image according to the fact that the first difference degree and the second difference degree are both smaller than the set threshold value, and focusing to the center of the target image.
As another possible scenario, the determining module 520 may further be configured to:
calculating gradient values of all pixel points in the central area and the edge area;
taking the sum of the gradient values of all the pixel points in the central area as a first difference degree;
and taking the sum of the gradient values of all the pixel points in the edge area as a second difference degree.
As another possible scenario, the gradient value is used to indicate the difference between the corresponding pixel point and the neighboring pixel point.
As another possible scenario, the obtaining module 510 may further be configured to:
collecting a preview image;
dividing the preview image into a plurality of regions;
and extracting the region with the maximum value difference degree of each pixel point and the adjacent region thereof from the plurality of regions of the preview image as a target image.
As another possible case, the apparatus 500 for identifying a subject of an image may further include:
and the selection module is used for selecting the main body with the largest connected domain from the plurality of first main body areas or the plurality of second main body areas to focus if the target image is identified to contain the plurality of first main body areas or the plurality of second main body areas.
It should be noted that the foregoing explanation of the embodiment of the subject identification method is also applicable to the subject identification apparatus of the embodiment, and is not repeated herein.
The subject recognition device of the image of the embodiment of the application acquires the target image; wherein the target image comprises a central area and an edge area surrounding the central area; determining a first difference degree of values of all the pixel points in the central area and a second difference degree of values of all the pixel points in the edge area; determining a maximum of the first degree of difference and the second degree of difference; if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area; if the maximum value is the first difference degree, performing general main body segmentation on the main body in the target image to obtain a second main body area; focusing according to the first body region or the second body region. Therefore, the areas corresponding to the maximum value of the difference degree of the values of the pixel points are determined from the central area and the edge area of the image, so that the corresponding models are adopted for main body segmentation, the detection rate of the edge main body is greatly improved, the problem of focusing failure caused by low detection rate of the edge main body in the existing detection method is solved, and the user experience is improved.
In order to implement the foregoing embodiments, the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the electronic device implements the subject identification method as described in the foregoing embodiments.
In order to implement the above embodiments, the present application also proposes a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the subject identification method as described in the above embodiments.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.

Claims (10)

1. A method for subject recognition of an image, the method comprising:
acquiring a target image; wherein the target image includes a central region and an edge region surrounding the central region;
determining a first difference degree of values of all the pixels in the central area and a second difference degree of values of all the pixels in the edge area;
determining a maximum of the first degree of difference and the second degree of difference;
if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area;
if the maximum value is the first difference degree, performing general main body segmentation on a main body in the target image to obtain a second main body area;
focusing according to the first body region or the second body region.
2. The subject identification method according to claim 1, wherein before determining the maximum value of the first degree of difference and the second degree of difference, further comprising:
comparing the first difference degree and the second difference degree with a set threshold value respectively;
and determining that the main body exists in the target image according to the fact that at least one of the first difference degree and the second difference degree is larger than a set threshold value.
3. The subject identification method according to claim 2, wherein after comparing the first difference degree and the second difference degree with a set threshold value, respectively, the method further comprises:
and according to the fact that the first difference degree and the second difference degree are both smaller than the set threshold value, determining that no main body exists in the target image, and focusing to the center of the target image.
4. The subject identification method of claim 1, wherein the determining a first degree of difference between values of each pixel point in the central region and a second degree of difference between values of each pixel point in the edge region comprises:
calculating gradient values of all pixel points in the central region and the edge region;
taking the sum of the gradient values of all the pixel points in the central area as the first difference degree;
and taking the sum of the gradient values of all the pixel points in the edge area as the second difference degree.
5. The subject recognition method of claim 4, wherein the gradient values are indicative of differences between corresponding pixel points and neighboring pixel points.
6. The subject recognition method of any one of claims 1-5, wherein the obtaining a target image comprises:
collecting a preview image;
dividing the preview image into a plurality of regions;
and extracting the region with the maximum value difference degree of each pixel point and the adjacent region thereof from the plurality of regions of the preview image to be used as the target image.
7. The method of any one of claims 1-5, wherein before focusing according to the first body region or the second body region, further comprising:
and if the target image is identified to contain a plurality of first main body areas or a plurality of second main body areas, selecting the main body with the largest connected domain from the plurality of first main body areas or the plurality of second main body areas for focusing.
8. An apparatus for recognizing a subject of an image, the apparatus comprising:
the acquisition module is used for acquiring a target image; wherein the target image includes a central region and an edge region surrounding the central region;
the first determining module is used for determining a first difference degree of values of all the pixels in the central area and a second difference degree of values of all the pixels in the edge area;
a second determining module for determining a maximum of the first degree of difference and the second degree of difference;
a first identification module, configured to perform edge segmentation identification on the main body in the target image to obtain a first main body region if the maximum value is the second difference degree;
a second identification module, configured to perform general subject segmentation on the subject in the target image to obtain a second subject region if the maximum value is the first difference degree;
the focusing module is used for focusing according to the first main body area or the second main body area.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the subject identification method as claimed in any one of claims 1 to 7 when executing the program.
10. A non-transitory computer-readable storage medium on which a computer program is stored, the program, when executed by a processor, implementing the subject identification method according to any one of claims 1 to 7.
CN202010131197.8A 2020-02-28 2020-02-28 Method and device for identifying image subject, electronic equipment and medium Pending CN111275045A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010131197.8A CN111275045A (en) 2020-02-28 2020-02-28 Method and device for identifying image subject, electronic equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010131197.8A CN111275045A (en) 2020-02-28 2020-02-28 Method and device for identifying image subject, electronic equipment and medium

Publications (1)

Publication Number Publication Date
CN111275045A true CN111275045A (en) 2020-06-12

Family

ID=71004158

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010131197.8A Pending CN111275045A (en) 2020-02-28 2020-02-28 Method and device for identifying image subject, electronic equipment and medium

Country Status (1)

Country Link
CN (1) CN111275045A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112015936A (en) * 2020-08-27 2020-12-01 北京字节跳动网络技术有限公司 Method, device, electronic equipment and medium for generating article display diagram

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112015936A (en) * 2020-08-27 2020-12-01 北京字节跳动网络技术有限公司 Method, device, electronic equipment and medium for generating article display diagram

Similar Documents

Publication Publication Date Title
US9619708B2 (en) Method of detecting a main subject in an image
EP2164027B1 (en) Object detecting device, imaging apparatus, object detecting method, and program
JP5542889B2 (en) Image processing device
US9275464B2 (en) Method for determining the extent of a foreground object in an image
JP4772839B2 (en) Image identification method and imaging apparatus
JP5775225B2 (en) Text detection using multi-layer connected components with histograms
JP5456159B2 (en) Method and apparatus for separating the top of the foreground from the background
CN109325954B (en) Image segmentation method and device and electronic equipment
US10079974B2 (en) Image processing apparatus, method, and medium for extracting feature amount of image
JP6309549B2 (en) Deformable expression detector
US20170011523A1 (en) Image processing apparatus, image processing method, and storage medium
US9412039B2 (en) Blur detection system for night scene images
CN107633237B (en) Image background segmentation method, device, equipment and medium
US20140056519A1 (en) Method, apparatus and system for segmenting an image in an image sequence
CN106603968B (en) Information processing apparatus and information processing method
JP2015082245A (en) Image processing apparatus, image processing method, and program
JP6157165B2 (en) Gaze detection device and imaging device
US9471979B2 (en) Image recognizing apparatus and method
JP2021176243A (en) Image processing apparatus, control method for the same, and imaging apparatus
EP3709266A1 (en) Human-tracking methods, apparatuses, systems, and storage media
CN111275045A (en) Method and device for identifying image subject, electronic equipment and medium
CN113269722A (en) Training method for generating countermeasure network and high-resolution image reconstruction method
CN112784854A (en) Method, device and equipment for segmenting and extracting clothing color based on mathematical statistics
CN111368698A (en) Subject recognition method, subject recognition device, electronic device, and medium
CN111669492A (en) Method for processing shot digital image by terminal and terminal

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination