CN111275045A - Method and device for identifying image subject, electronic equipment and medium - Google Patents
Method and device for identifying image subject, electronic equipment and medium Download PDFInfo
- Publication number
- CN111275045A CN111275045A CN202010131197.8A CN202010131197A CN111275045A CN 111275045 A CN111275045 A CN 111275045A CN 202010131197 A CN202010131197 A CN 202010131197A CN 111275045 A CN111275045 A CN 111275045A
- Authority
- CN
- China
- Prior art keywords
- main body
- target image
- difference degree
- difference
- degree
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 78
- 230000011218 segmentation Effects 0.000 claims abstract description 71
- 210000000746 body region Anatomy 0.000 claims abstract description 28
- 238000004590 computer program Methods 0.000 claims description 9
- 238000001514 detection method Methods 0.000 abstract description 33
- 238000003384 imaging method Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 11
- 238000012549 training Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 101100116390 Schizosaccharomyces pombe (strain 972 / ATCC 24843) ded1 gene Proteins 0.000 description 1
- 101100311460 Schizosaccharomyces pombe (strain 972 / ATCC 24843) sum2 gene Proteins 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
The application provides a method, a device, an electronic device and a medium for identifying a subject of an image, wherein the method comprises the following steps: by acquiring a target image; determining a first difference degree of values of all the pixel points in the central area and a second difference degree of values of all the pixel points in the edge area; determining a maximum of the first degree of difference and the second degree of difference; if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area; if the maximum value is the first difference degree, performing general main body segmentation on the main body in the target image to obtain a second main body area; focusing according to the first body region or the second body region. Therefore, the main body segmentation is carried out by adopting a corresponding segmentation method through determining the region corresponding to the maximum value of the difference degree of the values of all the pixel points, the detection rate of the edge main body is improved, and the problem of focusing failure caused by low detection rate of the edge main body in the existing detection method is solved.
Description
Technical Field
The present application relates to the field of image recognition technologies, and in particular, to a method and an apparatus for recognizing a subject of an image, an electronic device, and a medium.
Background
With the development of terminal devices, more and more users are used to shoot images or videos through imaging devices such as cameras on electronic devices. After the electronic device acquires the image, the image is often subjected to subject detection to detect the subject, so that a clearer image of the subject can be acquired.
However, when the subject in the captured image is not in the center position but in the edge area, the conventional subject detection method has a poor detection effect, and a large number of edge subjects are missed, thereby causing a problem of failure in auto-focusing.
Disclosure of Invention
The present application is directed to solving, at least to some extent, one of the technical problems in the related art.
An embodiment of a first aspect of the present application provides a method for identifying a subject of an image, including:
acquiring a target image; wherein the target image includes a central region and an edge region surrounding the central region;
determining a first difference degree of values of all the pixels in the central area and a second difference degree of values of all the pixels in the edge area;
determining a maximum of the first degree of difference and the second degree of difference;
if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area;
if the maximum value is the first difference degree, performing general main body segmentation on a main body in the target image to obtain a second main body area;
focusing according to the first body region or the second body region.
As a first possible implementation manner of the embodiment of the present application, before determining a maximum value of the first difference degree and the second difference degree, the method further includes:
comparing the first difference degree and the second difference degree with a set threshold value respectively;
and determining that the main body exists in the target image according to the fact that at least one of the first difference degree and the second difference degree is larger than a set threshold value.
As a second possible implementation manner of the embodiment of the present application, after comparing the first difference degree and the second difference degree with a set threshold, the method further includes:
and according to the fact that the first difference degree and the second difference degree are both smaller than the set threshold value, determining that no main body exists in the target image, and focusing to the center of the target image.
As a third possible implementation manner of the embodiment of the present application, the determining a first difference degree of values of each pixel point in the central area and a second difference degree of values of each pixel point in the edge area includes:
calculating gradient values of all pixel points in the central region and the edge region;
taking the sum of the gradient values of all the pixel points in the central area as the first difference degree;
and taking the sum of the gradient values of all the pixel points in the edge area as the second difference degree.
As a fourth possible implementation manner of the embodiment of the present application, the gradient value is used to indicate a difference between a corresponding pixel point and a neighboring pixel point.
As a fifth possible implementation manner of the embodiment of the present application, the acquiring a target image includes:
collecting a preview image;
dividing the preview image into a plurality of regions;
and extracting the region with the maximum value difference degree of each pixel point and the adjacent region thereof from the plurality of regions of the preview image to be used as the target image.
As a sixth possible implementation manner of the embodiment of the present application, before focusing according to the first main body region or the second main body region, the method further includes:
and if the target image is identified to contain a plurality of first main body areas or a plurality of second main body areas, selecting the main body with the largest connected domain from the plurality of first main body areas or the plurality of second main body areas for focusing.
According to the method for identifying the main body of the image, the target image is obtained; wherein the target image comprises a central area and an edge area surrounding the central area; determining a first difference degree of values of all the pixel points in the central area and a second difference degree of values of all the pixel points in the edge area; determining a maximum of the first degree of difference and the second degree of difference; if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area; if the maximum value is the first difference degree, performing general main body segmentation on the main body in the target image to obtain a second main body area; focusing according to the first body region or the second body region. Therefore, the areas corresponding to the maximum value of the difference degree of the values of the pixel points are determined from the central area and the edge area of the image, so that the main body is segmented by adopting the corresponding segmentation method, the detection rate of the edge main body is greatly improved, the problem of focusing failure caused by low detection rate of the edge main body in the existing detection method is solved, and the user experience is improved.
The embodiment of the second aspect of the present application provides an apparatus for identifying a subject of an image, including:
the acquisition module is used for acquiring a target image; wherein the target image includes a central region and an edge region surrounding the central region;
the first determining module is used for determining a first difference degree of values of all the pixels in the central area and a second difference degree of values of all the pixels in the edge area;
a second determining module for determining a maximum of the first degree of difference and the second degree of difference;
a first identification module, configured to perform edge segmentation identification on the main body in the target image to obtain a first main body region if the maximum value is the second difference degree;
a second identification module, configured to perform general subject segmentation on the subject in the target image to obtain a second subject region if the maximum value is the first difference degree;
the focusing module is used for focusing according to the first main body area or the second main body area.
The subject recognition device of the image of the embodiment of the application acquires the target image; wherein the target image comprises a central area and an edge area surrounding the central area; determining a first difference degree of values of all the pixel points in the central area and a second difference degree of values of all the pixel points in the edge area; determining a maximum of the first degree of difference and the second degree of difference; if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area; if the maximum value is the first difference degree, performing general main body segmentation on the main body in the target image to obtain a second main body area; focusing according to the first body region or the second body region. Therefore, the areas corresponding to the maximum value of the difference degree of the values of the pixel points are determined from the central area and the edge area of the image, so that the main body is segmented by adopting the corresponding segmentation method, the detection rate of the edge main body is greatly improved, the problem of focusing failure caused by low detection rate of the edge main body in the existing detection method is solved, and the user experience is improved.
An embodiment of a third aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the subject identification method according to the embodiment of the first aspect.
A fourth aspect of the present application is directed to a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the subject identification method according to the first aspect.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flowchart of a method for identifying a subject of an image according to an embodiment of the present disclosure;
fig. 2 is an exemplary diagram of a method for identifying a subject of an image according to an embodiment of the present application;
fig. 3 is an exemplary diagram of dividing an image into a plurality of regions according to an embodiment of the present application;
FIG. 4 is a diagram of a framework example of a general subject segmentation model provided in an embodiment of the present application;
fig. 5 is a schematic flowchart of another method for identifying a subject of an image according to an embodiment of the present application;
fig. 6 is a schematic flowchart of a method for identifying a subject of another image according to an embodiment of the present application;
fig. 7 is a schematic flowchart of a further method for identifying a subject of an image according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an image subject identification apparatus according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
In the existing edge subject detection method, a target segmentation or target detection method is usually adopted to detect an edge subject, and a segmentation result or a detection result is sent to automatic focusing to assist focusing.
The existing main body detection method based on segmentation includes a segmentation method based on graph theory, a segmentation method based on clustering, a segmentation method based on semantics and the like.
In the graph theory-based segmentation method, an image is generally divided into a plurality of subgraphs, the divided subgraphs keep the maximum similarity inside, and the similarities among the subgraphs keep the minimum, such as normalizeddut, GraphCut, and the like. The segmentation method based on clustering generally initializes a rough cluster, uses an iteration mode to cluster pixel points with similar characteristics to the same superpixel, and iterates until convergence, thereby obtaining the final image segmentation result, such as k-means, SLIC and the like. A segmentation method based on semantics generally adopts a convolutional neural network to perform softmax cross entropy classification on each pixel point in an image, so as to realize segmentation of targets, such as FCN, deep Lab series and the like.
The target detection method may be classified into a conventional target detection method and a deep learning-based target detection method.
The traditional target detection method adopts an integral map characteristic + AdaBoost method to detect a main body. There are methods of extracting HOG features from a subject target candidate region and performing decision making in conjunction with an SVM classifier or DPM, and the like. Target detection methods based on deep learning, such as One-stage (networks of the YOLO and SSD series): and directly regressing the position of the target frame, namely directly converting the problem of target frame positioning into a regression problem without generating a candidate frame. Two-stage (FasterRCNN series of networks): and recommending the candidate area by using the RPN network.
However, when a subject in an image is detected, when the subject is not in the center position and is in the edge area, the conventional subject detection method has a poor detection effect, and a large number of edge subjects are missed to be detected, thereby causing a technical problem of focusing failure.
In order to solve the technical problems, the application provides a subject identification method of an image, which comprises the steps of obtaining a target image; determining a first difference degree of values of all the pixel points in the central area and a second difference degree of values of all the pixel points in the edge area; if the first difference degree is smaller than the second difference degree, adopting an edge main body segmentation model to identify a main body in the target image; the edge main body segmentation model is obtained by training a training sample with a main body positioned at the edge of an image; if the first difference degree is larger than or equal to the second difference degree, adopting a general main body segmentation model to identify a main body in the target image; the general subject segmentation model is obtained by training a training sample with a subject in the center of an image and a training sample with a subject in the edge of the image.
A subject recognition method, apparatus, electronic device, and medium of an image according to an embodiment of the present application are described below with reference to the drawings.
Fig. 1 is a schematic flowchart of a method for identifying a subject of an image according to an embodiment of the present disclosure.
The embodiment of the present application is exemplified in that the method for identifying the subject of the image is configured in the subject identification apparatus of the image, and the subject identification apparatus of the image can be applied to any electronic device, so that the electronic device can execute the function of identifying the subject of the image.
The electronic device may be a Personal Computer (PC), a cloud device, a mobile device, and the like, and the mobile device may be a hardware device having various operating systems and imaging devices, such as a mobile phone, a tablet Computer, a Personal digital assistant, a wearable device, and an in-vehicle device.
As shown in fig. 1, the method for recognizing a subject of an image includes the steps of:
In the embodiment of the application, the target image may be a preview image displayed on a photographing interface of the electronic device, or may be a partial image area in the preview image.
In the embodiment of the application, in the process of acquiring the image by the imaging device, the preview interface can be displayed according to the shooting operation of the user so as to display the image on the preview interface of the imaging device, and the preview image acquired by the imaging device is acquired, so that the user can clearly see the imaging effect of each frame of image in the process of image processing.
It should be noted that, when the image sensors of the imaging devices are different, the acquired target images are also different. For example, when the image sensor is an RGB sensor, the acquired target image is an RGB image; when the image sensor is a depth sensor, the acquired target image is a depth image, and so on. The target image in the embodiment of the present application is not limited to the RGB image and the depth image, and may be another type of image.
In this embodiment of the application, in order to reduce the calculation amount of subject identification of subsequent images, after the target image is acquired, the target image may be reduced to a smaller size, such as 224 × 224, or may be of other sizes, which is not limited herein.
In the embodiment of the application, after the target image including the central area and the edge area surrounding the central area is obtained, a first difference degree of values of all the pixel points in the central area and a second difference degree of values of all the pixel points in the edge area are further determined.
As a possible implementation manner, gradient values are calculated for each pixel point in a central region and each edge region of the target image, and the sum of the gradient values of each pixel point in the central region is used as a first difference degree; and taking the sum of the gradient values of all the pixel points in the edge area as a second difference degree.
And the gradient value is used for indicating the difference between the corresponding pixel point and the neighborhood pixel point.
In this embodiment, the gradient value of the pixel point may be a gray difference between each pixel point and an adjacent pixel point. Alternatively, gradient operators, such as Roberts operator (Roberts), Sobel operator, Prewitt operator, Laplacian operator (Laplacian), Log operator, and the like, may be used to calculate gradient values of each pixel point in the central region and the edge region of the target image.
For example, the Sobel operator is one of the most important operators in pixel image edge detection, and plays a significant role in information technology fields such as machine learning, digital media, computer vision, and the like. Technically, it is a discrete first order difference operator used to calculate the approximation of the first order gradient of the image intensity function. Using this operator at any point in the image will produce the corresponding gradient vector or its normal vector.
As an example, as shown in fig. 2, for the pixel point a, the Sobel operator is used to first calculate Gx and Gy, and modulo sum of Gx and Gy is performed to obtain a gradient value of the pixel point a, or a value obtained by calculating a square sum of Gx and Gy and then squaring is used as the gradient value of the pixel point a.
In the embodiment of the application, after the first difference degree of each pixel value in the central area of the target image and the second difference degree of each pixel value in the edge area are determined, the maximum value of the first difference degree and the second difference degree can be determined.
For example, the maximum value may be a first difference degree of values of each pixel point in the central area, and may also be a second difference degree of values of each pixel point in the edge area.
And 104, if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area.
The subject in the target image is not limited to a person, a landscape, an object, and the like.
In the embodiment of the application, after determining the first difference degree of each pixel value in the central area of the target image and the second difference degree of each pixel value in the edge area, and determining the maximum value of the first difference degree and the second difference degree, the main body in the target image can be segmented by adopting a corresponding main body segmentation method according to the area with the maximum difference degree.
In a possible case, the maximum value is determined as the second difference degree, which indicates that an edge subject exists in the target image, and the subject in the target image is subjected to edge segmentation to obtain a first subject region. As a possible implementation manner, an edge body segmentation model may be used to perform edge segmentation on the target body, so as to obtain the first body region. The edge subject segmentation model is obtained by training a training sample with a subject at the edge of an image. The edge main body segmentation model trained by the training sample has high detection rate of the edge main body in the actual edge main body scene.
In the embodiment of the application, when the current shooting scene is judged to be the edge main body scene according to the difference degree of the values of the pixel points in the central area and the edge area in the target image, the target image can be further cut to cut out the edge main body area and the adjacent area, and then the edge main body area is sent to the edge main body segmentation model to be segmented. Thus, by reducing the edge body segmentation region of the target image, the segmentation accuracy of the edge body can be further improved.
The following illustrates how to crop the target image to crop the edge body region and the adjacent region, and the target image in fig. 3 is taken as an example to describe in detail.
In fig. 3, the target image is uniformly divided into several regions, as shown in fig. 3, the target image is uniformly divided into 9 regions, and the numbers 1, 2, 3, 4, 5, 6, 7, 8, and 9 are marked. Calculating gradient values of all pixel points in each divided region, and taking the sum of the gradient values of all the pixel points in each region as the difference degree of the region, and recording the difference degree as sum 1. And (5) obtaining the maximum value from sum1 to sum9, namely sum _ max.
When sum _ max is sum1, i.e., when the edge body is in region 1, cutting regions 1, 2, 4, 5;
when sum _ max is sum2, i.e., when the edge body is in region 2, cutting 1, 2, 3, 4, 5, 6 regions;
when sum _ max is sum3, i.e., when the edge body is in region 3, cutting 2, 3, 5, 6 regions;
when sum _ max is sum4, i.e., when the edge body is in region 4, cutting 1, 2, 4, 5, 7, 8 regions;
when sum _ max is sum6, i.e., when the edge body is in region 6, cutting 2, 3, 5, 6, 8, 9 regions;
when sum _ max is sum7, i.e., when the edge body is in region 7, cutting 4, 5, 7, 8 regions;
when sum _ max is sum8, i.e., when the edge body is in region 8, cutting out regions 4, 5, 6, 7, 8, 9;
when sum _ max is sum9, that is, when the edge body is in the region 9, the regions 5, 6, 8, and 9 are cut.
Therefore, the target image is cut according to the sum of the gradient values of all the pixel points of all the areas of the target image, so that the main body in the cut area is subjected to edge segmentation, and the edge main body segmentation precision is improved.
And 105, if the maximum value is the first difference degree, performing general subject segmentation on the subject in the target image to obtain a second subject region. In another possible case, the maximum value is determined as the first degree of difference, and the subject in the target image is determined to be at the center of the image. In this case, the subject in the target image is segmented using a general subject segmentation method to obtain the second subject region.
As a possible implementation manner, the maximum value is determined as the first difference degree, and a general subject segmentation model may be used to segment the subject in the target image to obtain the second subject region. The general subject segmentation model is obtained by training a training sample with a subject in the center of an image and a training sample with a subject in the edge of the image. Therefore, the main body in the target image is segmented through the trained general main body segmentation model, and the detection rate of the image main body is improved.
It should be noted that the target segmentation algorithm adopted by the general subject segmentation model may be, but is not limited to, a deplab series segmentation algorithm, U-Net, FCN, and the like. The object segmentation algorithm generally includes an Encoder (Encoder) feature encoding module and a Decoder (Decoder) object template generation module.
As an example, as shown in fig. 4, fig. 4 is a diagram illustrating a framework of a general subject segmentation model according to an embodiment of the present application.
The input is a target image, and when the size of the target image is larger, the reduced target image can be used as the input; the Encoder module can be composed of basic neural network models such as MobileNet, ShuffleNet, ResNet and the like; the Decoder module may consist of deconvolution or interpolated upsampling; jump connection means that shallow features are fused to deep features to increase generalization capability of the algorithm model; the model output is a segmented target binary mask, i.e. the subject identified by the generic subject segmentation model from the target image.
It should be noted that, the steps 104 and 105 are not performed sequentially, but only the step 103 or only the step 104 is determined to be performed according to the maximum value of the first difference degree and the second difference degree.
And 106, focusing according to the first main body area or the second main body area.
And in a possible case, determining the maximum value as the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area, and then focusing according to the first main body area.
In another possible case, the maximum value is determined as the first difference degree, the main body in the target image is subjected to edge segmentation to obtain a second main body area, and then focusing is performed according to the second main body area.
According to the method for identifying the main body of the image, the target image is obtained; wherein the target image comprises a central area and an edge area surrounding the central area; determining a first difference degree of values of all the pixel points in the central area and a second difference degree of values of all the pixel points in the edge area; determining a maximum of the first degree of difference and the second degree of difference; if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area; if the maximum value is the first difference degree, performing general main body segmentation on the main body in the target image to obtain a second main body area; focusing according to the first body region or the second body region. Therefore, the areas corresponding to the maximum value of the difference degree of the values of the pixel points are determined from the central area and the edge area of the image, so that the main body is segmented by adopting the corresponding segmentation method, the detection rate of the edge main body is greatly improved, the problem of focusing failure caused by low detection rate of the edge main body in the existing detection method is solved, and the user experience is improved.
On the basis of the above embodiment, in step 101, when the target image is obtained, a preview image may be further acquired, and the preview image is divided into a plurality of regions, so as to extract a region with the largest value difference of each pixel point and an adjacent region thereof as the target image. The above process is described in detail with reference to fig. 5, and fig. 5 is a flowchart illustrating another method for identifying a subject of an image according to an embodiment of the present application.
As shown in fig. 5, the step 101 may further include the following steps:
The preview image is an image displayed on a photographing interface of the imaging device.
In the embodiment of the application, in the process of acquiring the image by the imaging device, the preview interface can be displayed according to the shooting operation of the user, so that the image is displayed on the preview interface of the imaging device, and the preview image acquired by the imaging device is acquired.
In the embodiment of the application, after the preview image acquired by the imaging device is acquired, the preview image can be divided into a plurality of areas.
As an example, as shown in fig. 3, the preview image is uniformly divided into several regions in fig. 3, e.g., the target image is uniformly divided into 9 regions, and each region is numbered 1, 2, 3, 4, 5, 6, 7, 8, 9.
In the embodiment of the application, after the preview image is divided into a plurality of regions, gradient values are calculated for all pixel points in the central region and the edge region of the preview image, and then the sum of the gradient values of all the pixel points in all the regions is used as the value difference degree of all the pixel points in the corresponding regions.
And further, extracting the region with the maximum value difference degree of each pixel point and the adjacent region thereof from the plurality of regions of the preview image to be used as a target image.
Continuing with the preview image example in fig. 3, after the gradient values of the pixels in the respective regions of the preview image are calculated, the sum of the gradient values of the pixels in the respective regions is used as the value difference degree of the pixels in the region. Assuming that the value difference degree of each pixel point of the region 1 is maximum, the region 1 and adjacent regions 2, 4 and 5 thereof can be used as target images.
According to the image subject identification method, the preview image is collected, the preview image is divided into a plurality of areas, and the area with the largest value difference degree of each pixel point and the adjacent area of the area are extracted from the plurality of areas of the preview image and serve as the target image. Therefore, the target image is extracted from the preview image according to the value difference degree of each pixel point of the plurality of regions of the acquired preview image, so that the calculated amount is reduced, and the detection rate of the image main body identification is improved.
On the basis of the above embodiment, before identifying the subject in the target image, it may be further determined whether the subject exists in the target image according to the first difference degree of the central region and the second difference degree of the edge region, so as to determine whether to segment the target image by using a subject segmentation method. The above process is described in detail with reference to fig. 6, and fig. 6 is a flowchart illustrating a method for identifying a subject of another image according to an embodiment of the present application.
As shown in fig. 6, the method for identifying a subject of an image may further include the following steps:
The threshold is set to be the minimum value of the sum of the gradient values of all the pixel points when the main body exists in the preset area.
In the embodiment of the application, after the first difference degree of each pixel value in the central area and the second difference degree of each pixel value in the edge area are determined, the first difference degree and the second difference degree are respectively compared with the set threshold value, so that whether a main body exists in the target image or not is determined according to the comparison result.
Step 302, determining that the main body exists in the target image according to the fact that at least one of the first difference degree and the second difference degree is larger than a set threshold value.
In one possible case, the first degree of difference and the second degree of difference are respectively compared with a set threshold value, and at least one of the first degree of difference and the second degree of difference is determined to be greater than the set threshold value. In which case it is determined that a subject is present in the target image.
It can be understood that when a subject exists in the target image, an area with a large value difference degree of each pixel point exists in the target image, and therefore, when at least one of the first difference degree and the second difference degree is greater than a set threshold value, the subject exists in the target image.
In the embodiment of the application, after the main body exists in the target image, the maximum value of the first difference degree and the second difference degree is determined to determine that the main body in the target image is an edge main body or a center main body, and then the main body in the target image is segmented by adopting a corresponding main body segmentation method.
In the embodiment of the present application, a subject in a target image is segmented, and it may be recognized that the target image includes a plurality of first subject regions or a plurality of second subject regions. In this case, the subject focus having the largest connected domain may be selected from among the plurality of first subject regions or from among the plurality of second subject regions.
In one possible case, the first degree of difference and the second degree of difference are compared with a set threshold value, respectively, and it is determined that both of the first degree of difference and the second degree of difference are smaller than the set threshold value. In this case, if it is determined that the subject does not exist in the target image, the subject is focused on the center of the target image.
It should be noted that, the steps 302 and 303 are not executed sequentially, but only the step 302 or only the step 303 is determined to be executed according to the first difference degree and the second difference degree and the magnitude of the set threshold value.
According to the method for identifying the main body of the image, the first difference degree and the second difference degree are respectively compared with the set threshold, the main body in the target image is determined to exist according to the fact that at least one of the first difference degree and the second difference degree is larger than the set threshold, and the main body does not exist in the target image according to the fact that the first difference degree and the second difference degree are smaller than the set threshold, and then the main body is focused to the center of the target image. Therefore, whether a main body exists in the target image or not is determined according to the first difference degree of each pixel value in the central area and the second difference degree of each pixel value in the marginal area and the size relation of the set threshold, so that when the main body does not exist in the target image, the main body is directly focused to the center of the target image, the image processing time is shortened, and the shooting experience of a user is improved.
On the basis of the above embodiments, referring to fig. 7, fig. 7 is a schematic flowchart of a further method for identifying a subject of an image according to an embodiment of the present application.
As shown in fig. 7, the method for recognizing a subject of an image may include the steps of:
In step 403, the maximum value of the first degree of difference and the second degree of difference is determined.
In the embodiment of the present application, the implementation process of step 401 to step 403 refers to the implementation process of step 101 to step 103 in the above embodiment, and is not described herein again.
In step 404, it is determined whether the maximum value is less than a set threshold.
In the embodiment of the application, after the maximum value of the first difference degree and the second difference degree is determined, the maximum value is compared with the set threshold value to judge whether the maximum value is smaller than the set threshold value.
In the embodiment of the present application, it is determined that the maximum value of the first difference degree and the second difference degree is smaller than the set threshold, that is, both the first difference degree and the second difference degree are smaller than the set threshold. In this case, it can be determined that the subject is not present in the target image, and the subject is focused on the center of the target image.
In the embodiment of the application, it is determined that the maximum value of the first difference degree and the second difference degree is greater than a set threshold, and further, whether the maximum value is the first difference degree of values of each pixel point in the central area is judged.
In a possible case, the maximum value is determined to be the first difference degree of the values of each pixel point in the central area, which is referred to the implementation process of step 105 in the foregoing embodiment and is not described herein again.
And step 408, if the maximum value is determined not to be the first difference degree, performing edge segmentation on the main body in the target image to obtain a first main body region.
In a possible case, it is determined that the maximum value is not the first difference degree of the values of the pixels in the central area, and the maximum value is the second difference degree of the values of the pixels in the edge area, refer to the implementation process of step 104 in the above embodiment, which is not described herein again.
According to the image subject identification method, whether a subject exists in each region is judged according to the comparison between the difference degree of each pixel value in each region of the target image and the set threshold, after the edge subject or the central subject exists in the region is determined, the subject in the target image is segmented by adopting a corresponding subject segmentation method, and therefore the detection rate of the edge subject is improved.
In order to implement the above embodiments, the present application also provides a subject recognition apparatus for an image.
Fig. 8 is a schematic structural diagram of an image subject identification apparatus according to an embodiment of the present application.
As shown in fig. 8, the subject recognition apparatus 500 for the image may include: an acquisition module 510, a first determination module 520, a second determination module 530, a first recognition module 540, a second recognition module 550, and a focus module 560.
The acquiring module 510 is configured to acquire a target image; wherein the target image includes a central region and an edge region surrounding the central region.
The first determining module 520 is configured to determine a first difference degree of values of each pixel point in the central area and a second difference degree of values of each pixel point in the edge area.
A second determining module 530, configured to determine a maximum value of the first difference degree and the second difference degree.
The first identifying module 540 is configured to, if the maximum value is the second difference degree, perform edge segmentation identification on the main body in the target image to obtain a first main body region.
The second identifying module 550 is configured to perform general subject segmentation on the subject in the target image to obtain a second subject region if the maximum value is the first difference degree.
A focusing module 560 for focusing according to the first body region or the second body region.
As a possible case, the apparatus 500 for recognizing a subject of an image may further include:
and the comparison module is used for comparing the first difference degree and the second difference degree with a set threshold respectively.
And the determining module is used for determining that the main body exists in the target image according to the fact that at least one of the first difference degree and the second difference degree is larger than a set threshold value.
As another possible case, the apparatus 500 for identifying a subject of an image may further include:
and the focusing module is used for determining that no main body exists in the target image according to the fact that the first difference degree and the second difference degree are both smaller than the set threshold value, and focusing to the center of the target image.
As another possible scenario, the determining module 520 may further be configured to:
calculating gradient values of all pixel points in the central area and the edge area;
taking the sum of the gradient values of all the pixel points in the central area as a first difference degree;
and taking the sum of the gradient values of all the pixel points in the edge area as a second difference degree.
As another possible scenario, the gradient value is used to indicate the difference between the corresponding pixel point and the neighboring pixel point.
As another possible scenario, the obtaining module 510 may further be configured to:
collecting a preview image;
dividing the preview image into a plurality of regions;
and extracting the region with the maximum value difference degree of each pixel point and the adjacent region thereof from the plurality of regions of the preview image as a target image.
As another possible case, the apparatus 500 for identifying a subject of an image may further include:
and the selection module is used for selecting the main body with the largest connected domain from the plurality of first main body areas or the plurality of second main body areas to focus if the target image is identified to contain the plurality of first main body areas or the plurality of second main body areas.
It should be noted that the foregoing explanation of the embodiment of the subject identification method is also applicable to the subject identification apparatus of the embodiment, and is not repeated herein.
The subject recognition device of the image of the embodiment of the application acquires the target image; wherein the target image comprises a central area and an edge area surrounding the central area; determining a first difference degree of values of all the pixel points in the central area and a second difference degree of values of all the pixel points in the edge area; determining a maximum of the first degree of difference and the second degree of difference; if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area; if the maximum value is the first difference degree, performing general main body segmentation on the main body in the target image to obtain a second main body area; focusing according to the first body region or the second body region. Therefore, the areas corresponding to the maximum value of the difference degree of the values of the pixel points are determined from the central area and the edge area of the image, so that the corresponding models are adopted for main body segmentation, the detection rate of the edge main body is greatly improved, the problem of focusing failure caused by low detection rate of the edge main body in the existing detection method is solved, and the user experience is improved.
In order to implement the foregoing embodiments, the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the electronic device implements the subject identification method as described in the foregoing embodiments.
In order to implement the above embodiments, the present application also proposes a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the subject identification method as described in the above embodiments.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
Claims (10)
1. A method for subject recognition of an image, the method comprising:
acquiring a target image; wherein the target image includes a central region and an edge region surrounding the central region;
determining a first difference degree of values of all the pixels in the central area and a second difference degree of values of all the pixels in the edge area;
determining a maximum of the first degree of difference and the second degree of difference;
if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body area;
if the maximum value is the first difference degree, performing general main body segmentation on a main body in the target image to obtain a second main body area;
focusing according to the first body region or the second body region.
2. The subject identification method according to claim 1, wherein before determining the maximum value of the first degree of difference and the second degree of difference, further comprising:
comparing the first difference degree and the second difference degree with a set threshold value respectively;
and determining that the main body exists in the target image according to the fact that at least one of the first difference degree and the second difference degree is larger than a set threshold value.
3. The subject identification method according to claim 2, wherein after comparing the first difference degree and the second difference degree with a set threshold value, respectively, the method further comprises:
and according to the fact that the first difference degree and the second difference degree are both smaller than the set threshold value, determining that no main body exists in the target image, and focusing to the center of the target image.
4. The subject identification method of claim 1, wherein the determining a first degree of difference between values of each pixel point in the central region and a second degree of difference between values of each pixel point in the edge region comprises:
calculating gradient values of all pixel points in the central region and the edge region;
taking the sum of the gradient values of all the pixel points in the central area as the first difference degree;
and taking the sum of the gradient values of all the pixel points in the edge area as the second difference degree.
5. The subject recognition method of claim 4, wherein the gradient values are indicative of differences between corresponding pixel points and neighboring pixel points.
6. The subject recognition method of any one of claims 1-5, wherein the obtaining a target image comprises:
collecting a preview image;
dividing the preview image into a plurality of regions;
and extracting the region with the maximum value difference degree of each pixel point and the adjacent region thereof from the plurality of regions of the preview image to be used as the target image.
7. The method of any one of claims 1-5, wherein before focusing according to the first body region or the second body region, further comprising:
and if the target image is identified to contain a plurality of first main body areas or a plurality of second main body areas, selecting the main body with the largest connected domain from the plurality of first main body areas or the plurality of second main body areas for focusing.
8. An apparatus for recognizing a subject of an image, the apparatus comprising:
the acquisition module is used for acquiring a target image; wherein the target image includes a central region and an edge region surrounding the central region;
the first determining module is used for determining a first difference degree of values of all the pixels in the central area and a second difference degree of values of all the pixels in the edge area;
a second determining module for determining a maximum of the first degree of difference and the second degree of difference;
a first identification module, configured to perform edge segmentation identification on the main body in the target image to obtain a first main body region if the maximum value is the second difference degree;
a second identification module, configured to perform general subject segmentation on the subject in the target image to obtain a second subject region if the maximum value is the first difference degree;
the focusing module is used for focusing according to the first main body area or the second main body area.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the subject identification method as claimed in any one of claims 1 to 7 when executing the program.
10. A non-transitory computer-readable storage medium on which a computer program is stored, the program, when executed by a processor, implementing the subject identification method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010131197.8A CN111275045B (en) | 2020-02-28 | 2020-02-28 | Image main body recognition method and device, electronic equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010131197.8A CN111275045B (en) | 2020-02-28 | 2020-02-28 | Image main body recognition method and device, electronic equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111275045A true CN111275045A (en) | 2020-06-12 |
| CN111275045B CN111275045B (en) | 2024-02-06 |
Family
ID=71004158
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010131197.8A Active CN111275045B (en) | 2020-02-28 | 2020-02-28 | Image main body recognition method and device, electronic equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111275045B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112015936A (en) * | 2020-08-27 | 2020-12-01 | 北京字节跳动网络技术有限公司 | Method, device, electronic equipment and medium for generating article display diagram |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007189704A (en) * | 2007-01-29 | 2007-07-26 | Denso Corp | Recognition device for image |
| WO2019042216A1 (en) * | 2017-08-29 | 2019-03-07 | Oppo广东移动通信有限公司 | Image blurring processing method and device, and photographing terminal |
| CN110517278A (en) * | 2019-08-07 | 2019-11-29 | 北京旷视科技有限公司 | Image segmentation and the training method of image segmentation network, device and computer equipment |
| CN110689007A (en) * | 2019-09-16 | 2020-01-14 | Oppo广东移动通信有限公司 | Subject recognition method and device, electronic equipment and computer-readable storage medium |
-
2020
- 2020-02-28 CN CN202010131197.8A patent/CN111275045B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007189704A (en) * | 2007-01-29 | 2007-07-26 | Denso Corp | Recognition device for image |
| WO2019042216A1 (en) * | 2017-08-29 | 2019-03-07 | Oppo广东移动通信有限公司 | Image blurring processing method and device, and photographing terminal |
| CN110517278A (en) * | 2019-08-07 | 2019-11-29 | 北京旷视科技有限公司 | Image segmentation and the training method of image segmentation network, device and computer equipment |
| CN110689007A (en) * | 2019-09-16 | 2020-01-14 | Oppo广东移动通信有限公司 | Subject recognition method and device, electronic equipment and computer-readable storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112015936A (en) * | 2020-08-27 | 2020-12-01 | 北京字节跳动网络技术有限公司 | Method, device, electronic equipment and medium for generating article display diagram |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111275045B (en) | 2024-02-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109325954B (en) | Image segmentation method and device and electronic equipment | |
| US9619708B2 (en) | Method of detecting a main subject in an image | |
| EP2164027B1 (en) | Object detecting device, imaging apparatus, object detecting method, and program | |
| JP5542889B2 (en) | Image processing device | |
| JP4772839B2 (en) | Image identification method and imaging apparatus | |
| US9275464B2 (en) | Method for determining the extent of a foreground object in an image | |
| JP5456159B2 (en) | Method and apparatus for separating the top of the foreground from the background | |
| EP3709266A1 (en) | Human-tracking methods, apparatuses, systems, and storage media | |
| JP6309549B2 (en) | Deformable expression detector | |
| US10079974B2 (en) | Image processing apparatus, method, and medium for extracting feature amount of image | |
| CN110059666B (en) | Attention detection method and device | |
| CN107633237B (en) | Image background segmentation method, device, equipment and medium | |
| JP2014531097A (en) | Text detection using multi-layer connected components with histograms | |
| CN106603968B (en) | Information processing apparatus and information processing method | |
| CN111028170B (en) | Image processing method, image processing apparatus, electronic device, and readable storage medium | |
| JP2015082245A (en) | Image processing apparatus, image processing method, and program | |
| JP6157165B2 (en) | Gaze detection device and imaging device | |
| US20140056519A1 (en) | Method, apparatus and system for segmenting an image in an image sequence | |
| JP6924064B2 (en) | Image processing device and its control method, and image pickup device | |
| US20130169824A1 (en) | Blur detection system for night scene images | |
| JP2018124689A (en) | Moving body detection device, moving body detection system and moving body detection method | |
| CN112883940A (en) | Silent in-vivo detection method, silent in-vivo detection device, computer equipment and storage medium | |
| CN111368698A (en) | Subject recognition method, subject recognition device, electronic device, and medium | |
| CN113158773B (en) | Training method and training device for living body detection model | |
| CN112784854B (en) | Clothing color segmentation extraction method, device and equipment based on mathematical statistics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |