CN111275045B - Image main body recognition method and device, electronic equipment and medium - Google Patents
Image main body recognition method and device, electronic equipment and medium Download PDFInfo
- Publication number
- CN111275045B CN111275045B CN202010131197.8A CN202010131197A CN111275045B CN 111275045 B CN111275045 B CN 111275045B CN 202010131197 A CN202010131197 A CN 202010131197A CN 111275045 B CN111275045 B CN 111275045B
- Authority
- CN
- China
- Prior art keywords
- main body
- difference
- degree
- target image
- difference degree
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 77
- 230000011218 segmentation Effects 0.000 claims abstract description 70
- 210000000746 body region Anatomy 0.000 claims abstract description 37
- 238000004590 computer program Methods 0.000 claims description 6
- 238000001514 detection method Methods 0.000 abstract description 33
- 238000003384 imaging method Methods 0.000 description 12
- 238000012549 training Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
The application provides a main body identification method, a main body identification device, electronic equipment and a medium of an image, wherein the method comprises the following steps: acquiring a target image; determining a first difference degree of the values of all the pixel points in the central area and a second difference degree of the values of all the pixel points in the edge area; determining a maximum of the first degree of difference and the second degree of difference; if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body region; if the maximum value is the first difference degree, carrying out general main body segmentation on a main body in the target image to obtain a second main body region; focusing according to the first body region or the second body region. Therefore, the main body segmentation is carried out by adopting a corresponding segmentation method by determining the region corresponding to the maximum value of the difference degree of the values of the pixel points, so that the detection rate of the edge main body is improved, and the problem of focusing failure caused by low detection rate of the edge main body in the existing detection method is solved.
Description
Technical Field
The present disclosure relates to the field of image recognition technologies, and in particular, to a method and apparatus for recognizing a main body of an image, an electronic device, and a medium.
Background
With the development of terminal devices, more and more users are used to shooting images or videos through imaging devices such as cameras on electronic devices. After the electronic device acquires the image, the main body detection is often required to be performed on the image, and the main body is detected, so that a clearer image of the main body can be acquired.
However, when the subject in the photographed image is not at the center position and is in the edge region, the conventional subject detection method has poor detection effect, and a large number of edge subjects are missed, which results in the problem of auto-focus failure.
Disclosure of Invention
The present application aims to solve, at least to some extent, one of the technical problems in the related art.
An embodiment of a first aspect of the present application provides a method for identifying a subject of an image, including:
acquiring a target image; wherein the target image includes a center region and an edge region surrounding the center region;
determining a first difference degree of the values of all the pixel points in the central area and a second difference degree of the values of all the pixel points in the edge area;
determining a maximum of the first degree of difference and the second degree of difference;
if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body region;
If the maximum value is the first difference degree, performing general main body segmentation on a main body in the target image to obtain a second main body region;
focusing according to the first body region or the second body region.
As a first possible implementation manner of the embodiment of the present application, before determining the maximum value of the first difference degree and the second difference degree, the method further includes:
comparing the first difference degree and the second difference degree with set thresholds respectively;
and determining that a subject exists in the target image according to at least one of the first difference degree and the second difference degree being greater than a set threshold.
As a second possible implementation manner of the embodiment of the present application, after comparing the first difference degree and the second difference degree with set thresholds, the method further includes:
and according to the fact that the first difference degree and the second difference degree are smaller than the set threshold, determining that no main body exists in the target image, and focusing to the center of the target image.
As a third possible implementation manner of the embodiment of the present application, the determining a first degree of difference between values of each pixel point in the center area and a second degree of difference between values of each pixel point in the edge area includes:
Calculating gradient values for each pixel point in the central area and the edge area;
taking the sum of gradient values of all pixel points in the central area as the first difference degree;
and taking the sum of gradient values of all pixel points in the edge area as the second difference degree.
As a fourth possible implementation manner of the embodiment of the present application, the gradient value is used to indicate a difference between the corresponding pixel point and the neighboring pixel point.
As a fifth possible implementation manner of the embodiment of the present application, the acquiring a target image includes:
collecting a preview image;
dividing the preview image into a plurality of regions;
and extracting a region with the greatest difference of the values of the pixel points and adjacent regions thereof from the plurality of regions of the preview image as the target image.
As a sixth possible implementation manner of the embodiments of the present application, before focusing according to the first body area or the second body area, the method further includes:
and if the target image is identified to contain a plurality of first main body areas or a plurality of second main body areas, selecting main body focusing with the largest connected domain from the plurality of first main body areas or the plurality of second main body areas.
According to the main body identification method of the image, the target image is obtained; wherein the target image includes a center region and an edge region surrounding the center region; determining a first difference degree of the values of all the pixel points in the central area and a second difference degree of the values of all the pixel points in the edge area; determining a maximum of the first degree of difference and the second degree of difference; if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body region; if the maximum value is the first difference degree, carrying out general main body segmentation on a main body in the target image to obtain a second main body region; focusing according to the first body region or the second body region. Therefore, the main body segmentation is carried out by adopting a corresponding segmentation method by determining the region corresponding to the maximum value of the difference degree of the values of each pixel point from the central region and the edge region of the image, so that the detection rate of the edge main body is greatly improved, the problem of focusing failure caused by low detection rate of the edge main body in the existing detection method is solved, and the use experience of a user is improved.
An embodiment of a second aspect of the present application proposes a subject identification device of an image, including:
The acquisition module is used for acquiring a target image; wherein the target image includes a center region and an edge region surrounding the center region;
the first determining module is used for determining a first difference degree of the values of the pixel points in the central area and a second difference degree of the values of the pixel points in the edge area;
a second determining module configured to determine a maximum value of the first degree of difference and the second degree of difference;
the first recognition module is used for carrying out edge segmentation recognition on the main body in the target image if the maximum value is the second difference degree to obtain a first main body region;
the second recognition module is used for carrying out general main body segmentation on the main body in the target image if the maximum value is the first difference degree to obtain a second main body region;
and the focusing module is used for focusing according to the first main body area or the second main body area.
The main body recognition device of the image of the embodiment of the application obtains the target image; wherein the target image includes a center region and an edge region surrounding the center region; determining a first difference degree of the values of all the pixel points in the central area and a second difference degree of the values of all the pixel points in the edge area; determining a maximum of the first degree of difference and the second degree of difference; if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body region; if the maximum value is the first difference degree, carrying out general main body segmentation on a main body in the target image to obtain a second main body region; focusing according to the first body region or the second body region. Therefore, the main body segmentation is carried out by adopting a corresponding segmentation method by determining the region corresponding to the maximum value of the difference degree of the values of each pixel point from the central region and the edge region of the image, so that the detection rate of the edge main body is greatly improved, the problem of focusing failure caused by low detection rate of the edge main body in the existing detection method is solved, and the use experience of a user is improved.
An embodiment of a third aspect of the present application proposes an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements a subject identification method according to an embodiment of the first aspect when the program is executed by the processor.
An embodiment of a fourth aspect of the present application proposes a non-transitory computer readable storage medium, on which a computer program is stored, which program, when executed by a processor, implements a subject identification method as described in the embodiment of the first aspect.
Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
fig. 1 is a flow chart of a method for identifying a subject of an image according to an embodiment of the present application;
fig. 2 is an exemplary diagram of a method for identifying a subject of an image according to an embodiment of the present application;
FIG. 3 is a diagram illustrating an exemplary image division of multiple regions according to an embodiment of the present application;
FIG. 4 is a diagram illustrating a framework of a generic body segmentation model according to an embodiment of the present application;
fig. 5 is a flowchart of another method for identifying a subject of an image according to an embodiment of the present application;
fig. 6 is a flowchart of a method for identifying a subject of another image according to an embodiment of the present application;
fig. 7 is a flowchart of a method for identifying a subject of another image according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of a main body recognition device for an image according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and intended for the purpose of explaining the present application and are not to be construed as limiting the present application.
In the existing edge main body detection method, an object segmentation or object detection method is generally adopted to detect an edge main body, and a segmentation result or a detection result is given to automatic focusing so as to assist focusing.
Existing segmentation-based subject detection methods include graph theory-based segmentation methods, cluster-based segmentation methods, semantic-based segmentation methods, and the like.
In the graph theory-based segmentation method, an image is generally divided into a plurality of sub-graphs, the similarity of the divided sub-graphs is kept to be the largest inside, and the similarity between the sub-graphs is kept to be the smallest, such as NormalizedCut, graphCut. The segmentation method based on the clustering generally initializes a rough cluster, gathers the pixel points with similar characteristics to the same super pixel by using an iterative mode, iterates until convergence, and thus obtains the final image segmentation result, such as k-means, SLIC and the like. According to the semantic-based segmentation method, a convolutional neural network is generally adopted to carry out softmax cross entropy classification on each pixel point in the image, so that the segmentation of targets, such as FCN, deep Lab series and the like, is realized.
The target detection method may be classified into a conventional target detection method and a target detection method based on deep learning.
The traditional target detection method adopts an integral graph feature+AdaBoost method to detect a main body. There are methods of extracting HOG features from subject target candidate regions, making decisions in conjunction with SVM classifiers or DPMs, and the like. Target detection method based on deep learning, such as One-stage (network of YOLO and SSD series): and directly returning to the position of the target frame, namely directly converting the problem of target frame positioning into a regression problem without generating a candidate frame. Two-stage (network of the fast RCNN series): and recommending the candidate areas by utilizing the RPN network.
However, when detecting a subject in an image, if the subject is not in the center position and is in an edge region, the conventional subject detection method has poor detection effect, and a large number of edge subjects are missed to be detected, so that the technical problem of focusing failure is caused.
Aiming at the technical problems, the application provides a main body identification method of an image, which comprises the steps of obtaining a target image; determining a first difference degree of the values of all the pixel points in the central area and a second difference degree of the values of all the pixel points in the edge area; if the first difference degree is smaller than the second difference degree, identifying a main body in the target image by adopting an edge main body segmentation model; the edge main body segmentation model is obtained by training a training sample with a main body at the edge of an image; if the first difference degree is greater than or equal to the second difference degree, a general subject segmentation model is adopted to identify a subject in the target image; the general subject segmentation model is obtained by training a training sample with a subject at the center of an image and a training sample with a subject at the edge of the image.
The following describes a subject recognition method, apparatus, electronic device, and medium of an image of an embodiment of the present application with reference to the accompanying drawings.
Fig. 1 is a flowchart of a method for identifying a subject of an image according to an embodiment of the present application.
The embodiment of the application is exemplified by the fact that the image subject identification method is configured in the image subject identification device, and the image subject identification device can be applied to any electronic equipment, so that the electronic equipment can execute the image subject identification function.
The electronic device may be a personal computer (Personal Computer, abbreviated as PC), a cloud device, a mobile device, etc., and the mobile device may be a hardware device with various operating systems and imaging devices, such as a mobile phone, a tablet computer, a personal digital assistant, a wearable device, a vehicle-mounted device, etc.
As shown in fig. 1, the subject recognition method of the image includes the steps of:
step 101, obtaining a target image; wherein the target image includes a center region and an edge region surrounding the center region.
In this embodiment of the present application, the target image may be a preview image displayed on a photographing interface of the electronic device, or may be a partial image area in the preview image.
In the embodiment of the application, in the process of acquiring the image by the imaging device, the preview interface can be displayed according to shooting operation of the user so as to display the image on the preview interface of the imaging device and acquire the preview image acquired by the imaging device, so that the user can clearly see the imaging effect of each frame of image in the image processing process.
It should be noted that, when the image sensors of the imaging device are different, the acquired target images are also different. For example, when the image sensor is an RGB sensor, the acquired target image is an RGB image; when the image sensor is a depth sensor, the acquired target image is a depth image, and so on. The target image in the embodiment of the present application is not limited to RGB images and depth images, but may be other types of images.
In this embodiment, in order to reduce the calculation amount of the subject recognition of the subsequent image, after the target image is acquired, the target image may be reduced to a smaller size, for example 224×224, but may be of other sizes, which is not limited herein.
Step 102, determining a first difference degree of the values of the pixel points in the central area and a second difference degree of the values of the pixel points in the edge area.
In this embodiment of the present application, after obtaining a target image including a central area and an edge area surrounding the central area, further, a first difference degree of values of each pixel point in the central area and a second difference degree of values of each pixel point in the edge area are determined.
As one possible implementation manner, calculating gradient values for each pixel point in a central area and each edge area of the target image, and taking the sum of the gradient values of each pixel point in the central area as a first difference degree; and taking the sum of gradient values of all pixel points in the edge area as a second difference degree.
The gradient value is used for indicating the difference between the corresponding pixel point and the neighborhood pixel point.
In this embodiment of the present application, the gradient value of the pixel point may be a gray scale difference value between each pixel point and an adjacent pixel point. Alternatively, gradient operators, such as a Roberts operator (Roberts), a Sobel operator, a Prewitt operator, a laplace operator (Laplacian), a Log operator, or the like, may be used to calculate gradient values of each pixel point in the center region and the edge region of the target image.
For example, the Sobel operator is one of the most important operators in pixel image edge detection, and plays a significant role in the information technology fields such as machine learning, digital media, computer vision, and the like. Technically, it is a discrete first order difference operator that is used to calculate an approximation of the first order gradient of the image brightness function. Using this operator at any point in the image will result in a gradient vector or normal vector for that point.
As an example, as shown in fig. 2, for the pixel point a, gx and Gy are first calculated by using a Sobel operator, and a gradient value of the pixel point a is obtained by modulo summation of Gx and Gy, or a value obtained by squaring sum of Gx and Gy and then opening is calculated as the gradient value of the pixel point a.
Step 103, determining the maximum value of the first difference degree and the second difference degree.
In this embodiment of the present application, after determining the first difference degree of the values of each pixel point in the center area of the target image and the second difference degree of the values of each pixel point in the edge area, the maximum value of the first difference degree and the second difference degree may be determined.
For example, the maximum value may be a first difference degree of the values of the pixels in the central region, or may be a second difference degree of the values of the pixels in the edge region.
And 104, if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body region.
Wherein the subject in the target image is not limited to a person, a landscape, an object, etc.
In this embodiment of the present application, after determining the first difference degree of the values of each pixel point in the center area of the target image and the second difference degree of the values of each pixel point in the edge area, after determining the maximum value of the first difference degree and the second difference degree, the main body in the target image may be segmented by adopting a corresponding main body segmentation method according to the area with the maximum difference degree.
In one possible case, determining that the maximum value is the second difference degree, and indicating that an edge main body exists in the target image, performing edge segmentation on the main body in the target image to obtain a first main body area. As a possible implementation manner, an edge principal segmentation model may be used to perform edge segmentation on the target principal, so as to obtain the first principal region. The edge main body segmentation model is obtained by training a training sample with a main body at the edge of an image. The edge main body segmentation model trained by the training sample has high edge main body detection rate in an actual edge main body scene.
According to the method and the device, when the current shooting scene is judged to be the edge main scene according to the difference degree of the values of the pixels of the central area and the pixels of the edge area in the target image, the target image can be further cut to cut out the edge main area and the adjacent areas, and then the edge main area is sent into the edge main body segmentation model to be segmented. Thus, the edge main body segmentation accuracy can be further improved by reducing the edge main body segmentation area of the target image.
The following illustrates how the target image is cropped to crop out the edge main region and the adjacent region, and the target image in fig. 3 is taken as an example for a detailed description.
In fig. 3, the target image is uniformly divided into a plurality of regions, and as shown in fig. 3, the target image is uniformly divided into 9 regions, and the numbers 1, 2, 3, 4, 5, 6, 7, 8, and 9 are noted. The gradient value is calculated for each pixel point in each divided region, and the sum of the gradient values of each pixel point in each region is referred to as sum1, sum9 as the degree of difference of the regions. And obtaining the maximum value of sum1 to sum9, namely sum_max.
When sum_max=sum 1, i.e. the edge body is in region 1, clipping 1, 2, 4, 5 regions;
when sum_max=sum 2, i.e. the edge body is in region 2, clipping 1, 2, 3, 4, 5, 6 regions;
when sum_max=sum 3, i.e. the edge body is in region 3, the regions 2, 3, 5, 6 are cropped;
when sum_max=sum 4, i.e. the edge body is in region 4, clipping 1, 2, 4, 5, 7, 8 regions;
when sum_max=sum 6, i.e. the edge body is in region 6, the regions 2, 3, 5, 6, 8, 9 are cropped;
when sum_max=sum 7, i.e. the edge body is in region 7, the regions 4, 5, 7, 8 are cropped;
when sum_max=sum 8, i.e. the edge body is in region 8, the regions 4, 5, 6, 7, 8, 9 are cropped;
when sum_max=sum 9, i.e. the edge body is in region 9, the regions 5, 6, 8, 9 are cropped.
Therefore, the target image is cut according to the sum of gradient values of all pixel points of all areas of the target image, so that the edge of the main body in the cut area is divided, and the edge main body dividing precision is improved.
In step 105, if the maximum value is the first difference degree, the general subject segmentation is performed on the subject in the target image, so as to obtain a second subject region. In another possible case, the maximum value is determined as the first degree of difference, and it is determined that the subject in the target image is at the center of the image. In this case, the subject in the target image is segmented by a general subject segmentation method to obtain the second subject region.
As a possible implementation manner, the maximum value is determined to be the first difference degree, and a general subject segmentation model may be used to segment the subject in the target image, so as to obtain the second subject region. The general subject segmentation model is obtained by training a training sample with a subject at the center of an image and a training sample with a subject at the edge of the image. Therefore, the trained general subject segmentation model is used for segmenting the subject in the target image, so that the detection rate of the image subject is improved.
It should be noted that the target segmentation algorithm adopted by the general subject segmentation model may be, but is not limited to, a deeplab series segmentation algorithm, U-Net, FCN, and the like. The object segmentation algorithm generally includes an Encoder (Encoder) feature encoding module and a Decoder (Decoder) object template generation module.
As an example, as shown in fig. 4, fig. 4 is a diagram illustrating a framework of a generic body segmentation model according to an embodiment of the present application.
The input is a target image, and when the size of the target image is larger, the reduced target image can be used as the input; the Encoder module can be composed of a basic neural network model such as MobileNet, shuffleNet, resNet; the Decoder module may consist of deconvolution or interpolation upsampling; jump connection means that shallow features are fused to deep features to increase the generalization capability of an algorithm model; the model output is a segmented target binary mask, i.e., a generic subject segmentation model identifies a subject from the target image.
It should be noted that, the above steps 104 and 105 are not sequentially performed, but only the step 103 is performed or only the step 104 is performed according to the maximum value of the first difference degree and the second difference degree.
Step 106, focusing according to the first main body area or the second main body area.
In one possible case, determining that the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body region, and focusing according to the first main body region.
In another possible case, determining the maximum value as the first difference degree, performing edge segmentation on the subject in the target image to obtain a second subject region, and focusing according to the second subject region.
According to the main body identification method of the image, the target image is obtained; wherein the target image includes a center region and an edge region surrounding the center region; determining a first difference degree of the values of all the pixel points in the central area and a second difference degree of the values of all the pixel points in the edge area; determining a maximum of the first degree of difference and the second degree of difference; if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body region; if the maximum value is the first difference degree, carrying out general main body segmentation on a main body in the target image to obtain a second main body region; focusing according to the first body region or the second body region. Therefore, the main body segmentation is carried out by adopting a corresponding segmentation method by determining the region corresponding to the maximum value of the difference degree of the values of each pixel point from the central region and the edge region of the image, so that the detection rate of the edge main body is greatly improved, the problem of focusing failure caused by low detection rate of the edge main body in the existing detection method is solved, and the use experience of a user is improved.
On the basis of the above embodiment, in the step 101, when the target image is acquired, the preview image may be acquired and divided into a plurality of regions, so as to extract the region with the greatest difference between the values of the pixels and the adjacent regions thereof, as the target image. The above process is described in detail with reference to fig. 5, and fig. 5 is a schematic flow chart of another method for identifying a subject of an image according to an embodiment of the present application.
As shown in fig. 5, the step 101 may further include the following steps:
in step 201, a preview image is acquired.
Wherein the preview image is an image displayed on a photographing interface of the imaging device.
In the embodiment of the application, in the process of acquiring the image by the imaging device, the preview interface can be displayed according to shooting operation of a user so as to display the image on the preview interface of the imaging device and acquire the preview image acquired by the imaging device.
Step 202, dividing the preview image into a plurality of regions.
In this embodiment of the present application, after obtaining the preview image collected by the imaging device, the preview image may be divided into a plurality of areas.
As an example, as shown in fig. 3, the preview image is uniformly divided into several areas in fig. 3, for example, the target image is uniformly divided into 9 areas, and numbers 1, 2, 3, 4, 5, 6, 7, 8, 9 are noted for each area.
In step 203, a region with the greatest difference in value between each pixel point and its neighboring region is extracted from the plurality of regions in the preview image as the target image.
In the embodiment of the application, after the preview image is divided into a plurality of areas, gradient values are calculated for each pixel point in the central area and the edge area of the preview image, and then the sum of the gradient values of each pixel point in each area is used as the value difference degree of each pixel point in the corresponding area.
Further, from among the plurality of regions of the preview image, a region having the greatest degree of difference in the values of the respective pixels and its adjacent regions are extracted as the target image.
Continuing with the preview image example in fig. 3, after calculating the gradient value for each pixel point in each region of the preview image, the sum of the gradient values for each pixel point in each region is used as the degree of difference in the values of each pixel point in the region. Assuming that the pixel values of the region 1 have the greatest degree of difference, the region 1 and its adjacent regions 2, 4, 5 may be regarded as the target image.
According to the image main body recognition method, the preview image is collected and divided into a plurality of areas, and the area with the largest difference degree of the values of the pixel points and the adjacent areas are extracted from the plurality of areas of the preview image to serve as the target image. Therefore, according to the difference degree of the values of the pixels of the plurality of areas of the acquired preview image, the target image is extracted from the preview image, so that the calculated amount is reduced, and the detection rate of the image main body recognition is improved.
On the basis of the above embodiment, before identifying the subject in the target image, if possible, it may also be determined whether the subject is present in the target image according to the first difference degree of the center region and the second difference degree of the edge region, so as to determine whether the subject is segmented by using the subject segmentation method. The above process will be described in detail with reference to fig. 6, and fig. 6 is a schematic flow chart of a subject identification method for another image according to an embodiment of the present application.
As shown in fig. 6, the method for recognizing a subject of an image may further include the steps of:
in step 301, the first difference degree and the second difference degree are compared with a set threshold value, respectively.
The threshold value is set as the minimum value of the sum of gradient values of all pixel points when the main body exists in the preset area.
In the embodiment of the application, after determining the first difference degree of the values of the pixel points in the center area and the second difference degree of the values of the pixel points in the edge area, comparing the first difference degree and the second difference degree with a set threshold value respectively to determine whether a main body exists in a target image according to a comparison result.
Step 302, determining that a subject exists in the target image according to at least one of the first difference degree and the second difference degree being greater than a set threshold.
In one possible scenario, the first degree of difference and the second degree of difference are compared with a set threshold value, respectively, and it is determined that at least one of the first degree of difference and the second degree of difference is greater than the set threshold value. In this case it is determined that a subject is present in the target image.
It can be understood that when the subject exists in the target image, a region with a larger difference degree of the values of the pixels exists in the target image, so that when at least one of the first difference degree and the second difference degree is larger than the set threshold value, the subject can be determined to exist in the target image.
In the embodiment of the application, after determining that the subject exists in the target image, determining the maximum value of the first difference degree and the second difference degree to determine that the subject in the target image is an edge subject or a center subject, and further dividing the subject in the target image by adopting a corresponding subject dividing method.
In this embodiment of the present application, the subject in the target image may be segmented, and a plurality of first subject areas or a plurality of second subject areas may be identified in the target image. In this case, the subject having the largest connected domain may be selected to be in focus from among the plurality of first subject regions or from among the plurality of second subject regions.
Step 303, determining that the subject does not exist in the target image according to the first difference degree and the second difference degree being smaller than the set threshold, and focusing to the center of the target image.
In one possible scenario, the first degree of difference and the second degree of difference are compared with a set threshold value, respectively, and it is determined that both the first degree of difference and the second degree of difference are smaller than the set threshold value. In this case, if it is determined that the subject is not present in the target image, focusing is performed on the center of the target image.
It should be noted that, the above steps 302 and 303 are not sequentially performed, but only the step 302 is determined to be performed or only the step 303 is determined to be performed according to the first difference degree and the second difference degree and the set threshold value, respectively.
According to the method for identifying the main body of the image, the first difference degree and the second difference degree are respectively compared with the set threshold, the main body exists in the target image according to the fact that at least one of the first difference degree and the second difference degree is larger than the set threshold, the main body does not exist in the target image according to the fact that the first difference degree and the second difference degree are smaller than the set threshold, and focusing is conducted on the center of the target image. Therefore, according to the first difference degree of the values of the pixel points in the center area and the second difference degree of the values of the pixel points in the edge area, the size relation between the first difference degree and the set threshold value is used for determining whether a main body exists in the target image or not, so that when the main body does not exist in the target image, the main body is directly focused to the center of the target image, the processing time of the image is shortened, and the shooting experience of a user is improved.
On the basis of the above embodiment, referring to fig. 7, fig. 7 is a schematic flow chart of a method for identifying a subject of another image according to an embodiment of the present application.
As shown in fig. 7, the subject recognition method of the image may include the steps of:
step 401, obtaining a target image; wherein the target image includes a center region and an edge region surrounding the center region.
Step 402, determining a first difference degree of the values of the pixels in the center area and a second difference degree of the values of the pixels in the edge area.
In step 403, the maximum of the first degree of difference and the second degree of difference is determined.
In this embodiment, the implementation process of steps 401 to 403 refers to the implementation process of steps 101 to 103 in the above embodiment, and is not described herein.
Step 404, determining whether the maximum value is less than a set threshold.
In this embodiment of the present application, after determining the maximum value of the first difference degree and the second difference degree, the maximum value is compared with a set threshold value to determine whether the maximum value is smaller than the set threshold value.
And step 405, determining that the maximum value is smaller than the set threshold value, and focusing to the center of the target image if no subject exists in the target image.
In this embodiment of the present application, it is determined that the maximum value of the first difference degree and the second difference degree is smaller than the set threshold, that is, the first difference degree and the second difference degree are both smaller than the set threshold. In this case, it can be determined that there is no subject in the target image, and then focusing is performed to the center of the target image.
Step 406, determining whether the maximum value is greater than the set threshold, and determining whether the maximum value is the first difference degree.
In this embodiment of the present application, it is determined that a maximum value of the first difference degree and the second difference degree is greater than a set threshold, and further, it is determined whether the maximum value is the first difference degree of the values of the pixel points in the central area.
Step 407, determining that the maximum value is the first difference degree, and performing general subject segmentation on the subject in the target image to obtain a second subject region.
In a possible case, the maximum value is determined as the first difference degree of the values of the pixel points in the central area, and the implementation process of step 105 in the above embodiment is referred to herein and will not be described in detail.
In step 408, if the maximum value is not the first difference degree, edge segmentation is performed on the subject in the target image to obtain a first subject region.
In a possible case, the maximum value is determined to be the first difference degree of the values of the pixels in the central area, and the maximum value is determined to be the second difference degree of the values of the pixels in the edge area, so the implementation process of step 104 in the above embodiment is not repeated here.
Step 409, focusing according to the first body region or the second body region.
According to the main body identification method of the image, whether the main body exists in each region is judged according to comparison between the difference degree of the pixel point values in each region of the target image and the set threshold value, after the edge main body or the center main body exists in each region is determined, the main body in the target image is segmented by adopting a corresponding main body segmentation method, and therefore the detection rate of the edge main body is improved.
In order to implement the above embodiment, the present application also proposes a subject recognition apparatus of an image.
Fig. 8 is a schematic structural diagram of a main body recognition device for an image according to an embodiment of the present application.
As shown in fig. 8, the subject recognition apparatus 500 of the image may include: the device comprises an acquisition module 510, a first determination module 520, a second determination module 530, a first identification module 540, a second identification module 550 and a focusing module 560.
The acquiring module 510 is configured to acquire a target image; wherein the target image includes a center region and an edge region surrounding the center region.
The first determining module 520 is configured to determine a first degree of difference between the values of the pixels in the center area and a second degree of difference between the values of the pixels in the edge area.
A second determining module 530 is configured to determine a maximum value of the first degree of difference and the second degree of difference.
The first recognition module 540 is configured to perform edge segmentation recognition on the subject in the target image if the maximum value is the second difference degree, so as to obtain a first subject region.
And the second identifying module 550 is configured to perform general subject segmentation on the subject in the target image to obtain a second subject region if the maximum value is the first difference degree.
The focusing module 560 is configured to focus according to the first body area or the second body area.
As a possible case, the subject recognition apparatus 500 of the image may further include:
and the comparison module is used for comparing the first difference degree and the second difference degree with set thresholds respectively.
And the determining module is used for determining that a main body exists in the target image according to the fact that at least one of the first difference degree and the second difference degree is larger than a set threshold value.
As another possible case, the subject recognition apparatus 500 of the image may further include:
and the focusing module is used for determining that the main body does not exist in the target image according to the fact that the first difference degree and the second difference degree are smaller than the set threshold value, and focusing to the center of the target image.
As another possible scenario, the determining module 520 may also be configured to:
calculating gradient values for each pixel point in the central area and the edge area;
taking the sum of gradient values of all pixel points in the central area as a first difference degree;
and taking the sum of gradient values of all pixel points in the edge area as a second difference degree.
As another possible case, gradient values are used to indicate the difference between the corresponding pixel point and the neighboring pixel point.
As another possible scenario, the acquisition module 510 may also be configured to:
collecting a preview image;
dividing the preview image into a plurality of regions;
and extracting the region with the greatest difference of the values of the pixel points and the adjacent regions from the regions of the preview image as a target image.
As another possible case, the subject recognition apparatus 500 of the image may further include:
and the selection module is used for selecting the main body with the largest connected domain from the plurality of first main body areas or the plurality of second main body areas to focus if the plurality of first main body areas or the plurality of second main body areas are contained in the target image.
It should be noted that the foregoing explanation of the embodiment of the subject identifying method is also applicable to the subject identifying device of this embodiment, and will not be repeated here.
The main body recognition device of the image of the embodiment of the application obtains the target image; wherein the target image includes a center region and an edge region surrounding the center region; determining a first difference degree of the values of all the pixel points in the central area and a second difference degree of the values of all the pixel points in the edge area; determining a maximum of the first degree of difference and the second degree of difference; if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body region; if the maximum value is the first difference degree, carrying out general main body segmentation on a main body in the target image to obtain a second main body region; focusing according to the first body region or the second body region. Therefore, the region corresponding to the maximum value of the difference degree of the values of each pixel point is determined from the central region and the edge region of the image, so that the main body is divided by adopting the corresponding model, the detection rate of the edge main body is greatly improved, the problem of focusing failure caused by low detection rate of the edge main body in the existing detection method is solved, and the use experience of a user is improved.
In order to implement the above embodiment, the present application further proposes an electronic device including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the subject identification method as described in the above embodiment when executing the program.
In order to achieve the above-described embodiments, the present application also proposes a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the subject identification method as described in the above-described embodiments.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "plurality" is at least two, such as two, three, etc., unless explicitly defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and additional implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As with the other embodiments, if implemented in hardware, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product.
The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like. Although embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives, and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.
Claims (9)
1. A method of identifying a subject of an image, the method comprising:
acquiring a target image; wherein the target image includes a center region and an edge region surrounding the center region;
determining a first difference degree of the values of all the pixel points in the central area and a second difference degree of the values of all the pixel points in the edge area;
determining a maximum of the first degree of difference and the second degree of difference;
if the maximum value is the second difference degree, performing edge segmentation on the main body in the target image to obtain a first main body region;
if the maximum value is the first difference degree, performing general main body segmentation on a main body in the target image to obtain a second main body region;
Focusing according to the first body region or the second body region;
the determining the first difference degree of the values of the pixel points in the central area and the second difference degree of the values of the pixel points in the edge area comprises the following steps:
calculating gradient values for each pixel point in the central area and the edge area;
taking the sum of gradient values of all pixel points in the central area as the first difference degree;
and taking the sum of gradient values of all pixel points in the edge area as the second difference degree.
2. The method of claim 1, wherein prior to determining the maximum of the first degree of variance and the second degree of variance, further comprising:
comparing the first difference degree and the second difference degree with set thresholds respectively;
and determining that a subject exists in the target image according to at least one of the first difference degree and the second difference degree being greater than a set threshold.
3. The method of claim 2, wherein comparing the first degree of difference and the second degree of difference with a set threshold value, respectively, further comprises:
And according to the fact that the first difference degree and the second difference degree are smaller than the set threshold, determining that no main body exists in the target image, and focusing to the center of the target image.
4. The subject identification method of claim 1 wherein the gradient values are indicative of differences between respective pixels and neighboring pixels.
5. The subject identification method as in any one of claims 1-4 wherein the acquiring the target image comprises:
collecting a preview image;
dividing the preview image into a plurality of regions;
and extracting a region with the greatest difference of the values of the pixel points and adjacent regions thereof from the plurality of regions of the preview image as the target image.
6. The subject identification method as in any one of claims 1-4, further comprising, prior to focusing according to the first subject region or the second subject region:
and if the target image is identified to contain a plurality of first main body areas or a plurality of second main body areas, selecting main body focusing with the largest connected domain from the plurality of first main body areas or the plurality of second main body areas.
7. A subject identification device for an image, the device comprising:
the acquisition module is used for acquiring a target image; wherein the target image includes a center region and an edge region surrounding the center region;
the first determining module is used for determining a first difference degree of the values of the pixel points in the central area and a second difference degree of the values of the pixel points in the edge area;
a second determining module configured to determine a maximum value of the first degree of difference and the second degree of difference;
the first recognition module is used for carrying out edge segmentation recognition on the main body in the target image if the maximum value is the second difference degree to obtain a first main body region;
the second recognition module is used for carrying out general main body segmentation on the main body in the target image if the maximum value is the first difference degree to obtain a second main body region;
the focusing module is used for focusing according to the first main body area or the second main body area;
the first determining module is further used for calculating gradient values for each pixel point in the central area and the edge area; taking the sum of gradient values of all pixel points in the central area as the first difference degree; and taking the sum of gradient values of all pixel points in the edge area as the second difference degree.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the subject identification method of any one of claims 1-6 when the program is executed by the processor.
9. A non-transitory computer-readable storage medium, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the subject identification method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010131197.8A CN111275045B (en) | 2020-02-28 | 2020-02-28 | Image main body recognition method and device, electronic equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010131197.8A CN111275045B (en) | 2020-02-28 | 2020-02-28 | Image main body recognition method and device, electronic equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111275045A CN111275045A (en) | 2020-06-12 |
| CN111275045B true CN111275045B (en) | 2024-02-06 |
Family
ID=71004158
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010131197.8A Active CN111275045B (en) | 2020-02-28 | 2020-02-28 | Image main body recognition method and device, electronic equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111275045B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112015936B (en) * | 2020-08-27 | 2021-10-26 | 北京字节跳动网络技术有限公司 | Method, device, electronic equipment and medium for generating article display diagram |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007189704A (en) * | 2007-01-29 | 2007-07-26 | Denso Corp | Recognition device for image |
| WO2019042216A1 (en) * | 2017-08-29 | 2019-03-07 | Oppo广东移动通信有限公司 | Image blurring processing method and device, and photographing terminal |
| CN110517278A (en) * | 2019-08-07 | 2019-11-29 | 北京旷视科技有限公司 | Image segmentation and the training method of image segmentation network, device and computer equipment |
| CN110689007A (en) * | 2019-09-16 | 2020-01-14 | Oppo广东移动通信有限公司 | Subject recognition method and device, electronic equipment and computer-readable storage medium |
-
2020
- 2020-02-28 CN CN202010131197.8A patent/CN111275045B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007189704A (en) * | 2007-01-29 | 2007-07-26 | Denso Corp | Recognition device for image |
| WO2019042216A1 (en) * | 2017-08-29 | 2019-03-07 | Oppo广东移动通信有限公司 | Image blurring processing method and device, and photographing terminal |
| CN110517278A (en) * | 2019-08-07 | 2019-11-29 | 北京旷视科技有限公司 | Image segmentation and the training method of image segmentation network, device and computer equipment |
| CN110689007A (en) * | 2019-09-16 | 2020-01-14 | Oppo广东移动通信有限公司 | Subject recognition method and device, electronic equipment and computer-readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111275045A (en) | 2020-06-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9619708B2 (en) | Method of detecting a main subject in an image | |
| EP2164027B1 (en) | Object detecting device, imaging apparatus, object detecting method, and program | |
| US9118846B2 (en) | Apparatus for generating an image with defocused background and method thereof | |
| KR101468351B1 (en) | Object tracking device, object tracking method, and control program | |
| JP5775225B2 (en) | Text detection using multi-layer connected components with histograms | |
| CN105404884B (en) | Image analysis method | |
| US10079974B2 (en) | Image processing apparatus, method, and medium for extracting feature amount of image | |
| CN110059666B (en) | Attention detection method and device | |
| JP4533836B2 (en) | Fluctuating region detection apparatus and method | |
| US9412039B2 (en) | Blur detection system for night scene images | |
| JP6924064B2 (en) | Image processing device and its control method, and image pickup device | |
| CN111047615A (en) | Image-based line detection method and device and electronic equipment | |
| Sakpal et al. | Adaptive background subtraction in images | |
| CN109714530B (en) | Aerial camera image focusing method | |
| CN111028170A (en) | Image processing method, image processing apparatus, electronic device, and readable storage medium | |
| CN111368698B (en) | Main body identification method, main body identification device, electronic equipment and medium | |
| CN111275045B (en) | Image main body recognition method and device, electronic equipment and medium | |
| CN112784854B (en) | Clothing color segmentation extraction method, device and equipment based on mathematical statistics | |
| Janardhana Rao et al. | Hybridized cuckoo search with multi-verse optimization-based patch matching and deep learning concept for enhancing video inpainting | |
| CN113158773B (en) | Training method and training device for living body detection model | |
| CN111669492A (en) | Method for processing shot digital image by terminal and terminal | |
| CN112883940A (en) | Silent in-vivo detection method, silent in-vivo detection device, computer equipment and storage medium | |
| CN111277753A (en) | Focusing method and device, terminal equipment and storage medium | |
| CN113728357B (en) | Image processing method, image processing apparatus, and image processing system | |
| CN113409331B (en) | Image processing method, image processing device, terminal and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |