CN111277753A - Focusing method, device, terminal device and storage medium - Google Patents

Abstract

The application discloses a focusing method, a focusing device, terminal equipment and a storage medium. The method can comprise the following steps: acquiring a current preview image; acquiring a main body confidence coefficient of a current preview image; detecting whether the current preview image is a main image or a non-main image according to the main confidence coefficient; and carrying out focusing processing by adopting a corresponding focusing mode according to the detection result. The method can detect whether the current preview image is a main image or a non-main image by acquiring the confidence coefficient of the main body of the preview image, and adopts a corresponding focusing mode according to the detection result to realize focusing processing, thereby realizing focusing processing in the scene with or without the main image, reducing target error segmentation or target error detection in the scene without the main image, avoiding the inaccuracy of automatic focusing caused by the error of main body segmentation or main body detection, improving the precision of target main body segmentation or target main body detection, and improving the accuracy of automatic focusing.

Description

Focusing method, device, terminal device and storage medium

technical field

The present application relates to the field of imaging technologies, and in particular, to a focusing method, an apparatus, a terminal device, and a computer-readable storage medium.

Background technique

With the development of imaging technology, the phenomenon of using terminal equipment to capture images is becoming more and more common. In the process of image shooting, the subject is usually detected by the method of target segmentation or target detection, and the segmentation result or detection result is sent to the automatic focusing module, and the focusing process is performed.

However, the existing subject segmentation or subject detection algorithms are performed in a scene with a subject target. However, in a scene without a subject target, mis-segmentation or mis-detection are prone to occur, resulting in the result of mis-segmentation or mis-detection being transmitted to The auto focus module will lead to inaccurate focus, resulting in poor user experience.

SUMMARY OF THE INVENTION

The purpose of this application is to solve one of the technical problems in the related art at least to a certain extent.

In a first aspect, an embodiment of the present application provides a focusing method, including the following steps: acquiring a current preview image; acquiring a subject confidence level of the current preview image; and detecting whether the current preview image is a subject based on the subject confidence level There is a subject image or no subject image; according to the detection result, the corresponding focusing method is used for focusing processing.

In a second aspect, an embodiment of the present application provides a focusing device, including: a preview image acquisition module for acquiring a current preview image; a subject confidence acquisition module for acquiring subject confidence in the current preview image; a detection module , for detecting whether the current preview image is an image with or without a subject according to the confidence of the subject; the focusing module is used for focusing processing in a corresponding focusing mode according to the detection result.

In a third aspect, an embodiment of the present application provides a terminal device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, when the processor executes the program , to implement the focusing method described in the embodiments of the first aspect of the present application.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the focusing method described in the embodiment of the first aspect of the present application is implemented.

The technical solutions of the embodiments of the present application can acquire the current preview image, then acquire the subject confidence of the current preview image, and detect whether the current preview image is an image with or without subject according to the confidence of the subject, and then, according to the detection result, use the corresponding The focusing method is used for focusing processing. The method can detect whether the current preview image is an image with or without a subject by obtaining the confidence of the subject of the preview image, and adopt a corresponding focusing method according to the detection result to realize focusing processing, which realizes the detection in the scene with or without subject image. Focus processing reduces target mis-segmentation or target mis-detection in scenes without subject images, avoids subject segmentation or subject detection errors leading to inaccurate auto-focusing, improves the accuracy of target subject segmentation or target subject detection, and improves Autofocus accuracy.

Additional aspects and advantages of the present application will be set forth, in part, in the following description, and in part will be apparent from the following description, or learned by practice of the present application.

Description of drawings

The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

FIG. 1 is a flowchart of a focusing method according to an embodiment of the present application.

FIG. 2 is a flowchart of a focusing method according to a specific embodiment of the present application.

FIG. 3 is a schematic diagram of a target segmentation structure according to a specific embodiment of the present application.

FIG. 4 is a flowchart of a focusing method according to another specific embodiment of the present application.

FIG. 5 is a schematic structural diagram of a focusing device according to an embodiment of the present application.

FIG. 6 is a schematic structural diagram of a terminal device according to an embodiment of the present application.

Detailed ways

The following describes in detail the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to explain the present invention and should not be construed as limiting the present invention.

In the prior art, in the current subject detection method, the subject is usually detected by the method of target segmentation or target detection, and the segmentation result or the detection result is automatically focused, and the focus processing is performed.

Among them, the target segmentation method usually divides the image into several sub-images. The divided sub-images maintain the maximum similarity internally, while the similarity between the sub-images remains the smallest, such as Normalized Cut (normalized cut), Graph Cut (Figure cut) and so on. The clustering-based segmentation method usually initializes a rough cluster first, and uses an iterative method to gather pixels with similar features to the same superpixel, and iterate until convergence, so as to obtain the final image segmentation result, such as k-means(k -meansclustering algorithm, k-means clustering algorithm), SLIC (Simple Linear Iterative Clustering, simple linear iterative clustering) and so on. Semantic-based segmentation methods usually use convolutional neural networks to perform softmax cross-entropy classification on each pixel in the image to achieve target segmentation, such as FCN (Fully Convolutional Networks, Fully Convolutional Neural Networks), Deep Lab series, etc.

The target detection method includes the use of integral graph feature + AdaBoost method to detect the subject. In some cases, the HOG feature is extracted from the subject target candidate area, and the judgment is made in combination with the SVM (Support Vector Machine, Support Vector Machine) classifier or DPM. The target detection method based on deep learning, One-stage (YOLO (You Only LookOnce, only look at the target detection once) and SSD series of networks): directly return to the position of the target frame, that is, directly locate the target frame without generating a candidate frame The problem is transformed into a regression problem. Two-stage (Faster RCNN series of networks): Use the RPN network to recommend candidate regions.

However, the existing subject segmentation or subject detection algorithms are performed in a scene with a subject target. However, in a scene without a subject target, mis-segmentation or mis-detection are prone to occur, resulting in the result of mis-segmentation or mis-detection being transmitted to The auto focus module will lead to inaccurate focus, resulting in poor user experience.

The focusing method, apparatus, terminal device, and storage medium of the embodiments of the present application are described below with reference to the accompanying drawings.

FIG. 1 is a flowchart of a focusing method according to an embodiment of the present application. It should be noted that, the focusing method in the embodiment of the present application can be applied to the focusing device in the embodiment of the present application, and the device can be configured on a terminal device. Wherein, in the embodiments of the present application, the terminal device may be a mobile terminal, such as a smart phone, a tablet computer, a wearable device, and the like.

As shown in Figure 1, the focusing method may include:

S110, acquire the current preview image.

In the embodiment of the present application, when the camera collects the current image, the current preview image can be obtained. Wherein, the preview image is generated by the terminal device capturing the picture of the current scene in real time through the camera, which may include a person image or a background image, wherein the preview image includes but is not limited to RGB (colors of three channels of red, green, and blue) image, depth map, in which, in order to reduce the calculation amount of subsequent processing, the preview image can be reduced to a smaller size, for example, 224*224.

Wherein, in an embodiment of the present application, the use of a camera to collect the current image may be an image collected after the camera on the terminal device is turned on.

The camera may be a device installed in the terminal device, and the camera includes but is not limited to one or more of various wide-angle cameras, telephoto cameras, color cameras, or black-and-white cameras.

In an embodiment of the present application, the preview image may be displayed on the display screen of the terminal device in real time.

In an embodiment of the present application, the current preview image may also be a frame of image in the video acquired by the terminal device when recording the video.

S120: Obtain the subject confidence of the current preview image.

In this embodiment of the present application, after the current preview image is acquired, the subject confidence of the current preview image can be acquired through a support vector machine SVM regression model or a classification network model based on a convolutional neural network CNN. For example, the subject confidence of the current preview image can be obtained in the following two ways:

As an example of implementation, the features of the current preview image can be extracted, and then the features of the current preview image can be input into a trained support vector machine regression model for regression prediction to obtain a regression value, and the obtained regression value is determined as the current preview. The subject confidence of the image.

As another implementation example, the current preview image is input into a trained neural network classifier, wherein the neural network classifier is a classification network model based on a convolutional neural network, and the neural network classifier includes a The input layer, and the output layer for outputting the original output value of the network, then obtain the original output value of the network output by the neural network classifier, convert the target output value in the original output value of the network to the probability, and determine the probability obtained after the conversion. is the subject confidence of the current preview image, wherein the target output value refers to the output value corresponding to the category label having the subject category label. For a specific implementation manner, reference may be made to subsequent embodiments.

S130, according to the subject confidence, detect whether the current preview image is an image with a subject or an image without a subject.

In the embodiment of the present application, the subject confidence of the current preview image is obtained, and it can be determined whether the subject confidence is less than the target threshold. If the subject confidence is less than the target threshold, it is determined that the current preview image is an image without subject. If it is greater than or equal to the target threshold, it is determined that the current preview image is a subject image.

That is to say, after obtaining the subject confidence of the current preview image, the subject confidence of the current preview image can be compared with the target threshold. If the subject confidence is less than the target threshold, it is determined that the current preview image is an image without subject. If the degree is greater than or equal to the target threshold, it is determined that the current preview image is a subject image. As an example, the target threshold may be 0.5.

S140, according to the detection result, use a corresponding focusing mode to perform focusing processing.

Optionally, in the embodiment of the present application, when it is detected that the current preview image is an image with a subject, the size and area position of the subject target are detected from the current preview image, and the size and area of the subject target are detected according to the detected size and area. Focus processing is performed according to the position of the current preview image; when it is detected that the current preview image is an image without a subject, focus processing is performed according to the target area in the current preview image.

The target area may be an intermediate area, that is, an intermediate area in the current preview image.

As an example, when it is detected that the current preview image is an image without a subject, the middle area in the current preview image can be automatically focused. When it is detected that the current preview image is an image with a subject, the target segmentation or target detection can In the method, the size and area position of the main target are detected from the current preview image, and focus processing is performed according to the detected size and area position of the main target. For the specific implementation process, reference may be made to subsequent embodiments.

According to the focusing method of the embodiment of the present application, the current preview image can be acquired, and then the subject confidence of the current preview image can be acquired, and according to the subject confidence, it is detected whether the current preview image is an image with a subject or an image without a subject, and then according to the detection result, using The corresponding focusing method is used for focusing processing. The method can detect whether the current preview image is an image with or without a subject by obtaining the confidence of the subject of the preview image, and adopt a corresponding focusing method according to the detection result to realize focusing processing, which realizes the detection in the scene with or without subject image. Focus processing reduces target mis-segmentation or target mis-detection in scenes without subject images, avoids inaccurate auto-focusing caused by subject segmentation or subject detection errors, improves the accuracy of target subject segmentation or target subject detection, and improves the accuracy of autofocus.

FIG. 2 is a flowchart of a focusing method according to a specific embodiment of the present application. As shown in FIG. 2 , the focusing method may include:

S210, acquire the current preview image.

For example, when the camera takes a picture of a user, the user faces the sea, and the current preview image can be obtained. The obtained preview image includes the user and the sea.

S220: Obtain the subject confidence of the current preview image.

As an example of implementation, the features of the current preview image can be extracted, and then the features of the current preview image can be input into a trained support vector machine regression model for regression prediction to obtain a regression value, and the obtained regression value is determined as the current preview. The subject confidence of the image.

The features of the current preview image include but are not limited to features such as HOG (Histogram of Oriented Gradient, histogram of directional gradients), texture, color, and shape.

The support vector machine regression model includes but is not limited to a support vector machine regression model configured with a linear kernel function, a support vector machine regression model configured with a polynomial kernel function, and a support vector machine regression model configured with a radial basis kernel function.

For example, after obtaining the current preview image, the HOG feature of the current preview person image can be extracted, and the HOG feature can be input into the support vector machine regression model configured by the linear kernel function, and the regression value can be obtained. For example, the regression value is between 0 and 0. 1, and then the obtained regression value is determined as the subject confidence of the current preview image.

Among them, the specific implementation method of extracting the HOG feature of the current preview image is as follows: after obtaining the current preview image, the entire image can be normalized, and then the gradients in the abscissa and ordinate directions of the image are calculated, and each pixel is calculated accordingly. The value of the gradient direction of the position, which then provides an encoding for the local image region, while maintaining a weak sensitivity to the pose and appearance of human objects in the image, due to changes in local illumination and changes in foreground-background contrast, the gradient strength changes The range is very large, and the gradient strength needs to be normalized. Finally, HOG features are collected for all overlapping blocks in the detection window, and they are combined into a final feature vector for classification, and then the HOG features of the current preview image can be obtained. .

For another example, after the current preview image is obtained, the texture feature of the current preview character image can be extracted through the extraction module, wherein the texture image is the regular distribution of gray values caused by the repeated arrangement of the objects on the image, and then the texture feature is input into the image. In the support vector machine regression model configured by the polynomial kernel function, the regression value can be obtained, and then the obtained regression value is determined as the subject confidence of the current preview image.

Among them, the extraction of the texture features of the currently previewed person image can be performed through statistical algorithms, geometric algorithms, and signal processing algorithms.

For another example, when the current preview image is obtained, the shape feature of the current preview person image can be extracted by the extraction module, and then the shape feature is input into the support vector machine regression model configured by the radial basis kernel function, the regression value can be obtained, and then the obtained value is obtained. The regression value of is determined as the subject confidence of the current preview image.

Among them, the shape feature of the currently previewed person image can be extracted through a boundary feature algorithm, a shape invariant moment algorithm, and a Fourier shape descriptor algorithm.

As another implementation example, the current preview image is input into a trained neural network classifier, wherein the neural network classifier is a classification network model based on a convolutional neural network, and the neural network classifier includes a The input layer, and the output layer for outputting the original output value of the network, then obtain the original output value of the network output by the neural network classifier, convert the target output value in the original output value of the network to the probability, and determine the probability obtained after the conversion. is the subject confidence of the current preview image, wherein the target output value refers to the output value corresponding to the category label having the subject category label. As an example, the classification network model may be a binary classification network model.

For example, after obtaining the current preview image, the current preview image can be input into a trained neural network classifier, and the neural network classifier can extract the features of the preview image, and based on the features of the preview image, the original output value of the network can be output, wherein, The original output value of the network may have only one output value, and then the original output value of the network is converted into probability by the Sigmoid function, and the probability obtained after the conversion is determined as the subject confidence of the current preview image.

For another example, when the current preview image is obtained, the current preview image can be input into a trained neural network classifier, and the neural network classifier can extract the features of the preview image, and based on the features of the preview image, the original output value of the network can be output, wherein , the original output value of the network can include two output values, and then the two output values in the original output value of the network are converted into probability by the Softmax function, and the conversion probability of the output value corresponding to the category label "with subject" is determined as the current preview image subject confidence.

S230, according to the subject confidence, detect whether the current preview image is an image with a subject or an image without a subject.

For example, to obtain the subject confidence of the current preview image, the subject confidence can be compared with the target threshold. For example, the subject confidence is 0.6 and the target threshold is 0.5. At this time, the subject confidence 0.6 is greater than the target threshold 0.5, then It is determined that the current preview image is an image with a subject.

S240, when it is detected that the current preview image is an image without a subject, focus processing is performed according to the target area in the current preview image.

Wherein, the target area may be an intermediate area.

That is to say, when it is detected that the current preview image is an image without a subject, the center area in the current preview image can be automatically focused.

S250, when it is detected that the current preview image is an image with a subject, the size and regional position of the subject object may be detected from the current preview image by means of target segmentation, and focus is performed according to the detected size and regional position of the subject target deal with.

In the embodiment of the present application, when it is detected that the current preview image is an image with a subject, the current preview image can be input into a trained target segmentation network model, wherein the target segmentation network model has been learned to obtain the characteristics of each image and the subject. The mapping relationship between size and region position, the target segmentation network model includes an input layer for extracting image features, and an output layer for outputting the size and region position of the subject, and then obtaining the size of the subject object output by the target segmentation network model and area position, and focus processing is performed according to the detected size and area position of the main target.

The target segmentation network model is a segmentation model based on a target segmentation algorithm, wherein the target segmentation algorithms include but are not limited to deep learning series segmentation algorithms, U-Net, and FCN (fuzzy c-means, fuzzy C-means clustering) algorithms , wherein this type of target segmentation algorithm includes an Encoder (encoder) feature encoding module and a Decoder (decoder) target template generation module. For example, the network structure of the target segmentation network model can be shown in Figure 3.

That is to say, when it is detected that the current preview image is an image with a subject, the current preview image can be input into the trained target segmentation network model, and the target segmentation network model can extract image features, and based on the size and area of each image feature and the subject The mapping relationship between the positions can output the output layer of the size of the subject and the position of the region, and then obtain the size and region position of the subject target output by the target segmentation network model, and automatically focus according to the size and region position of the detected subject target. .

According to the focusing method of the embodiment of the present application, the current preview image can be acquired, and then the subject confidence level of the current preview image can be acquired. According to the subject confidence level, it is detected whether the current preview image is an image with or without subject. When there is no subject image, focus processing is performed according to the target area in the current preview image; when it is detected that the current preview image is an image with a subject, the size and area position of the subject target can be detected from the current preview image by means of target segmentation , and focus processing is performed according to the size and area position of the detected subject. The method can detect whether the current preview image is an image with or without a subject by obtaining the confidence of the subject of the preview image, and adopt a corresponding focusing method according to the detection result to realize focusing processing, which realizes the detection in the scene with or without subject image. Focusing processing reduces the target erroneous segmentation in scenes without subject images, avoids the inaccuracy of autofocus caused by subject segmentation, improves the accuracy of target subject segmentation, and improves the accuracy of autofocusing.

FIG. 4 is a flowchart of a focusing method according to another specific embodiment of the present application. As shown in FIG. 3 , the focusing method may include:

S410, obtain the current preview image.

S420: Obtain the subject confidence of the current preview image.

S430, according to the subject confidence, detect whether the current preview image is an image with a subject or an image without a subject.

S440, when it is detected that the current preview image is an image without a subject, focus processing is performed according to the target area in the current preview image.

Wherein, the target area may be an intermediate area.

That is to say, when it is detected that the current preview image is an image without a subject, the center area in the current preview image can be automatically focused.

It should be noted that, in the embodiments of the present application, for the implementation manner of the foregoing steps S410 to S440, reference may be made to the description of the implementation manner of the foregoing steps S210 to S240, and details are not described herein again.

S450, when it is detected that the current preview image is an image with a subject, the size and regional position of the subject object may be detected from the current preview image by means of target detection, and focus is performed according to the detected size and regional position of the subject target deal with.

In the embodiment of the present application, when it is detected that the current preview image is an image with a subject, the current preview image can be input into a trained target detection network model, wherein the target detection network model has been learned to obtain the characteristics of each image and the internal characteristics of the subject. The mapping relationship between the size and position of the inscribed rectangle, the target detection network model includes an input layer for extracting image features, and an output layer for outputting the size and position of the inscribed rectangle of the subject, and then obtains the output of the target detection network model. The size and position of the inscribed rectangle of the main body, the size of the inscribed rectangle of the main body determines the size of the subject object, and the position of the inscribed rectangle of the main body is determined as the area position of the main body target, and according to the detected size and area of the main body target position to focus.

The target detection model is a detection model based on a target detection algorithm, wherein the target detection algorithm includes but is not limited to algorithms such as SSD, YOLO, and Fast R-CNN (Fast Regional Convolutional Neural Network).

That is to say, when it is detected that the current preview image is an image with a subject, the current preview image can be input into the trained target detection network model, and the target detection network model can extract image features, and based on each image feature and the main body inscribed rectangle The mapping relationship between size and position can output the output layer of the size and position of the inscribed rectangle of the main body, and then obtain the size and position of the inscribed rectangle of the main body output by the target segmentation network model. Area position for autofocus.

It should be noted that, since the circumscribed rectangle of the main body includes the background area, it is easy to cause inaccuracy of auto focus. Therefore, in the embodiment of the present application, the target detection network model outputs the size and position of the inscribed rectangle of the main body.

According to the focusing method of the embodiment of the present application, the current preview image can be acquired, and then the subject confidence level of the current preview image can be acquired. According to the subject confidence level, it is detected whether the current preview image is an image with or without subject. When there is no subject image, focus processing is performed according to the target area in the current preview image; when it is detected that the current preview image is an image with a subject, the size and area position of the subject target can be detected from the current preview image by means of target detection. , and focus processing is performed according to the size and area position of the detected subject. The method can detect whether the current preview image is an image with or without a subject by obtaining the confidence of the subject of the preview image, and adopt a corresponding focusing method according to the detection result to realize focusing processing, which realizes the detection in the scene with or without subject image. Focusing processing reduces target false detection in scenes without subject images, avoids the inaccuracy of autofocus caused by subject detection, improves the accuracy of target subject detection, and improves the accuracy of autofocus.

Corresponding to the focusing methods provided by the above-mentioned embodiments, an embodiment of the present application further provides a focusing device. Since the focusing device provided by the embodiments of the present application corresponds to the focusing methods The embodiments of the focusing method are also applicable to the focusing device provided in this embodiment, which will not be described in detail in this embodiment. FIG. 5 is a schematic structural diagram of a focusing device according to an embodiment of the present application.

As shown in FIG. 5 , the focusing device 500 includes: a preview image acquisition module 510 , a subject confidence degree acquisition module 520 , a detection module 530 and a focusing module 540 . in:

The preview image acquisition module 510 is used to acquire the current preview image.

The subject confidence level acquiring module 520 is configured to acquire the subject confidence level of the current preview image; as an example, the subject confidence level acquiring module is specifically used to: extract the features of the current preview image; Input the trained support vector machine regression model for regression prediction, and obtain a regression value; determine the obtained regression value as the subject confidence level of the current preview image.

In the embodiment of the present application, the subject confidence obtaining module 520 is specifically configured to: input the current preview image into a trained neural network classifier; the neural network classifier is a classification network model based on a convolutional neural network, The neural network classifier includes an input layer for extracting features, and an output layer for outputting the original output value of the network; obtaining the original output value of the network output by the neural network classifier; The target output value is subjected to probability conversion, and the converted probability is determined as the subject confidence of the current preview image, wherein the target output value refers to the output value corresponding to the category label having the subject category label.

The detection module 530 is configured to detect whether the current preview image is an image with a subject or an image without a subject according to the confidence of the subject; as an example, the detection module is specifically configured to: determine whether the confidence of the subject is less than the target Threshold; if the subject confidence is less than the target threshold, the current preview image is determined to be an image without subject; if the subject confidence is greater than or equal to the target threshold, the current preview image is determined to have a subject image.

The focusing module 540 is configured to use a corresponding focusing method to perform focusing processing according to the detection result. As an example, the focusing module is specifically configured to: when it is detected that the current preview image is an image with a subject, detect the size and area position of the subject target from the current preview image, The focus processing is performed according to the size and area position of the target; when it is detected that the current preview image is an image without a subject, the focus processing is performed according to the target area in the current preview image.

In the embodiment of the present application, the focusing module 540 is specifically configured to: input the current preview image into a trained target segmentation network model; the target segmentation network model is obtained by learning the size and size of each image feature and the subject. The mapping relationship between the region positions, the target segmentation network model includes an input layer for extracting image features, and an output layer for outputting the size of the subject and the region position; obtaining the subject target output by the target segmentation network model. size and regional location.

In the embodiment of the present application, the focusing module 540 is specifically configured to: input the current preview image into a trained target detection network model; the target detection network model is obtained by learning the features of each image and the inscribed rectangle of the main body The mapping relationship between the size and position of the target detection network model includes an input layer for extracting image features, and an output layer for outputting the size and position of the inscribed rectangle of the main body; Obtain the output of the target detection network model The size and position of the main body inscribed rectangle; the size of the main body inscribed rectangle determines the size of the main body target, and the position of the main body inscribed rectangle is determined as the area position of the main body target.

According to the focusing method of the embodiment of the present application, the current preview image can be obtained, and then the subject confidence of the current preview image can be obtained, and according to the subject confidence, it is detected whether the current preview image is an image with or without a subject, and then, according to the detection result, using The corresponding focusing method is used for focusing processing. The method can detect whether the current preview image is an image with or without a subject by obtaining the confidence of the subject of the preview image, and adopt a corresponding focusing method according to the detection result to realize focusing processing, which realizes the detection in the scene with or without subject image. Focus processing reduces target mis-segmentation or target mis-detection in scenes without subject images, avoids inaccurate auto-focusing caused by subject segmentation or subject detection errors, improves the accuracy of target subject segmentation or target subject detection, and improves the accuracy of autofocus.

In order to realize the above embodiments, the present application also proposes a terminal device.

FIG. 6 is a schematic structural diagram of a terminal device according to an embodiment of the present application. As shown in FIG. 6 , the terminal device 600 may include: a memory 610, a processor 620, and a computer program 630 stored on the memory 610 and executable on the processor 620, and the processor 620 executes the During the program, the focusing method described in any one of the above-mentioned applications in this application is implemented.

In order to implement the above embodiments, the present application further provides a computer-readable storage medium, and when the computer program is executed by a processor, any one of the focusing methods described above is implemented.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

Any description of a process or method in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing a specified logical function or step of the process , and the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application belong.

The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of this application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or a combination of the following techniques known in the art: Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

Those skilled in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program can be stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations to the present application. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (16)

1. A focusing method, comprising:

acquiring a current preview image;

acquiring the subject confidence of the current preview image;

detecting whether the current preview image is a main image or a non-main image according to the main confidence coefficient;

and carrying out focusing processing by adopting a corresponding focusing mode according to the detection result.

2. The focusing method of claim 1, wherein obtaining the subject confidence level of the current preview image comprises:

extracting the characteristics of the current preview image;

inputting the characteristics of the current preview image into a trained support vector machine regression model for regression prediction to obtain a regression value;

and determining the obtained regression value as the subject confidence of the current preview image.

3. The focusing method of claim 1, wherein obtaining the subject confidence level of the current preview image comprises:

inputting the current preview image into a trained neural network classifier; the neural network classifier is a classification network model based on a convolutional neural network, and comprises an input layer for extracting features and an output layer for outputting network original output values;

acquiring a network original output value output by the neural network classifier;

performing probability conversion on a target output value in the network original output values, and determining the probability obtained after conversion as a main confidence coefficient of the current preview image; the target output value is an output value corresponding to the class label with a main class label.

4. The focusing method of claim 1, wherein detecting whether the current preview image is a subject image or a non-subject image according to the subject confidence level comprises:

judging whether the subject confidence is smaller than a target threshold value;

if the subject confidence is smaller than the target threshold, judging that the current preview image is a non-subject image;

and if the subject confidence is greater than or equal to the target threshold, judging that the current preview image is a subject image.

5. The focusing method according to claim 1, wherein the performing the focusing process by using the corresponding focusing manner according to the detection result comprises:

when the current preview image is detected to be a main image, detecting the size and the area position of a main object from the current preview image, and carrying out focusing processing according to the detected size and the area position of the main object;

and when the current preview image is detected to be a no-body image, focusing according to a target area in the current preview image.

6. The focusing method of claim 5, wherein detecting the size and the area position of the subject target from the current preview image comprises:

inputting the current preview image into a trained target segmentation network model; the target segmentation network model is a mapping relation between each image feature and the size and the region position of the main body, and comprises an input layer for extracting the image feature and an output layer for outputting the size and the region position of the main body;

and acquiring the size and the area position of the main target output by the target segmentation network model.

7. The focusing method of claim 5, wherein detecting the size and the area position of the subject target from the current preview image comprises:

inputting the current preview image into a trained target detection network model; the target detection network model is obtained by learning the mapping relation between each image feature and the size and the position of the main body inscribed rectangle, and comprises an input layer for extracting the image feature and an output layer for outputting the size and the position of the main body inscribed rectangle;

acquiring the size and the position of a main body inscribed rectangle output by the target detection network model;

determining the size of the main body inscribed rectangle to be the size of the main body target, and determining the position of the main body inscribed rectangle to be the area position of the main body target.

8. A focusing apparatus, comprising:

the preview image acquisition module is used for acquiring a current preview image;

a subject confidence coefficient obtaining module, configured to obtain a subject confidence coefficient of the current preview image;

the detection module is used for detecting whether the current preview image is a main image or a non-main image according to the main confidence coefficient;

and the focusing module is used for carrying out focusing processing by adopting a corresponding focusing mode according to the detection result.

9. The focusing device of claim 8, wherein the subject confidence acquisition module is specifically configured to:

extracting the characteristics of the current preview image;

inputting the characteristics of the current preview image into a trained support vector machine regression model for regression prediction to obtain a regression value;

and determining the obtained regression value as the subject confidence of the current preview image.

10. The focusing device of claim 8, wherein the subject confidence acquisition module is specifically configured to:

inputting the current preview image into a trained neural network classifier; the neural network classifier is a classification network model based on a convolutional neural network, and comprises an input layer for extracting features and an output layer for outputting network original output values;

acquiring a network original output value output by the neural network classifier;

performing probability conversion on a target output value in the network original output values, and determining the probability obtained after conversion as a main confidence coefficient of the current preview image; the target output value is an output value corresponding to the class label with a main class label.

11. The focusing device of claim 8, wherein the detection module is specifically configured to:

judging whether the subject confidence is smaller than a target threshold value;

if the subject confidence is smaller than the target threshold, judging that the current preview image is a non-subject image;

and if the subject confidence is greater than or equal to the target threshold, judging that the current preview image is a subject image.

12. The focusing device of claim 8, wherein the focusing module is specifically configured to:

when the current preview image is detected to be a main image, detecting the size and the area position of a main object from the current preview image, and carrying out focusing processing according to the detected size and the area position of the main object;

and when the current preview image is detected to be a no-body image, focusing according to a target area in the current preview image.

13. The focusing device of claim 12, wherein the focusing module is specifically configured to:

inputting the current preview image into a trained target segmentation network model; the target segmentation network model is a mapping relation between each image feature and the size and the region position of the main body, and comprises an input layer for extracting the image feature and an output layer for outputting the size and the region position of the main body;

and acquiring the size and the area position of the main target output by the target segmentation network model.

14. The focusing device of claim 12, wherein the focusing module is specifically configured to:

inputting the current preview image into a trained target detection network model; the target detection network model is obtained by learning the mapping relation between each image feature and the size and the position of the main body inscribed rectangle, and comprises an input layer for extracting the image feature and an output layer for outputting the size and the position of the main body inscribed rectangle;

acquiring the size and the position of a main body inscribed rectangle output by the target detection network model;

determining the size of the main body inscribed rectangle to be the size of the main body target, and determining the position of the main body inscribed rectangle to be the area position of the main body target.

15. A terminal device, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the focusing method as claimed in any one of claims 1 to 7 when executing the program.

16. A computer-readable storage medium on which a computer program is stored, the program, when executed by a processor, implementing a focusing method as recited in any one of claims 1 to 7.

Images