WO2021057652A1 - Focusing method and apparatus, electronic device, and computer readable storage medium - Google Patents

Abstract

A focusing method, comprising: obtaining a first preview image; determining a target main body in the first preview image; when the first preview image satisfies a preset condition, determining a centroid region of the target main body, wherein the preset condition comprises at least one of a face area corresponding to the target main body being less than or equal to a face area threshold and the brightness of the first preview image being less than or equal to a brightness threshold; and focusing according to the centroid region.

Description

Focusing method and device, electronic equipment, and computer readable storage medium

Cross-references to related applications

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on September 29, 2019, the application number is 201910931431.2, and the invention title is "focusing method and device, electronic equipment, computer-readable storage medium", and the entire content of it is approved The reference is incorporated in this application.

Technical field

This application relates to the field of image processing, in particular to a focusing method and device, electronic equipment, and computer-readable storage medium.

Background technique

With the development of image processing technology, people have higher and higher requirements for images. In the image shooting process, the traditional focusing method is to focus on the center of the entire image. However, in some scenes, the current focusing method has the problem that the subject cannot be accurately focused and the subject is blurred.

Summary of the invention

According to various embodiments of the present application, a focusing method, device, electronic device, and computer-readable storage medium are provided.

A focusing method includes:

Obtain the first preview image;

Determine the target subject in the first preview image;

When the first preview image meets the preset conditions, determine the centroid area of the target subject, where the preset conditions include that the face area corresponding to the target subject is less than or equal to the face area threshold, and the brightness of the first preview image is less than or equal to the brightness threshold At least one of

Focus based on the center of mass area.

A focusing device includes:

An obtaining module for obtaining the first preview image;

The subject determination module is used to determine the target subject in the first preview image;

The centroid area determination module is used to determine the centroid area of the target subject when the first preview image meets preset conditions, where the preset conditions include that the face area corresponding to the target subject is less than or equal to the face area threshold, and the first preview image The brightness of is less than or equal to at least one of the brightness thresholds;

The focusing module is used for focusing according to the center of mass area.

An electronic device includes a memory and a processor, and a computer program is stored in the memory. When the computer program is executed by the processor, the processor executes the following steps:

Obtain the first preview image;

Determine the target subject in the first preview image;

When the first preview image meets the preset conditions, determine the centroid area of the target subject, where the preset conditions include that the face area corresponding to the target subject is less than or equal to the face area threshold, and the brightness of the first preview image is less than or equal to the brightness threshold At least one of

Focus based on the center of mass area.

A computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the following steps:

Obtain the first preview image;

Determine the target subject in the first preview image;

When the first preview image meets the preset conditions, determine the centroid area of the target subject, where the preset conditions include that the face area corresponding to the target subject is less than or equal to the face area threshold, and the brightness of the first preview image is less than or equal to the brightness threshold At least one of

Focus based on the center of mass area.

The above-mentioned focusing method and device, electronic equipment, and computer-readable storage medium acquire a first preview image, and determine the target subject in the first preview image. When the first preview image satisfies the target subject’s corresponding face area is less than or equal to the face area When the threshold value and the brightness of the first preview image are less than or equal to the brightness threshold value, determine the center of mass area of the target subject and focus based on the center of mass area, which can quickly find an appropriate position when the face is too small or the light is too dark Carry out autofocus to improve the focus efficiency, while avoiding focusing on the background area, improving the focus accuracy and the sharpness of the subject.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

Fig. 1 is a schematic diagram of an image processing circuit in an embodiment.

Fig. 2 is a flowchart of a focusing method in an embodiment.

Fig. 3 is a schematic diagram of the process of determining the centroid region of the target subject in an embodiment.

Fig. 4 is a schematic diagram of a process of determining a target subject in a first preview image in an embodiment.

Fig. 5 is a schematic diagram of the process of obtaining the centroid coordinates in an embodiment.

Fig. 6 is a schematic diagram of a subject detection process in an embodiment.

FIG. 7 is a schematic flowchart of a focusing method in another embodiment.

Fig. 8 is a structural block diagram of a focusing device in an embodiment.

Fig. 9 is a schematic diagram of the internal structure of an electronic device in an embodiment.

detailed description

In order to make the purpose, technical solutions, and advantages of this application clearer, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the application, and not used to limit the application.

It can be understood that the terms "first", "second", etc. used in this application can be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from another element. For example, without departing from the scope of the present application, the first preview image may be referred to as the second preview image, and similarly, the second preview image may be referred to as the first preview image. Both the first preview image and the second preview image are preview images, but they are not the same preview image.

The focusing method in the embodiment of the present application can be applied to electronic equipment. The electronic device may be a computer device with a camera, a personal digital assistant, a tablet computer, a smart phone, a wearable device, etc. When the camera in the electronic device takes an image, it will automatically focus to ensure that the captured image is clear. The number of cameras in the electronic device is not limited, for example, it can be one, two, three, and so on.

In one embodiment, the above electronic device may include an image processing circuit, which may be implemented by hardware and/or software components, and may include various processing units that define an ISP (Image Signal Processing, image signal processing) pipeline. Fig. 1 is a schematic diagram of an image processing circuit in an embodiment. As shown in FIG. 1, for ease of description, only various aspects of the image processing technology related to the embodiments of the present application are shown.

As shown in FIG. 1, the image processing circuit includes an ISP processor 140 and a control logic 150. The image data captured by the imaging device 110 is first processed by the ISP processor 140, and the ISP processor 140 analyzes the image data to capture image statistics that can be used to determine and/or one or more control parameters of the imaging device 110. The imaging device 110 may include a camera having one or more lenses 112, an image sensor 114, and an actuator 116. The actuator 116 can drive the lens 112 to move. The image sensor 114 may include a color filter array (such as a Bayer filter). The image sensor 114 may obtain the light intensity and wavelength information captured by each imaging pixel of the image sensor 114, and provide a set of raw materials that can be processed by the ISP processor 140. Image data. The sensor 120 (such as a gyroscope) can provide the collected image processing parameters (such as anti-shake parameters) to the ISP processor 140 based on the interface type of the sensor 120. The sensor 120 interface may use SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above interfaces.

In addition, the image sensor 114 may also send raw image data to the sensor 120, and the sensor 120 may provide the raw image data to the ISP processor 140 based on the interface type of the sensor 120, or the sensor 120 may store the raw image data in the image memory 130.

The ISP processor 140 processes the original image data pixel by pixel in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 140 may perform one or more image processing operations on the original image data, and collect statistical information about the image data. Among them, the image processing operations can be performed with the same or different bit depth accuracy.

The ISP processor 140 may also receive image data from the image memory 130. For example, the sensor 120 interface sends the original image data to the image memory 130, and the original image data in the image memory 130 is then provided to the ISP processor 140 for processing. The image memory 130 may be a part of a memory device, a storage device, or an independent dedicated memory in an electronic device, and may include DMA (Direct Memory Access) features.

When receiving raw image data from the image sensor 114 interface or from the sensor 120 interface or from the image memory 130, the ISP processor 140 may perform one or more image processing operations, such as temporal filtering. The processed image data can be sent to the image memory 130 for additional processing before being displayed. The ISP processor 140 receives the processed data from the image memory 130, and performs image data processing in the original domain and in the RGB and YCbCr color spaces on the processed data. The image data processed by the ISP processor 140 may be output to the display 170 for viewing by the user and/or further processed by a graphics engine or a GPU (Graphics Processing Unit, graphics processor). In addition, the output of the ISP processor 140 can also be sent to the image memory 130, and the display 170 can read image data from the image memory 130. In one embodiment, the image memory 130 may be configured to implement one or more frame buffers. In addition, the output of the ISP processor 140 may be sent to the encoder/decoder 160 in order to encode/decode image data. The encoded image data can be saved and decompressed before being displayed on the display 170 device. The encoder/decoder 160 may be implemented by a CPU or GPU or a co-processor.

The statistical data determined by the ISP processor 140 may be sent to the control logic 150. For example, the statistical data may include image sensor 114 statistical information such as automatic exposure, automatic white balance, automatic focus, flicker detection, black level compensation, and lens 112 shading correction. The control logic 150 may include a processor and/or a microcontroller that executes one or more routines (such as firmware). The one or more routines can determine the control parameters and ISP processing of the imaging device 110 based on the received statistical data. The control parameters of the device 140. For example, the control parameters of the imaging device 110 may include sensor 120 control parameters (such as gain, integration time for exposure control, anti-shake parameters, etc.), camera flash control parameters, lens 112 control parameters (such as focal length for focusing or zooming), or these The combination of parameters. The control logic 150 can output the control parameters of the lens 112 to the actuator 116, and the actuator 116 drives the lens 112 to move according to the control parameters. The ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (for example, during RGB processing), and lens 112 shading correction parameters.

In the embodiment of the present application, the processor 140 included in the electronic device executes the computer program stored on the memory to obtain the first preview image; determine the target subject in the first preview image; when the first preview image satisfies the preset In the condition, determine the centroid area of the target subject, where the preset condition includes at least one of the face area corresponding to the target subject is less than or equal to the face area threshold, and the brightness of the first preview image is less than or equal to the brightness threshold; according to the centroid area Focus.

Fig. 2 is a flowchart of a focusing method in an embodiment. The focusing method in this embodiment is described by taking the terminal operating in FIG. 1 as an example. As shown in FIG. 5, the focusing method includes operation 202 to operation 208.

In operation 202, a first preview image is obtained.

Among them, the preview image may be a visible light image. The preview image refers to the image presented on the screen of the electronic device when the camera is not shooting. The first preview image may be a preview image of the current frame.

Specifically, when the collection instruction is acquired, the camera of the electronic device collects the first preview image and displays it on the display device of the electronic device.

In operation 204, the target subject in the first preview image is determined.

Among them, the target subject can be a pre-configured recognizable object. The object can be a person, thing, animal, etc. Objects can include flowers, mountains, trees, etc. Animals may include cats, dogs, cows, sheep, tigers, etc. The target body can be an irregular graphic contour or a regular graphic contour.

Specifically, the electronic device may perform subject detection on the first preview image to determine the target subject in the first preview image. Or the electronic device can obtain the target subject by detecting the feature points in the first preview image.

Operation 206: When the first preview image satisfies a preset condition, determine the centroid area of the target subject, where the preset condition includes that the face area corresponding to the target subject is less than or equal to the face area threshold, and the brightness of the first preview image is less than or Equal to at least one of the brightness thresholds.

Among them, the center of mass refers to an imaginary point where mass is considered to be concentrated in matter. The center of mass area refers to the area where the center of mass is located. The centroid area may be a square area, a circular area, a triangular area, etc., and is not limited thereto. The face area threshold refers to the critical value of the face area stored in the electronic device. The face area threshold can be set according to requirements. For example, at the same distance, the face area of user A is greater than the preset face area threshold. In order to meet the personalized settings, user A can configure the face area threshold by himself. The brightness threshold refers to the critical value of the brightness of the image stored in the electronic device. The brightness threshold can also be configured according to time. For example, during the day, that is, from seven in the morning to six in the evening, is the first brightness threshold; at night, that is, from six in the evening to seven in the morning, is the second brightness threshold. The first brightness threshold is greater than the second brightness threshold. Or, the brightness threshold is configured according to the location. The electronic device obtains the location, and when the location is outdoor, the first brightness threshold is automatically selected; when the location is indoor, the second brightness threshold is automatically selected, where the first brightness threshold is greater than the second brightness threshold.

Specifically, when the electronic device detects that the first preview image satisfies the preset condition, that is, the preset condition includes that the face area corresponding to the target subject is less than the face area threshold, the face area is equal to the face area threshold, and the first preview image The brightness is less than the brightness threshold, the brightness of the first preview image is equal to the brightness threshold, the face area corresponding to the target subject is less than the face area threshold and the brightness is less than the brightness threshold, the face area is less than the face area threshold and the brightness is equal to the brightness threshold, face area It is equal to the area threshold and the brightness is less than the brightness threshold, or the face area is equal to the area threshold and the brightness is equal to the brightness threshold, and the center of mass region of the target subject is determined.

In this embodiment, the electronic device determines the area where the target subject in the first preview image is located. The electronic device detects whether there is a human face in the area where the target subject is located. When there is a face in the area where the target subject is located, it is determined whether the face area of the target subject meets the face area threshold.

In this embodiment, the electronic device obtains the face area by detecting the feature points of the face, and then obtains the number of pixels corresponding to the area enclosed by the face feature points to calculate the face area corresponding to the target subject.

In this embodiment, the electronic device can obtain the brightness of the first preview image through the HSV (Hue Saturation Value) color model algorithm.

In this embodiment, the electronic device can obtain the environmental brightness, convert the environmental brightness into the image brightness, and obtain the image brightness.

In this embodiment, the electronic device can obtain the distance from the human face to the screen of the mobile phone through the depth camera, and calculate the area of the human face according to the distance. Alternatively, the electronic device may obtain the face area corresponding to the target subject according to the correspondence between the distance and the face area.

In operation 208, focus is performed according to the center of mass region.

Wherein, the auto-focusing method may be Contrast Detection Auto Focus (CDAF), Phase Detection Auto Focus (PDAF) or Laser Focus (Laser Detection Auto Focus, LADF), etc., which are not limited to this.

Specifically, the electronic device may use the center of mass area as the focus area in the first preview image. The electronic device can control the camera to auto-focus according to the focus area. That is, the electronic device adjusts the lens according to the focus area to perform automatic focusing. The phase detection autofocus is to obtain the phase difference through the sensor, calculate the defocus value according to the phase difference, control the lens movement according to the defocus value, and then look for the focus value (FV) peak value. Laser focusing is to calculate the distance between the target and the test instrument by recording the time difference between the infrared laser emitted from the camera, reflected by the target surface, and finally received by the rangefinder to achieve focusing.

In this embodiment, the electronic device may use the centroid area as the reference area, and process the focus area according to the reference area to perform autofocus.

In the focusing method in this embodiment, a first preview image is acquired, and the target subject in the first preview image is determined. When the first preview image satisfies the target subject's face area less than or equal to the face area threshold, the first preview image When the brightness is less than or equal to at least one of the brightness thresholds, the centroid area of the target subject is determined, and the centroid area is used as the focus area for focusing. It can quickly find an appropriate position for autofocus when the face is too small or the light is too dark. Improve the focus efficiency, while avoiding focusing on the background area, improving the focus accuracy and the sharpness of the subject.

In an embodiment, as shown in FIG. 3, FIG. 3 is a schematic diagram of the process of determining the centroid region of the target subject in an embodiment, including:

In operation 302, a set of pixels corresponding to the target subject is obtained.

Among them, the set of pixels refers to multiple pixels corresponding to the target subject.

Specifically, the electronic device determines the set of pixel points corresponding to the target subject. For example, if the target subject is a human face, then the set of pixels corresponding to the target subject is a plurality of pixels surrounded by the contour of the human face. The target subject is a portrait, so the set of pixels corresponding to the target subject is a plurality of pixels surrounded by the portrait.

In operation 304, the coordinates of each pixel in the set of pixel points corresponding to the target subject are obtained.

Among them, each pixel has pixel coordinates.

Specifically, the electronic device obtains the coordinates of each pixel in the set of pixel points corresponding to the target body. For example, if there are 100 pixels in the set of pixels, the electronic device obtains the coordinates of each pixel in the 100 pixels.

In operation 306, the centroid coordinates of the target body are obtained according to the coordinates of each pixel point.

Specifically, the electronic device finds the minimum value of the abscissa, the maximum value of the abscissa, the minimum value of the ordinate, and the maximum value of the ordinate from the coordinates of each pixel point. The electronic device obtains the abscissa of the centroid coordinate by averaging the abscissa minimum value and the abscissa maximum value. The electronic device obtains the ordinate of the center of mass coordinate by averaging the minimum and maximum values of the ordinate.

In this embodiment, the electronic device can obtain the pixel coordinates corresponding to the contour of the target subject. The centroid coordinates of the target body are obtained according to the target pixel point coordinates corresponding to each pixel point on the contour of the target body. The electronic device can average the abscissa and the ordinate of the pixel coordinates on the contour to obtain the centroid coordinates of the target body.

In this embodiment, when the calculated centroid coordinates contain decimals, the centroid coordinates are rounded off. Or, round the decimal to get the centroid coordinate.

In operation 308, the area where the centroid coordinates are located is determined.

Specifically, the electronic device can determine the area where the centroid coordinates are located with the centroid coordinates as the center. For example, take the centroid coordinates as the center, take 150 pixels in the horizontal direction, and take 150 pixels in the vertical direction as the area where the centroid coordinates are located. Or taking the centroid coordinates as the center, taking a circle with 150 pixels as the radius as the area where the centroid coordinates are located, etc. are not limited to this.

In operation 310, the area where the center of mass coordinates is located is taken as the center of mass area of the target subject.

The focusing method in the embodiment of the present application obtains a collection of pixel points corresponding to the target subject, obtains the coordinates of each pixel point, and obtains the centroid coordinates, and uses the area where the centroid coordinates are located as the centroid area of the target subject, which can determine the focus area and improve the target subject The clarity of the shot.

In one embodiment, obtaining the coordinates of the center of mass of the target subject according to the coordinates of each pixel includes: calculating a weighted average of the coordinates of each pixel to obtain the coordinates of the center of mass of the target subject.

Specifically, the electronic device calculates the weighted average value of the abscissa of the coordinates of each pixel point, and calculates the weighted average value of the ordinate of the coordinates of each pixel point, to obtain the centroid coordinates of the target body. For example, there are 100 target pixel coordinates, namely (X1,Y1), (X2,Y2)...(X100,Y100), set the center of mass coordinates to (A, B), then A=(X1+X2+...+X100 )/100, Y=(Y1+Y2+...+Y100)/100.

The focusing method in the embodiment of the present application calculates the weighted average of the coordinates of each pixel point to obtain the centroid coordinates of the target body, which can obtain the coordinates of all the target pixels for calculation, and improve the accuracy of the centroid coordinate calculation.

In one embodiment, when the first preview image satisfies a preset condition, the centroid area of the target subject is determined, where the preset condition includes that the face area corresponding to the target subject is less than or equal to the face area threshold and the first preview image The brightness is less than or equal to at least one of the brightness thresholds, including:

When the first preview image satisfies that the face area corresponding to the target subject is greater than the face area threshold, determine whether the brightness of the first preview image is greater than the brightness threshold;

When the brightness of the first preview image is less than or equal to the brightness threshold, the center of mass region of the target subject is determined.

Among them, the priority of determining that the face area corresponding to the target subject is greater than the threshold of the face area may be higher than the priority of determining the brightness condition.

Specifically, when the electronic device detects that the first preview image satisfies that the face area corresponding to the target subject in the first preview image is greater than the face area threshold, it determines whether the brightness of the first preview image is greater than the brightness threshold. When the electronic device detects that the brightness of the first preview image is less than or equal to the brightness threshold, the center of mass region of the target subject is determined.

The focusing method in the embodiment of the present application determines whether it is necessary to determine the center of mass region of the target subject based on the two conditions of whether the face area is greater than the face area threshold and whether the brightness of the first preview image is greater than the brightness threshold. When the face area threshold is too dark and the light is too dark, it quickly finds the appropriate position for autofocus, which improves the focusing efficiency, while avoiding focusing on the background area, improving the focusing accuracy and the sharpness of the subject.

In one embodiment, as shown in FIG. 4, it is a schematic diagram of the process of determining the target subject in the first preview image in one embodiment, including:

In operation 402, subject detection is performed on the first preview image to obtain the area where the target subject is located, where the image corresponding to the area where the target subject is located is the subject mask map.

Among them, the area where the target subject is located can be a regular area. For example, the area where the target subject is located is a rectangular area. The area where the target subject is located is larger than the area of the target subject. The subject mask (mask) is an image filter template used to identify the subject in the image, which can block other parts of the image and filter out the subject in the image.

Specifically, the electronic device performs subject detection on the first preview image to obtain the area where the target subject is located.

In operation 404, the gray value of each pixel in the area where the target subject is located is obtained.

Among them, the main mask image generally contains two colors, namely black and white. And the main body mask image generally includes two gray values, for example, 0 and 255 are not limited thereto.

Specifically, the electronic device obtains the gray value of each pixel in the area where the target subject is located in the subject mask.

In operation 406, a set of target pixels meeting the target gray value in the area where the target subject is located is determined.

Specifically, the target gray value refers to the gray value corresponding to the target subject. Since the subject mask image generally contains two gray values, such as 0 and 255, 0 can represent the background and 255 represents the subject. The electronic device determines a set of target pixels meeting the target gray value in the area where the target subject is located.

In operation 408, the area corresponding to the target pixel set is used as the target subject.

Specifically, the electronic device uses the area in the first preview image corresponding to the target pixel point set as the target subject. For example, the target pixel point set in the subject mask image is (X1, Y1), (X2, Y2)...(X100, Y100), then the pixel point coordinates corresponding to the target subject in the first preview image are also (X1, Y1), (X2, Y2)... (X100, Y100).

In the focusing method in the embodiment of the present application, subject detection is performed on the first preview image to obtain the area where the target subject is located, obtain the gray value of each pixel, determine the area that meets the target gray value, and obtain the target subject, which can be passed through the subject mask The map determines the target subject, thereby determining the center of mass area of the target subject, and using the center of mass area as the focus area for focusing. It can quickly find the appropriate position for autofocus when the face is too small or the light is too dark, which improves the focus efficiency while avoiding Focus on the background area to improve the clarity of the subject.

Fig. 5 is a schematic diagram of the process of obtaining the centroid coordinates in an embodiment. As shown in Figure 5, it includes the following operations:

In operation 502, the area where the target subject is located is obtained.

In operation 504, each pixel in the area where the target subject is traversed.

In operation 506, the pixel coordinates with a gray value of 255 are recorded.

In operation 508, the average value of the pixel point coordinates with the gray value of 255 is calculated to obtain the centroid coordinates.

The focusing method in the embodiment of the present application obtains the area where the target body is located, traverses each pixel point in the area where the target body is located, records the pixel coordinates with a gray value of 255, and calculates the average value to obtain the centroid coordinates, which can obtain the centroid coordinates. Improve focus accuracy and sharpness of the subject.

In one embodiment, determining the target subject in the first preview image includes: acquiring a second preview image, where the second preview image is a forward frame image of the first preview image; subject detection is performed on the second preview image, Obtain the reference subject; use the reference subject as the target subject in the first preview image.

Wherein, the second preview image is a forward frame image of the first preview image. The frame number of the forward frame is less than the preset number. For example, the preset number may be 5 frames, then the forward frame image may be an image corresponding to the first five frames of the first preview image, and may not be an image corresponding to the first six frames of the first preview image.

Specifically, the electronic device acquires the second preview image, and performs subject detection on the second preview image to obtain the reference subject. Since the image gap between adjacent preset frames is small, the electronic device can use the reference subject as the target subject in the first preview image.

In the focusing method in the embodiment of the present application, a second preview image is taken, where the second preview image is a forward frame image of the first preview image, and subject detection is performed on the second preview image to obtain a reference subject. The image gap between frames is small, and the reference subject is used as the target subject in the first preview image, which can improve the efficiency of image processing and the efficiency of focusing.

In one embodiment, performing subject detection on the second preview image to obtain the reference subject includes: inputting the second preview image into the subject detection model to obtain the reference subject in the second preview image, wherein the subject detection model is a pre- A model trained based on the visible light map, center weight map, and the corresponding labeled subject of the same scene, or the subject detection model is based on the visible light map, center weight map, depth map, and corresponding labeled subject of the same scene in advance The trained model.

Among them, the central weight map refers to a map used to record the weight value of each pixel in the visible light image. The weight value recorded in the center weight map gradually decreases from the center to the four sides, that is, the center weight is the largest, and the weight gradually decreases toward the four sides. The weight value from the center pixel of the visible light image to the edge pixel of the image is gradually reduced through the center weight map.

The subject detection model is obtained by pre-collecting a large amount of training data, and inputting the training data into the subject detection model containing the initial network weights for training. Each set of training data includes the visible light map, the center weight map and the labeled subject mask map corresponding to the same scene. Among them, the visible light map and the center weight map are used as the input of the trained subject detection model, and the labeled subject mask map is used as the ground truth that the trained subject detection model expects to output.

Specifically, the electronic device may input the second preview image into the subject detection model for detection, and obtain the reference subject in the second preview image.

In the focusing method in the embodiment of the present application, the second preview image is input into the subject detection model to obtain the reference subject in the second preview image, which can accurately detect the target subject in the second preview image, thereby accurately detecting the first Preview the target subject in the image to improve the accuracy of focus and the clarity of the subject.

In one embodiment, inputting the second preview image into the subject detection model to obtain the reference subject in the second preview image includes:

Generating a center weight map corresponding to the second preview image;

Input the second preview image and the center weight map into the subject detection model to obtain the subject area confidence map;

Process the confidence map of the subject area to obtain the subject mask map;

Determine the reference subject in the second preview image according to the subject mask map.

Specifically, the electronic device may generate a corresponding center weight map according to the size of the second preview image. The electronic device inputs the second preview image and the center weight map into the subject detection model for detection, and obtains the subject area confidence map. The subject area confidence map is used to record the probability of the subject which can be recognized. For example, the probability that a certain pixel belongs to a person is 0.6, the probability of a flower is 0.2, and the probability of a background is 0.2.

There are some low-confidence and scattered points in the confidence map of the subject area. The subject area confidence map can be filtered by the ISP processor or the central processing unit to obtain the subject mask map. The filtering process can be configured to configure a confidence threshold, and the electronic device filters pixels whose confidence values are lower than the confidence threshold in the confidence map of the subject area. The confidence threshold can be an adaptive confidence threshold, or a fixed threshold, or a threshold corresponding to a regional configuration. The electronic device determines the reference subject in the second preview image according to the subject mask map.

In this embodiment, determining the reference subject in the second preview image according to the subject mask image includes: obtaining the gray value of each pixel in the subject mask image, and determining the target pixel in the subject mask image that meets the target gray value Point set, the area corresponding to the target pixel point set is taken as the target subject.

The focusing method in the embodiment of the present application generates a center weight map corresponding to the second preview image, and inputs the second preview image and the center weight map into the subject detection model to obtain a confidence map of the subject area, and a confidence map of the subject area Processing is performed to obtain the subject mask map, and the reference subject in the second preview image is determined according to the subject mask map, and the target subject in the second preview image can be accurately detected, thereby performing centroid calculation and improving focus accuracy.

In one embodiment, processing the confidence map of the subject area to obtain the subject mask map includes:

Perform adaptive confidence threshold filtering processing on the subject region confidence map to obtain a binarized mask map; perform morphological processing and guided filtering processing on the binarized mask map to obtain a subject mask map.

Specifically, after the ISP processor or central processing unit filters the confidence map of the subject area according to the adaptive confidence threshold, the confidence value of the retained pixel is represented by 1, and the confidence value of the removed pixel is represented by 0. , Get the binarization mask map.

Morphological treatments can include corrosion and expansion. You can perform the erosion operation on the binarized mask first, and then perform the expansion operation to remove the noise; and then conduct the guided filtering process on the binarized mask after morphological processing to realize the edge filtering operation and obtain the main mask for edge extraction. Membrane diagram.

In the focusing method in the embodiment of the present application, morphological processing and guided filtering processing on the confidence map of the subject area can ensure that the resulting subject mask image has less or no noise, and the edges are softer, thereby improving the accuracy of centroid calculation. In turn, the accuracy of focusing and the sharpness of the subject are improved.

In an embodiment, the focusing method further includes: acquiring a backward frame image of the first preview image; using a target tracking algorithm to track the target subject in the first preview image to obtain the subject in the backward frame image; When the number of frames in the backward frame image reaches the preset number of frames, the subject in the image is re-determined.

Wherein, the backward frame image of the first preview image may be the next frame image, the next two frames of the first preview image, and so on, which is not limited thereto. The target tracking algorithm may be a TLD (Tracking Learning Detection) algorithm, a Minimum OutputSum of Squared Error MOSSE (minimum mean square error) filtering algorithm, and a Struck algorithm, etc. are not limited to this. The preset number of frames may be a threshold value of the number of frames stored in the electronic device. For example, the preset number of frames can be 10 frames, 20 frames, etc., and is not limited thereto.

Specifically, the electronic device acquires the backward frame image of the first preview image. Since the moving range of the target subject will not be too large within the preset number of frames, the electronic device uses a target tracking algorithm to track the target subject in the first preview image to obtain the subject in the backward frame. When the number of frames of the tracked backward frame image reaches the preset number of frames, the position or type of the target subject may be different from the target subject in the first preview image, and the electronic device re-determines the subject in the image.

In the focusing method in the embodiment of the present application, the backward frame image of the first preview image is obtained, and the target subject in the first preview image is tracked by the target tracking algorithm, and the subject in the backward frame image is obtained. When the number of frames of the frame image reaches the preset number of frames, re-determining the subject in the image can improve the efficiency of subject detection, thereby improving the focusing efficiency.

Fig. 6 is a schematic diagram of a subject detection process in an embodiment. As shown in Figure 6, there is a butterfly in the RGB image 602. After inputting the RGB image into the subject detection model, the subject area confidence map 604 is obtained, and then the subject area confidence map 604 is filtered and binarized to obtain the binarization. For the mask image 606, morphological processing and guided filtering are performed on the binary mask image 606 to realize edge enhancement, and the main body mask image 608 is obtained. The subject mask map 608 records the target subject obtained by image recognition and the corresponding target subject area.

In one embodiment, determining the target subject in the first preview image includes: when there are at least two subjects, according to the priority of each subject's category, the area occupied by each subject in the first preview image, At least one of the areas of each subject in the first preview image determines the target subject in the first preview image.

Among them, the category refers to the category of the subject, such as portraits, flowers, animals, landscapes and other categories.

Specifically, when there are multiple subjects, the priority of each subject's category is obtained, and the subject with the highest or the second highest priority is selected as the target subject. For example, if the priority is human face>animal>flower, and the first preview image contains a human face, an animal, and a flower at the same time, the electronic device determines that the human face is the target subject.

When there are multiple subjects, the area occupied by each subject in the image is obtained, and the subject with the largest area in the visible light image is selected as the target subject.

When there are multiple subjects, the position of each subject in the image is obtained, and the subject with the smallest distance between the position of the subject in the image and the center point of the image is selected as the target subject. For example, if the flower is at the center of the image and the face is on the other side of the center, the electronic device determines that the flower is the target subject.

When there are multiple subjects with the same priority and the highest category, the area occupied by the multiple subjects in the image is obtained, and the subject with the largest area in the image is selected as the target subject. For example, if there are two human faces in the first preview image, the electronic device selects the human face with the largest area as the target subject and calculates the centroid.

When there are multiple subjects with the same priority and the highest priority, the position of each subject in the image of the multiple subjects with the same priority and the highest is obtained, and the position of the subject in the image is selected from the center point of the image. The subject with the smallest distance between them is the target subject.

When there are multiple subjects with the same priority and the highest priority, the area occupied by each subject in the image from the multiple subjects with the same priority and the highest is obtained, and there are multiple subjects occupying the same area in the image , Obtain the positions of multiple subjects with the same area in the image, and select the subject with the smallest distance between the subject's position in the image and the center point of the image as the target subject.

When there are multiple subjects, the priority of each subject's category, the area occupied by each subject in the image, and the position of each subject in the image can be obtained, which can be filtered according to the three dimensions of priority, area and location , The order of priority, area, and location filtering can be set as needed and is not limited.

In this embodiment, when there are multiple subjects, one or at least two of the priority of the subject's category, the subject's area in the image, and the subject's position in the image are screened to determine the target subject, and the target subject can be accurately determined. .

In the focusing method in the embodiment of the present application, when there are at least two subjects, according to the priority of each subject's category, the area occupied by each subject in the first preview image, and the location of each subject in the first preview image Determine the target subject in the first preview image, which can determine the target subject when there are multiple subjects, improve the accuracy of subject detection, and thereby improve the accuracy of focusing.

In an embodiment, the focusing method further includes: when it is satisfied that the face area is greater than the threshold value of the face area and the brightness of the first preview image is greater than the brightness threshold, performing automatic focusing according to the preset focus area.

The preset focus area may be any focus area set by the electronic device, for example, it may be the center area of the image.

Specifically, when it is satisfied that the face area is greater than the face area threshold and the brightness of the first preview image is greater than the brightness threshold, it indicates that the shooting scene is in good condition, and the autofocus is performed according to the preset focus area.

In the focusing method in the embodiment of the present application, when it is satisfied that the face area is greater than the face area threshold and the brightness of the first preview image is greater than the brightness threshold, autofocus is performed according to the preset focus area without calculating the centroid coordinates, which improves the focus efficiency.

In one embodiment, FIG. 7 is a schematic flowchart of a focusing method in another embodiment. As shown in Figure 7, a focusing method includes the following operations:

Operation 702: Acquire a second preview image, where the second preview image is a forward frame image of the first preview image.

In operation 704, subject detection is performed on the second preview image to obtain an area where the reference subject is located.

Operation 706: Obtain a first preview image, and use the area where the reference subject is located as the area where the target subject of the first preview image is located.

In operation 708, the area where the target subject is located is input to the face detection module, and when a face appears in the detection result, it is determined whether the face area is greater than the face area threshold.

In operation 710, when the face area is greater than the area threshold, it is determined whether the brightness of the first preview image is greater than the brightness threshold.

In operation 712, when at least one of the face area corresponding to the target subject is less than or equal to the face area threshold and the brightness of the first preview image is less than or equal to the brightness threshold is satisfied, the centroid region of the target subject is determined.

In operation 714, auto focus is performed according to the center of mass area.

In the focusing method in the embodiment of the present application, a first preview image is acquired, and the target subject in the first preview image is determined. When the first preview image satisfies the target subject, the face area corresponding to the target subject is less than or equal to the face area threshold, and the first preview image When the brightness of is less than or equal to at least one of the brightness thresholds, determine the center of mass area of the target subject, and use the center of mass area as the focus area for focusing. It can quickly find an appropriate position for autofocus when the face is too small or the light is too dark , Improve the focusing efficiency, while avoiding focusing on the background area, improving the clarity of the subject.

In one embodiment, a focusing method includes:

Operation (a1) is to obtain a second preview image, where the second preview image is a forward frame image of the first preview image.

In operation (a2), a center weight map corresponding to the second preview image is generated.

Operation (a3), input the second preview image and the center weight map into the subject detection model to obtain the subject region confidence map.

Operation (a4) is to perform adaptive confidence threshold filtering processing on the confidence map of the main body area to obtain a binarized mask map.

In operation (a5), morphological processing and guided filtering processing are performed on the binarized mask image to obtain the main body mask image.

Operation (a6), obtain the gray value of each pixel in the area where the reference subject is located.

Operation (a7) is to determine the set of reference pixels meeting the target gray value in the area where the reference subject is located.

In operation (a8), the area corresponding to the set of reference pixels is used as the reference subject.

Operation (a9), use the reference subject as the target subject in the first preview image.

Operation (a10), when the first preview image meets a preset condition, obtain a set of pixels corresponding to the target subject, where the preset condition includes that the face area corresponding to the target subject is less than or equal to the face area threshold, and the first preview The brightness of the image is less than or equal to at least one of the brightness thresholds.

Operation (a11) is to obtain the coordinates of each pixel in the set of pixel points corresponding to the target subject.

Operation (a12), according to the coordinates of each pixel point, obtain the center of mass coordinates of the target body.

Operate (a13) to determine the area where the centroid coordinates are located.

Operation (a14), use the area where the center of mass coordinates are located as the center of mass area of the target subject.

Operation (a15), when it is satisfied that the face area is greater than the face area threshold and the brightness of the first preview image is greater than the brightness threshold, autofocus is performed according to the preset focus area.

Operate (a16) to focus based on the center of mass area.

Operation (a17), when there are at least two subjects, according to the priority of each subject's category, the area occupied by each subject in the first preview image, and the area of each subject in the first preview image At least one of determining the target subject in the first preview image.

In operation (a18), the backward frame image of the first preview image is acquired.

Operation (a19) is to use the target tracking algorithm to track the target subject in the first preview image to obtain the subject in the backward frame image.

Operation (a20), when the number of frames of the tracked backward frame image reaches the preset number of frames, the subject in the image is re-determined.

In the focusing method in this embodiment, a first preview image is acquired, and the target subject in the first preview image is determined. When the first preview image satisfies the target subject's face area less than or equal to the face area threshold, the first preview image When the brightness is less than or equal to at least one of the brightness thresholds, the centroid area of the target subject is determined, and the centroid area is used as the focus area for focusing. It can quickly find an appropriate position for autofocus when the face is too small or the light is too dark. Improve the focusing efficiency, while avoiding focusing on the background area, improving the clarity of the subject.

It should be understood that although the various operations in the flowcharts of FIGS. 2 to 7 are displayed in sequence as indicated by the arrows, these operations are not necessarily performed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these operations, and these operations can be executed in other orders. Moreover, at least part of the operations in FIGS. 2 to 7 may include multiple sub-operations or multiple stages. These sub-operations or stages are not necessarily executed at the same time, but can be executed at different times. These sub-operations or stages The execution order of is not necessarily performed sequentially, but may be performed alternately or alternately with other operations or at least a part of the sub-operations or stages of other operations.

Fig. 8 is a structural block diagram of a focusing device according to an embodiment. As shown in FIG. 8, a focusing device includes an acquiring module 802, a subject determining module 804, a centroid determining module 806, and a focusing module 808, wherein:

The obtaining module 802 is used to obtain the first preview image.

The subject determination module 804 is used to determine the target subject in the first preview image.

The centroid determination module 806 is configured to determine the centroid area of the target subject when the first preview image meets preset conditions, where the preset conditions include that the face area corresponding to the target subject is less than or equal to the face area threshold, and the first preview image The brightness of is less than or equal to at least one of the brightness thresholds.

The focusing module 808 is used for focusing according to the center of mass area.

The focusing device in this embodiment acquires the first preview image, and determines the target subject in the first preview image. When the first preview image satisfies the target subject, the face area corresponding to the target subject is less than or equal to the face area threshold and the first preview image When the brightness is less than or equal to at least one of the brightness thresholds, determine the center of mass area of the target subject, and focus according to the center of mass area, which can quickly find the appropriate position for autofocus when the face is too small or the light is too dark, and improve the focus efficiency , While avoiding focusing on the background area to improve the clarity of the subject.

In one embodiment, the centroid determination module 806 is used to obtain a set of pixel points corresponding to the target subject; obtain the coordinates of each pixel in the set of pixel points corresponding to the target subject; according to the coordinates of the target pixel corresponding to each pixel , Get the center of mass coordinates of the target subject; determine the area where the center of mass coordinates are located; take the area where the center of mass coordinates are located as the center of mass area of the target subject.

The focusing device in the embodiment of the present application obtains a set of pixel points corresponding to the target subject, obtains the coordinates of each pixel point, and obtains the centroid coordinates, and uses the area where the centroid coordinates are located as the centroid area of the target subject, which can determine the focus area and improve the target subject The clarity of the shot.

In one embodiment, the centroid determination module 806 is used to calculate the weighted average of the coordinates of the target pixel corresponding to each pixel to obtain the coordinates of the centroid of the target subject.

The focusing device in the embodiment of the present application calculates the weighted average of the coordinates of each pixel point to obtain the centroid coordinates of the target body, and can obtain all the target pixel coordinates for calculation, thereby improving the accuracy of the centroid coordinate calculation.

In one embodiment, the centroid determination module 806 is configured to determine whether the brightness of the first preview image is greater than the brightness threshold when the first preview image satisfies that the face area corresponding to the target subject is greater than the threshold value of the face area; When the brightness is less than or equal to the brightness threshold, the center of mass area of the target subject is determined.

The focusing device in the embodiment of the present application judges whether it is necessary to determine the centroid area of the target subject based on the two conditions of whether the face area is greater than the face area threshold and whether the brightness of the first preview image is greater than the brightness threshold. When the face area threshold is too dark and the light is too dark, it quickly finds the appropriate position for autofocus, which improves the focusing efficiency, while avoiding focusing on the background area, improving the focusing accuracy and the sharpness of the subject.

In one embodiment, the subject determination module 804 is used to perform subject detection on the first preview image to obtain the area where the target subject is located, where the image corresponding to the area where the target subject is located is the subject mask; to obtain each pixel in the area where the target subject is located The gray value of the point; determine the target pixel point set that meets the target gray value in the area where the target subject is located; take the area corresponding to the target pixel point set as the target subject.

The focusing device in the embodiment of the present application performs subject detection on the first preview image to obtain the area where the target subject is located, obtains the gray value of each pixel, determines the area that meets the target gray value, and obtains the target subject, which can be passed through the subject mask The map determines the target subject, thereby determining the center of mass area of the target subject, and using the center of mass area as the focus area for focusing. It can quickly find the appropriate position for autofocus when the face is too small or the light is too dark, which improves the focusing efficiency while avoiding Focus on the background area to improve the clarity of the subject.

In one embodiment, the acquiring module 802 is configured to acquire a second preview image, where the second preview image is a forward frame image of the first preview image. The subject determination module 804 is configured to perform subject detection on the second preview image to obtain a reference subject; and use the reference subject as the target subject in the first preview image.

The focusing device in the embodiment of the present application takes a second preview image, where the second preview image is a forward frame image of the first preview image, and subject detection is performed on the second preview image to obtain a reference subject. The image gap between frames is small, and the reference subject is used as the target subject in the first preview image, which can improve the efficiency of image processing and the efficiency of focusing.

In one embodiment, the subject determination module 804 is used to input the second preview image into the subject detection model to obtain the reference subject in the second preview image, where the subject detection model is based on the visible light image and center weight of the same scene in advance. The model obtained by training the image and the corresponding labeled subject, or the subject detection model is a model obtained by training in advance according to the visible light map, the center weight map, the depth map and the corresponding labeled subject of the same scene.

The focusing device in the embodiment of the present application inputs the second preview image into the subject detection model to obtain the reference subject in the second preview image, and can accurately detect the target subject in the second preview image, thereby accurately detecting the first Preview the target subject in the image to improve the accuracy of focus and the clarity of the subject.

In one embodiment, the subject determination module 804 is used to generate a center weight map corresponding to the second preview image; input the second preview image and the center weight map into the subject detection model to obtain a confidence map of the subject area; The confidence map is processed to obtain the subject mask map; the reference subject in the second preview image is determined according to the subject mask map.

The focusing device in the embodiment of the present application generates a center weight map corresponding to the second preview image, and inputs the second preview image and the center weight map into the subject detection model to obtain a confidence map of the subject area and a confidence map of the subject area Processing is performed to obtain the subject mask map, and the reference subject in the second preview image is determined according to the subject mask map, and the target subject in the second preview image can be accurately detected, thereby performing centroid calculation and improving focus accuracy.

In one embodiment, the subject determination module 804 is configured to perform adaptive confidence threshold filtering processing on the subject region confidence map to obtain a binarized mask map; perform morphological processing and guidance on the binarized mask map Filter processing to obtain the main body mask map.

The focusing device in the embodiment of the present application performs morphological processing and guided filtering processing on the confidence map of the subject area to ensure that the resulting subject mask image has less or no noise, and the edges are softer, thereby improving the accuracy of centroid calculation. In turn, the accuracy of focusing and the sharpness of the subject are improved.

In one embodiment, the acquiring module 802 is used to acquire the backward frame image of the first preview image. The focusing device also includes a tracking module. The tracking module is used to use the target tracking algorithm to track the target subject in the first preview image to obtain the subject in the backward frame image; when the number of frames of the tracked backward frame image reaches the preset frame number, re-determine the image Main body.

The focusing device in the embodiment of the present application obtains the backward frame image of the first preview image, and uses the target tracking algorithm to track the target subject in the first preview image to obtain the subject in the backward frame image. When the number of frames of the frame image reaches the preset number of frames, re-determining the subject in the image can improve the efficiency of subject detection, thereby improving the focusing efficiency.

In one embodiment, the subject determination module 804 is configured to, when there are at least two subjects, according to the priority of each subject's category, the area occupied by each subject in the first preview image, and the location of each subject in the first preview image. At least one of the areas in the preview image is used to determine the target subject in the first preview image.

In the focusing device in the embodiment of the present application, when there are at least two subjects, it is based on the priority of each subject's category, the area occupied by each subject in the first preview image, and the location of each subject in the first preview image. Determine the target subject in the first preview image, which can determine the target subject when there are multiple subjects, improve the accuracy of subject detection, and thereby improve the accuracy of focusing.

In one embodiment, the focusing module 808 is further configured to perform automatic focusing according to the preset focus area when it is satisfied that the face area is greater than the face area threshold and the brightness of the first preview image is greater than the brightness threshold.

The focusing device in the embodiment of the present application, when it is satisfied that the face area is greater than the face area threshold and the brightness of the first preview image is greater than the brightness threshold, autofocus is performed according to the preset focus area without calculating the centroid coordinates, which improves focus efficiency.

The division of the modules in the above-mentioned focusing device is only used for illustration. In other embodiments, the focusing device can be divided into different modules as required to complete all or part of the functions of the above-mentioned focusing device.

For the specific limitation of the focusing device, please refer to the above limitation of the focusing method, which will not be repeated here. Each module in the above-mentioned focusing device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

Fig. 9 is a schematic diagram of the internal structure of an electronic device in an embodiment. As shown in FIG. 9, the electronic device includes a processor and a memory connected through a system bus. Among them, the processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory may include a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by a processor to implement a focusing method provided in the following embodiments. The internal memory provides a cached operating environment for the operating system computer program in the non-volatile storage medium. The electronic device can be a mobile phone, a tablet computer, or a personal digital assistant or a wearable device.

The implementation of each module in the focusing device provided in the embodiment of the present application may be in the form of a computer program. The computer program can be run on a terminal or a server. The program module composed of the computer program can be stored in the memory of the terminal or the server. When the computer program is executed by the processor, the operation of the method described in the embodiment of the present application is realized.

The embodiment of the present application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, when the computer-executable instructions are executed by one or more processors, cause the processors to perform the operations of the focusing method.

A computer program product containing instructions that, when run on a computer, causes the computer to execute the focusing method.

Any reference to memory, storage, database, or other media used in this application may include non-volatile and/or volatile memory. Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which acts as external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The above-mentioned embodiments only express several implementation manners of the present application, and their description is relatively specific and detailed, but they should not be understood as a limitation to the scope of the present application. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can be made, and these all fall within the protection scope of this application. Therefore, the scope of protection of the patent of this application shall be subject to the appended claims.

Claims (20)

1. A focusing method, characterized in that it comprises:

   Obtain the first preview image;

   Determining the target subject in the first preview image;

   When the first preview image satisfies a preset condition, the centroid area of the target subject is determined, where the preset condition includes that the face area corresponding to the target subject is less than or equal to a face area threshold, and the first The brightness of a preview image is less than or equal to at least one of the brightness thresholds; and

   Focusing is performed according to the center of mass area.
2. The method according to claim 1, wherein the determining the center of mass region of the target subject comprises:

   Acquiring a set of pixels corresponding to the target subject;

   Acquiring the coordinates of each pixel in the set of pixel points corresponding to the target subject;

   Obtain the centroid coordinates of the target subject according to the coordinates of the respective pixel points;

   Determine the area where the centroid coordinates are located;

   The area where the centroid coordinates are located is taken as the centroid area of the target subject.
3. The method according to claim 2, wherein the obtaining the coordinates of the center of mass of the target subject according to the coordinates of the respective pixel points comprises:

   A weighted average of the coordinates of each pixel point is calculated to obtain the centroid coordinates of the target body.
4. The method according to claim 1, wherein when the first preview image satisfies a preset condition, the center of mass region of the target subject is determined, wherein the preset condition includes that the target subject corresponds to At least one of the face area of is less than or equal to the face area threshold, and the brightness of the first preview image is less than or equal to the brightness threshold, including:

   When the first preview image satisfies that the face area corresponding to the target subject is greater than the face area threshold, judging whether the brightness of the first preview image is greater than the brightness threshold;

   When the brightness of the first preview image is less than or equal to the brightness threshold, the center of mass region of the target subject is determined.
5. The method according to claim 1, wherein the determining the target subject in the first preview image comprises:

   Performing subject detection on the first preview image to obtain the area where the target subject is located, wherein the image corresponding to the area where the target subject is located is a subject mask map;

   Obtain the gray value of each pixel in the area where the target subject is located;

   Determine a set of target pixels that meet the target gray value in the area where the target subject is located;

   The area corresponding to the set of target pixel points is taken as the target subject.
6. The method according to claim 1, wherein the determining the target subject in the first preview image comprises:

   Acquiring a second preview image, where the second preview image is a forward frame image of the first preview image;

   Subject detection on the second preview image to obtain a reference subject;

   The reference subject is used as the target subject in the first preview image.
7. The method according to claim 6, wherein said performing subject detection on the second preview image to obtain a reference subject comprises:

   The second preview image is input into the subject detection model to obtain the reference subject in the second preview image, wherein the subject detection model is based on the visible light map, the center weight map and the corresponding labeled map of the same scene in advance A model obtained by training the subject, or the subject detection model is a model obtained by training in advance according to the visible light map, the center weight map, the depth map and the corresponding labeled subject of the same scene.
8. 8. The method according to claim 7, wherein said inputting said second preview image into a subject detection model to obtain a reference subject in said second preview image comprises:

   Generating a center weight map corresponding to the second preview image;

   Inputting the second preview image and the center weight map into the subject detection model to obtain the subject area confidence map;

   Processing the confidence map of the subject area to obtain a subject mask map;

   The reference subject in the second preview image is determined according to the subject mask map.
9. The method according to claim 8, wherein the processing the confidence map of the subject area to obtain a subject mask map comprises:

   Performing adaptive confidence threshold filtering processing on the confidence map of the main body area to obtain a binarized mask map;

   Morphological processing and guided filtering processing are performed on the binarized mask image to obtain a main body mask image.
10. The method according to any one of claims 1 to 9, wherein the method further comprises:

    Acquiring a backward frame image of the first preview image;

    Using a target tracking algorithm to track the target subject in the first preview image to obtain the subject in the backward frame image;

    When the number of frames of the tracked backward frame image reaches the preset number of frames, the subject in the image is re-determined.
11. The method according to any one of claims 1 to 9, wherein the determining the target subject in the first preview image comprises:

    When there are at least two subjects, according to the priority of each subject's category, the area occupied by each subject in the first preview image, and at least the area of each subject in the first preview image One is to determine the target subject in the first preview image.
12. The method according to any one of claims 1 to 9, wherein the method further comprises:

    When it is satisfied that the face area is greater than the face area threshold and the brightness of the first preview image is greater than the brightness threshold, autofocus is performed according to the preset focus area.
13. A focusing device is characterized in that it comprises:

    An obtaining module for obtaining the first preview image;

    A subject determination module, configured to determine the target subject in the first preview image;

    The centroid region determining module is configured to determine the centroid region of the target subject when the first preview image meets a preset condition, wherein the preset condition includes that the face area corresponding to the target subject is less than or equal to that of a person At least one of the face area threshold and the brightness of the first preview image is less than or equal to the brightness threshold;

    The focusing module is used for focusing according to the center of mass region.
14. The device according to claim 13, wherein the centroid determining module is used to obtain a set of pixels corresponding to the target subject;

    Acquiring the coordinates of each pixel in the set of pixel points corresponding to the target subject;

    Obtain the centroid coordinates of the target subject according to the coordinates of the respective pixel points;

    Determine the area where the centroid coordinates are located;

    The area where the centroid coordinates are located is taken as the centroid area of the target subject.
15. The device according to claim 14, wherein the centroid determination module is configured to calculate a weighted average of the coordinates of the respective pixel points to obtain the centroid coordinates of the target subject.
16. The device according to claim 13, wherein the centroid determining module is configured to determine the first preview image when the face area corresponding to the target subject is greater than the face area threshold value. Whether the brightness of a preview image is greater than the brightness threshold;

    When the brightness of the first preview image is less than or equal to the brightness threshold, the center of mass region of the target subject is determined.
17. The device according to claim 13, wherein the subject determination module is configured to perform subject detection on the first preview image to obtain the area where the target subject is located, wherein the image corresponding to the area where the target subject is located is the subject Mask map

    Obtain the gray value of each pixel in the area where the target subject is located;

    Determine a set of target pixels that meet the target gray value in the area where the target subject is located;

    The area corresponding to the set of target pixel points is taken as the target subject.
18. The device according to claim 13, wherein the subject determination module is configured to obtain a second preview image, wherein the second preview image is a forward frame image of the first preview image;

    Subject detection on the second preview image to obtain a reference subject;

    The reference subject is used as the target subject in the first preview image.
19. An electronic device, comprising a memory and a processor, and a computer program is stored in the memory. When the computer program is executed by the processor, the processor executes any one of claims 1 to 12 The steps of the focusing method.
20. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the method according to any one of claims 1 to 12 when the computer program is executed by a processor.

Images