CN112866542B

CN112866542B - Focus tracking method and apparatus, electronic device, and computer-readable storage medium

Info

Publication number: CN112866542B
Application number: CN201911101390.0A
Authority: CN
Inventors: 贾玉虎
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2019-11-12
Filing date: 2019-11-12
Publication date: 2022-08-12
Anticipated expiration: 2039-11-12
Also published as: WO2021093637A1; CN112866542A

Abstract

The application relates to a focus tracking method and device, electronic equipment and a computer readable storage medium, wherein the focus tracking method comprises the steps of acquiring a target subject detection area where a target subject is located in a preview image; when the target subject moves, determining a target subject prediction region according to the target subject detection region and the movement data of the target subject, and acquiring a detection image corresponding to the target subject prediction region; acquiring a phase difference value of the detection image by using the image sensor, wherein the phase difference value comprises a phase difference value in a first direction and a phase difference value in a second direction; the first direction and the second direction form a preset included angle; and controlling the lens to continuously focus the moving target body according to the phase difference value in the first direction and the phase difference value in the second direction. The scheme provided by the application can effectively utilize the phase difference value to perform focus tracking aiming at the scenes with horizontal textures or vertical textures, and improves the accuracy and stability of focus tracking.

Description

Focus tracking method and apparatus, electronic device, and computer-readable storage medium

Technical Field

The present application relates to the field of imaging, and in particular, to a focus tracking method and apparatus, an electronic device, and a computer-readable storage medium.

Background

With the development of electronic device technology, more and more users shoot images through electronic devices. For a moving object, in order to ensure that a captured image is clear, a camera module of the electronic device generally needs to be focused, that is, a distance between a lens and an image sensor is continuously adjusted so that a captured object is always on a focal plane. The focus tracking refers to a process of keeping focus on a photographic subject in a subsequent photographing process after the target camera focuses on the photographic subject. The conventional focusing method includes Phase Detection Auto Focus (PDAF).

Traditional phase detection automatic focusing sets up the phase detection pixel in pairs in the pixel that image sensor includes, wherein, a phase detection pixel in every phase detection pixel pair carries out the left side and shelters from, and another phase detection pixel carries out the right side and shelters from, so, just can separate into two parts about the formation of image light beam of every phase detection pixel pair of directive, through the image that two parts formation of image light beam about the contrast, can obtain the phase difference, can focus according to this phase difference after obtaining the phase difference, wherein, the phase difference refers to the difference on the formation of image position of the formation of image light that incides from different directions.

However, the above-mentioned method of setting phase detection pixel points in the image sensor to perform focus tracking has low accuracy.

Disclosure of Invention

The embodiment of the application provides a focus tracking method and device, electronic equipment and a computer readable storage medium, which can improve the accuracy of focus tracking.

A focus tracking method is applied to an electronic device, and the electronic device comprises: the image sensor comprises a plurality of pixel point groups arranged in an array, and each pixel point group comprises M x N pixel points arranged in an array; each pixel point corresponds to a photosensitive unit, wherein M and N are both natural numbers greater than or equal to 2; the method comprises the following steps:

acquiring a target body detection area where a target body in a preview image is located;

when the target subject moves, determining a target subject prediction region according to the target subject detection region and the movement data of the target subject, and acquiring a detection image corresponding to the target subject prediction region;

acquiring a phase difference value of the detection image by using the image sensor, wherein the phase difference value comprises a phase difference value in a first direction and a phase difference value in a second direction; the first direction and the second direction form a preset included angle;

And controlling the lens to continuously focus the moving target body according to the phase difference value in the first direction and the phase difference value in the second direction.

A focus tracking device is applied to electronic equipment, wherein the electronic equipment comprises an image sensor and a lens, the image sensor comprises a plurality of pixel groups arranged in an array, and each pixel group comprises M x N pixels arranged in an array; each pixel point corresponds to a photosensitive unit, wherein M and N are both natural numbers greater than or equal to 2; the method comprises the following steps:

a recognition module for acquiring a target body detection region where a target body is located in the preview image

The prediction module is used for determining a target main body prediction region according to the target main body detection region and the movement data of the target main body when the target main body moves, and acquiring a detection image corresponding to the target main body prediction region;

an obtaining module, configured to obtain a phase difference value of the detection image by using the image sensor, where the phase difference value includes a phase difference value in a first direction and a phase difference value in a second direction; the first direction and the second direction form a preset included angle;

and the focusing module is used for controlling the lens to continuously focus the moving target body according to the phase difference value in the first direction and the phase difference value in the second direction.

An electronic device comprising a memory and a processor, the memory having stored therein a computer program that, when executed by the processor, causes the processor to perform the steps of the focus tracking method as described.

A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of tracking as described.

According to the focus tracking method and device, the electronic equipment and the computer readable storage medium, the target body detection area where the target body is located in the preview image is obtained; when the target subject moves, determining a target subject prediction region according to the target subject detection region and the movement data of the target subject, and acquiring a detection image corresponding to the target subject prediction region; acquiring a phase difference value of the detection image by using the image sensor, wherein the phase difference value comprises a phase difference value in a first direction and a phase difference value in a second direction; the first direction and the second direction form a preset included angle; and controlling the lens to continuously focus the moving target body according to the phase difference value in the first direction and the phase difference value in the second direction. The scheme provided by the application can effectively utilize the phase difference value to perform focus tracking aiming at the scenes with horizontal textures or vertical textures, and improves the accuracy and stability of focus tracking.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a schematic diagram of phase detection autofocus;

fig. 2 is a schematic diagram of arranging phase detection pixels in pairs among pixels included in an image sensor;

FIG. 3 is a schematic diagram showing a partial structure of an image sensor according to an embodiment;

FIG. 4 is a schematic diagram of a pixel site in one embodiment;

FIG. 5 is a schematic diagram of an electronic device in one embodiment;

FIG. 6 is a diagram illustrating an embodiment of a filter disposed on a pixel group;

FIG. 7 is a flow diagram of a method of focus following in one embodiment;

FIG. 8 shows the steps in one embodiment: a flow chart of controlling the lens to continuously focus the moving target body by the phase difference value in the first direction and the phase difference value in the second direction;

FIG. 9 shows the steps in one embodiment: acquiring a target defocus distance flow chart according to the phase difference value in the first direction and the phase difference value in the second direction;

FIG. 10 shows the steps in one embodiment: obtaining a flow chart of a target phase difference value according to the phase difference value in the first direction and the phase difference value in the second direction;

FIG. 11 shows the steps in one embodiment: determining a flow chart of the target phase difference value according to the magnitude relation of the first confidence coefficient and the second confidence coefficient;

FIG. 12 shows the steps in one embodiment: acquiring a flow chart of a phase difference value of a detection image;

FIG. 13 shows the steps in one embodiment: a flow chart for obtaining a phase difference value in a first direction according to a phase relation corresponding to the first segmentation image and the second segmentation image and obtaining a phase difference value in a second direction according to a phase relation corresponding to the third segmentation image and the fourth segmentation image;

FIG. 14 is a block diagram showing a structure of a focusing device in one embodiment;

FIG. 15 is a block diagram of an electronic device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

It will be understood that, as used herein, the terms "first," "second," and the like may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, a first direction may be referred to as a second direction, and similarly, a second direction may be referred to as a first direction, without departing from the scope of the present application. The first direction and the second direction are both directions, but they are not the same direction.

When an image is shot, in order to ensure that the image of a moving object is shot clearly, the electronic equipment is generally required to be focused, and the focusing refers to a process of keeping focusing on a shot object in a subsequent shooting process after a target camera focuses on the shot object. For example, in the process of previewing a shot image by the electronic device, after focusing on the shot object, the focus on the shot object is still maintained in the subsequently acquired preview image, and the shot object in the acquired preview image is still clearly imaged. By "focus" is meant the process of adjusting the distance between the lens of the electronic device and the image sensor, thereby making the image sensor image sharp. Among them, Phase Detection Auto Focus (PDAF) is a common auto focus technology.

Hereinafter, the embodiment of the present application will briefly explain the principle of the PDAF technique.

Fig. 1 is a schematic diagram of a Phase Detection Auto Focus (PDAF) principle. As shown in fig. 1, M1 is a position where the image sensor is located when the electronic device is in a focusing state, where the focusing state refers to a successfully focused state. When the image sensor is located at the position M1, the imaging light rays g reflected by the object W in different directions toward the Lens converge on the image sensor, that is, the imaging light rays g reflected by the object W in different directions toward the Lens are imaged at the same position on the image sensor, and at this time, the image sensor is imaged clearly.

M2 and M3 are positions where the image sensor may be located when the electronic device is not in the in-focus state, and as shown in fig. 1, when the image sensor is located at the M2 position or the M3 position, the imaging light rays g reflected by the object W to the Lens in different directions will be imaged at different positions. Referring to fig. 1, when the image sensor is located at the position M2, the imaging light rays g reflected by the object W in different directions toward the Lens are imaged at the position a and the position B, respectively, and when the image sensor is located at the position M3, the imaging light rays g reflected by the object W in different directions toward the Lens are imaged at the position C and the position D, respectively, and at this time, the image sensor is not clear.

In the PDAF technique, the difference in the position of the image formed by the imaging light rays entering the lens from different directions in the image sensor can be obtained, for example, as shown in fig. 1, the difference between the position a and the position B, or the difference between the position C and the position D can be obtained; after acquiring the difference of the positions of images formed by imaging light rays entering the lens from different directions in the image sensor, obtaining the out-of-focus distance according to the difference and the geometric relationship between the lens and the image sensor in the camera, wherein the out-of-focus distance refers to the distance between the current position of the image sensor and the position where the image sensor is supposed to be in the in-focus state; the electronic device can focus according to the obtained defocus distance.

From this, it is understood that the calculated PD value is 0 at the time of focusing, whereas the larger the calculated value is, the farther the position of the clutch focus is indicated, and the smaller the value is, the closer the clutch focus is indicated. When PDAF focusing is adopted, the PD value is calculated, the corresponding relation between the PD value and the defocusing distance is obtained according to calibration, the defocusing distance can be obtained, and then the lens is controlled to move to reach the focusing point according to the defocusing distance, so that focusing is realized.

In the related art, some phase detection pixel points may be provided in pairs among the pixel points included in the image sensor, and as shown in fig. 2, a phase detection pixel point pair (hereinafter, referred to as a pixel point pair) a, a pixel point pair B, and a pixel point pair C may be provided in the image sensor. In each pixel point pair, one phase detection pixel point performs Left shielding (English), and the other phase detection pixel point performs Right shielding (English).

For the phase detection pixel point which is shielded on the left side, only the light beam on the right side in the imaging light beam which is emitted to the phase detection pixel point can image on the photosensitive part (namely, the part which is not shielded) of the phase detection pixel point, and for the phase detection pixel point which is shielded on the right side, only the light beam on the left side in the imaging light beam which is emitted to the phase detection pixel point can image on the photosensitive part (namely, the part which is not shielded) of the phase detection pixel point. Therefore, the imaging light beam can be divided into a left part and a right part, and the phase difference can be obtained by comparing images formed by the left part and the right part of the imaging light beam.

However, since the phase detection pixel points arranged in the image sensor are generally sparse, only a horizontal phase difference can be obtained through the phase detection pixel points, and a scene with horizontal textures cannot be calculated, and the calculated PD values are mixed up to obtain an incorrect result, for example, a scene is photographed as a horizontal line, two left and right images are obtained according to PD characteristics, but the PD values cannot be calculated.

In order to solve the problem that the phase detection autofocus cannot calculate a PD value for some horizontal texture scenes to achieve focusing, an embodiment of the present application provides an imaging component, which may be configured to detect and output a phase difference value in a first direction and a phase difference value in a second direction, and may implement focusing by using the phase difference value in the second direction for horizontal texture scenes.

In one embodiment, the present application provides an imaging assembly. The imaging assembly includes an image sensor. The image sensor may be a Metal Oxide Semiconductor (CMOS) image sensor, a Charge-coupled Device (CCD), a quantum thin film sensor, an organic sensor, or the like.

Fig. 3 is a schematic structural diagram of a part of an image sensor in one embodiment. The image sensor 300 includes a plurality of pixel groups Z arranged in an array, each pixel group Z includes a plurality of pixels D arranged in an array, and each pixel D corresponds to one photosensitive unit. The pixel points comprise M pixel points, wherein M and N are natural numbers which are larger than or equal to 2. Each pixel point D includes a plurality of sub-pixel points D arranged in an array. That is, each photosensitive unit may be composed of a plurality of photosensitive elements arranged in an array. The photosensitive element is an element capable of converting an optical signal into an electrical signal. In one embodiment, the light sensing element may be a photodiode. In this embodiment, each pixel group Z includes 4 pixels D arranged in a 2 × 2 array, and each pixel may include 4 sub-pixels D arranged in a 2 × 2 array. Each pixel point D includes 2 × 2 photodiodes, and the 2 × 2 photodiodes are arranged corresponding to the 4 sub pixel points D arranged in the 2 × 2 array. Each photodiode is used for receiving an optical signal and performing photoelectric conversion, so that the optical signal is converted into an electric signal to be output. Each pixel point D includes 4 sub-pixel points D corresponding to the same color filter, so that each pixel point D corresponds to one color channel, such as a red R channel, a green G channel, or a blue B channel.

As shown in fig. 4, taking each pixel point including a sub-pixel point 1, a sub-pixel point 2, a sub-pixel point 3, and a sub-pixel point 4 as an example, the sub-pixel point 1 and the sub-pixel point 2 may be synthesized, the sub-pixel point 3 and the sub-pixel point 4 are synthesized to form a PD pixel pair in the up-and-down direction, and a horizontal edge is detected to obtain a phase difference value in the second direction, that is, a PD value (phase difference value) in the vertical direction; the sub-pixel point 1 and the sub-pixel point 3 are synthesized, and the sub-pixel point 2 and the sub-pixel point 4 are synthesized to form a PD pixel pair in the left and right directions, so that the vertical edge can be detected, and the phase difference value in the first direction, that is, the PD value (phase difference value) in the horizontal direction is obtained.

Fig. 5 is a schematic structural diagram of an electronic device in one embodiment. As shown in fig. 5, the electronic device includes a microlens 50, a filter 52, and an imaging component 54. The microlens 50, the filter 52 and the imaging component 54 are sequentially located on the incident light path, i.e. the microlens 50 is disposed on the filter 52, and the filter 52 is disposed on the imaging component 54.

The filter 52 may include three types of red, green and blue, which only transmit the light with the wavelengths corresponding to the red, green and blue colors, respectively. A filter 52 is disposed on one pixel.

The imaging assembly 54 includes the image sensor of fig. 3.

The lens 50 is used to receive incident light and transmit the incident light to the filter 52. The filter 52 smoothes incident light, and then the smoothed light is incident on the imaging element 54 on a pixel basis.

The light sensing unit in the image sensor converts light incident from the optical filter 52 into a charge signal by a photoelectric effect, and generates a pixel signal in accordance with the charge signal. The charge signal corresponds to the received light intensity.

Fig. 6 is a schematic diagram illustrating a filter disposed on a pixel group according to an embodiment. The pixel point group Z comprises 4 pixel points D arranged in an array arrangement manner of two rows and two columns, wherein color channels of the pixel points in the first row and the first column are green, that is, the optical filters arranged on the pixel points in the first row and the first column are green optical filters; the color channel of the pixel points in the first row and the second column is red, that is, the optical filter arranged on the pixel points in the first row and the second column is a red optical filter; the color channel of the pixel points in the second row and the first column is blue, that is, the optical filter arranged on the pixel points in the second row and the first column is a blue optical filter; the color channel of the pixel points in the second row and the second column is green, that is, the optical filter arranged on the pixel points in the second row and the second column is a green optical filter.

FIG. 7 is a flow diagram of a method of focus tracking in one embodiment. As shown in fig. 7, the focus tracking method includes steps 702 to 708.

Step 702, a target body detection area where a target body in the preview image is located is obtained.

The preview image refers to an image obtained after the camera is focused. The subject refers to various subjects, such as human, flower, cat, dog, cow, blue sky, white cloud, background, etc. The target subject refers to a desired subject, and can be selected as desired. The target subject detection region may be a region outlined based on the outline of the target subject, or may be a frame shape such as a rectangular frame or a circular frame surrounding the target subject. The shape of the target body region is not limited, and the target body region may include a large part of the target body.

Specifically, the image capturing device of the electronic device may be used to perform focusing to obtain a preview image, and perform subject detection on the preview image to obtain a target subject detection area including a target subject. Focusing refers to a process of imaging a photographed object clearly by adjusting a focal length. Where focal length refers to the distance from the optical center of a lens in a camera to the focal point of light collection. Subject detection (subject detection) refers to automatically processing regions of interest while selectively ignoring regions of no interest when facing a scene. The region of interest in this embodiment is referred to as a target subject detection region. In one embodiment, the subject detection model is obtained by collecting a large amount of training data in advance, and inputting the training data into the subject detection model including the initial network weight for training. The subject detection model may be trained to recognize and detect various subjects, such as people, flowers, cats, dogs, backgrounds, etc.

Step 704, when the target subject moves, determining a target subject prediction region according to the target subject detection region and the movement data of the target subject, and acquiring a detection image corresponding to the target subject prediction region.

The moving data of the target body is data such as the moving speed, the moving direction and the moving track of the target body. The movement data of the target subject can be obtained by using a trained neural network model. The target subject prediction region is a region where a next time sequence target subject is predicted. The detection image refers to an image including the target subject captured by the image capturing apparatus with the target subject prediction area as the in-focus area.

Specifically, when the target body is a movable body, the movement of the target body can be detected, and the tracking can be automatically performed. And predicting a target main body prediction area corresponding to the next time sequence according to the motion data of the target main body and the current target main body detection area. For example, a first image and a second image may be input into a trained neural network model, the first image and the second image include the same target subject, and the trained neural network model may predict movement data of a next time sequence target subject and a target subject prediction region according to different movement data of the target subject included in the first image and the second image; it is also possible to input a first image including a moving object, the first image including: the current time sequence corresponds to the motion data of the target subject detection area and the target subject, the corresponding network model can output a second image, and the second image carries the motion data of the target subject prediction area and the target subject corresponding to the next time sequence. Focusing the target main body prediction area, and acquiring a detection image by the electronic equipment according to pixel information of pixel points included in each pixel point group in the image sensor. The image sensor comprises a sub-pixel which is a photosensitive element capable of converting optical signals into electric signals, the intensity of the optical signals received by the sub-pixel can be obtained according to the electric signals output by the sub-pixel, and the pixel information of the sub-pixel can be obtained according to the intensity of the optical signals received by the sub-pixel.

Step 706, obtaining a phase difference value of the detected image, where the phase difference value includes a phase difference value in a first direction and a phase difference value in a second direction. The first direction and the second direction form a preset included angle.

Specifically, a phase difference value of the detection image is obtained, and the phase difference value includes a phase difference value in a first direction and a phase difference value in a second direction. The first direction and the second direction form a preset included angle

The angle can be 30 degrees, 40 degrees, 45 degrees, 60 degrees and the like.

It may also be 90 °, i.e. when the phase difference value in the first direction refers to the phase difference value in the horizontal direction, the phase difference value in the second direction refers to the phase difference value in the vertical direction.

And step 708, controlling the lens to continuously focus on the moving target body according to the phase difference value in the first direction and the phase difference value in the second direction.

Specifically, a target phase difference value in a mapping relation with the defocus distance value can be obtained according to the phase difference value in the first direction and the phase difference value in the second direction, and then the phase difference value in the first direction, the phase difference value in the second direction and the target phase difference value are obtained by calibrating the corresponding relation between the target phase difference value and the target defocus distance value, so that the target defocus distance can be obtained. And controlling the lens to continuously focus the moving target body according to the target defocusing distance value. The focusing refers to a process of keeping the target subject focused in the subsequent shooting process after the lens focuses the target subject, and the target subject in the acquired detection image keeps clearly imaged.

The focus tracking method provided by the embodiment acquires a target subject detection area where a target subject is located in a preview image. When the target subject moves, determining a target subject prediction region according to the target subject detection region and the movement data of the target subject, and acquiring a detection image corresponding to the target subject prediction region. And acquiring a phase difference value of the detection image, wherein the phase difference value comprises a phase difference value in a first direction and a phase difference value in a second direction. The first direction and the second direction form a preset included angle. And controlling the lens to continuously focus the moving target body according to the phase difference value in the first direction and the phase difference value in the second direction. The scheme provided by the application can effectively utilize the phase difference value to perform focus tracking aiming at the scenes with horizontal textures or vertical textures, and improves the accuracy and stability of focus tracking.

In one embodiment, the step of obtaining a detection image corresponding to the prediction region of the target subject includes: and controlling the lens to move so as to enable the focus to be aligned with the center of the target body prediction region and acquiring a detection image corresponding to the target body prediction region.

Specifically, the lens is controlled to move so as to focus on the center of the target subject prediction region and acquire a detection image corresponding to the target subject prediction region, wherein the detection image includes the target subject. For example, the process is as follows: when the target main body prediction area is rectangular, focusing on the center of the rectangle; when the target main body prediction area is circular, the focus is aligned to the round point; and when the target subject prediction region is an irregular figure, focusing on the gravity center of the target subject prediction region.

In one embodiment, determining the target subject prediction region according to the target subject detection region and the motion data of the target subject includes: and inputting a first image to the prediction network model, wherein the first image carries the information of the target subject detection area and the motion data of the target subject. And acquiring a second image output by the prediction network model, wherein the second image is marked with a target main body prediction region.

Specifically, the predictive network model refers to a trained network model, and a first image including a moving object is input, and the first image includes: the current time sequence corresponds to the target subject detection area and the motion data of the target subject. The network model can output a second image, the second image is marked with a target subject prediction area corresponding to the next time sequence, and motion data of the target subject can be acquired according to the second image. In one embodiment, the predictive network model is a network model established based on a recurrent neural network algorithm. The recurrent neural network has memory, parameter sharing and graph completion (training completion), and thus has certain advantages in learning the nonlinear characteristics of a sequence. The recurrent neural network has applications in Natural Language Processing (NLP) fields such as speech recognition, Language modeling, machine translation, and the like, and is also used for various time series predictions.

In one embodiment, as shown in fig. 8, the step of controlling the lens to continuously focus on the moving target object by the phase difference value in the first direction and the phase difference value in the second direction includes: step 802 and step 804.

And step 802, acquiring the defocusing distance of the target according to the phase difference value in the first direction and the phase difference value in the second direction.

Specifically, the target phase difference value is determined according to the magnitude relationship between the phase difference value in the first direction and the phase difference value in the second direction or the carried confidence information. And the target phase difference value and the target defocus distance have a mapping relation, and the target phase difference value is input into a function for representing the mapping relation, so that the target defocus distance can be obtained.

And step 804, controlling the lens of the electronic equipment to move according to the defocusing distance of the target, and continuously focusing the moving target body.

Specifically, the target defocus distance refers to a distance between a current position of the image sensor and a position where the image sensor should be in an in-focus state; the electronic equipment can control the lens to move to the in-focus position for tracking according to the obtained target out-of-focus distance. In one embodiment, the focus tracking method may further include: and generating a depth value according to the target defocus distance value. The target defocus distance value can calculate an image distance in an in-focus state, and an object distance can be obtained according to the image distance and the focal distance, wherein the object distance is a depth value.

In one embodiment, as shown in fig. 9, the step of obtaining the defocus distance of the target according to the phase difference value in the first direction and the phase difference value in the second direction includes: step 902 and step 904.

And step 902, obtaining a target phase difference value according to the phase difference value in the first direction and the phase difference value in the second direction.

Specifically, the target phase difference value can be determined according to the magnitude relationship or the confidence information between the phase difference value in the first direction and the phase difference value in the second direction, and the target defocus distance can be obtained according to the target phase difference value. For example, when the phase difference value in the first direction and the phase difference value in the second direction both exist, the confidence of the phase difference value in the first direction and the confidence of the phase difference value in the second direction may be obtained, one phase difference value is selected as the target phase difference value from the phase difference value in the first direction and the phase difference value in the second direction, and then the corresponding target defocus distance value is obtained from the mapping relationship between the phase difference value and the defocus distance value according to the determined target phase difference value.

And 904, acquiring the defocusing distance of the target according to the target phase difference value.

Specifically, a mapping relationship exists between the target phase difference value and the target defocus distance, and the target phase difference value is input into a function for representing the mapping relationship, so that the target defocus distance can be obtained.

In one embodiment, obtaining the defocus distance of the target according to the phase difference of the target includes: and calculating the target defocus distance according to the calibrated defocus function and the target phase difference value, wherein the calibrated defocus function is used for representing the relation between the target phase difference value and the target defocus distance.

Specifically, the correspondence between the target defocus distance value and the target phase difference value is as follows: the Defocus Coefficient (DCC) may be obtained by calibration, and the PD is a target phase difference value. And calculating to obtain the target defocus distance according to the calibrated defocus function and the target phase difference value. The calibration process of the corresponding relation between the target phase difference value and the defocus distance value comprises the following steps: dividing the effective focusing stroke of the camera module into N (N is more than or equal to 3) equal parts, namely (near-focus DAC-far-focus DAC)/N, so as to cover the focusing range of the motor; focusing is carried out at each focusing DAC (DAC can be 0-1023), and the phase difference of the current focusing DAC position is recorded; after the motor focusing stroke is finished, a group of N focusing DACs are compared with the obtained PD value; and generating N similar ratios K, and fitting the two-dimensional data consisting of the DAC and the PD to obtain a straight line with the slope K.

In one embodiment, as shown in fig. 10, the step of obtaining the target phase difference value according to the phase difference value in the first direction and the phase difference value in the second direction includes: step 1002 to step 1006.

Step 1002, a first confidence corresponding to the phase difference value in the first direction is obtained.

Specifically, the phase difference value in the first direction carries information of a first confidence coefficient, and the first confidence coefficient is used for representing the accuracy of the phase difference value in the first direction.

And 1004, acquiring a second confidence corresponding to the phase difference value in the second direction.

Specifically, the phase difference value in the second direction carries information of a second confidence coefficient, and the second confidence coefficient is used for representing the accuracy of the phase difference value in the second direction.

And step 1006, determining a target phase difference value according to the magnitude relation between the first confidence coefficient and the second confidence coefficient.

Specifically, the target phase difference value is determined according to the magnitude relationship between the first confidence coefficient and the second confidence coefficient. For example, when the confidence of the phase difference value in the first direction is greater than the confidence of the phase difference value in the second direction, identifying that the accuracy of the phase difference value in the first direction is higher than the accuracy of the phase difference value in the second direction, the phase difference value in the first direction may be selected as the target phase difference value; when the confidence of the phase difference value in the first direction is less than the confidence of the phase difference value in the second direction, identifying that the accuracy of the phase difference value in the first direction is lower than the accuracy of the phase difference value in the second direction, and selecting the phase difference value in the second direction as a target phase difference value; when the confidence of the phase difference value in the first direction is equal to the confidence of the phase difference value in the second direction, it is identified that the accuracy of the phase difference value in the first direction is equal to the accuracy of the phase difference value in the second direction, and a larger value of the phase difference values may be selected from the phase difference value in the first direction and the phase difference value in the second direction as a target phase difference value.

In one embodiment, as shown in fig. 11, the step of determining the target phase difference value according to the magnitude relationship between the first confidence level and the second confidence level includes: 1102 to 1106.

Step 1102, when the first confidence degree is greater than the second confidence degree, taking the phase difference value of the first direction corresponding to the first confidence degree as a target phase difference value.

Specifically, when the confidence of the phase difference value in the first direction is greater than the confidence of the phase difference value in the second direction, the phase difference value in the first direction is selected, a corresponding defocus distance value is obtained according to the phase difference value in the first direction, and the moving direction is determined to be the horizontal direction.

And 1104, when the second confidence coefficient is greater than the first confidence coefficient, taking the phase difference value in the second direction corresponding to the second confidence coefficient as a target phase difference value.

Specifically, when the confidence of the phase difference value in the first direction is smaller than the confidence of the phase difference value in the second direction, the phase difference value in the second direction is selected, a corresponding defocus distance value is obtained according to the phase difference value in the second direction, and the moving direction is determined to be the vertical direction.

And step 1106, when the first confidence coefficient is equal to the second confidence coefficient, taking both the first direction phase difference and the second direction phase difference as target phase difference values.

Specifically, when the confidence of the phase difference value in the first direction is equal to the confidence of the phase difference value in the second direction, the defocus distance value in the horizontal direction may be determined according to the phase difference value in the first direction, and the defocus distance value in the vertical direction may be determined according to the phase difference value in the second direction, and the defocus distance value in the horizontal direction may be moved first and then the defocus distance value in the vertical direction, or the defocus distance value in the vertical direction may be moved first and then the defocus distance value in the horizontal direction. It should be noted that, for a scene with horizontal texture, since the PD pixel pair in the horizontal direction cannot obtain the phase difference value in the first direction, the PD pixel pair in the vertical direction may be compared to the PD pixel pair in the vertical direction to calculate the phase difference value in the second direction in the vertical direction, and then the lens is controlled to move according to the phase difference value in the second direction to realize focusing; for a scene with vertical texture, because the phase difference value in the second direction cannot be obtained by the PD pixel pair in the vertical direction, the phase difference value in the first direction in the horizontal direction can be compared with the phase difference value in the horizontal direction, the defocusing distance value is calculated according to the phase difference value in the first direction, and then the lens is controlled to move according to the defocusing distance value in the horizontal direction to realize focusing.

In one embodiment, as shown in fig. 12, the step of acquiring the phase difference value of the detection image includes: step 1202 and step 1204.

Step 1202, the detection image is segmented into a first segmentation image and a second segmentation image according to a first direction. And obtaining a phase difference value in the first direction according to the corresponding phase relation of the first segmentation image and the second segmentation image.

Specifically, the electronic device may perform segmentation processing on the target image in the row direction (x-axis direction in the image coordinate system), and each segmentation line of the segmentation processing is perpendicular to the row direction during the segmentation processing of the target image in the row direction. The first and second sliced images obtained by slicing the target image in the row direction may be referred to as a left image and a right image, respectively. And acquiring a phase difference value in the first direction according to the phase difference of the 'matched pixels' in the left image and the right image.

And 1204, segmenting the detection image into a third segmented image and a fourth segmented image according to the second direction. And obtaining a phase difference value in the second direction according to the corresponding phase relation of the third segmentation image and the fourth segmentation image.

Specifically, the electronic device may perform a segmentation process on the target image in the column direction (y-axis direction in the image coordinate system), and each segmentation line of the segmentation process is perpendicular to the column direction during the segmentation process on the target image in the column direction. The first and second sliced images obtained by slicing the target image in the column direction may be referred to as an upper image and a lower image, respectively.

In one embodiment, the first direction is a row direction and the second direction is a column direction. The step of segmenting the detection image into a first segmentation image and a second segmentation image according to a first direction includes: and carrying out segmentation processing on the detection image according to the first direction to obtain a plurality of image areas, wherein each image area comprises a line of pixels in the detection image. A plurality of first sliced image regions and a plurality of second sliced image regions are obtained from the plurality of image regions, the first sliced image regions including pixels of even lines in the test image, and the second sliced image regions including pixels of odd lines in the test image. And splicing the plurality of first segmentation image regions into a first segmentation image, and forming a second segmentation image by using the plurality of second segmentation image regions.

Specifically, the first direction is a line direction, and the detection image is segmented according to the first direction, so that a plurality of image areas can be obtained, wherein each image area comprises a line of pixels in the detection image. A plurality of first sliced image regions and a plurality of second sliced image regions are obtained from the plurality of image regions, the first sliced image regions refer to pixels of even lines in the inspection image, and the second sliced image regions refer to pixels of odd lines in the inspection image. And sequentially splicing the plurality of first segmentation image areas according to the positions in the detection image to obtain a first segmentation image, and sequentially splicing the plurality of second segmentation image areas according to the positions in the detection image to obtain a second segmentation image.

The step of segmenting the detection image into a third segmentation image and a fourth segmentation image according to a second direction comprises the following steps: and carrying out segmentation processing on the detection image according to a second direction to obtain a plurality of image areas, wherein each image area comprises a column of pixels in the detection image. And acquiring a plurality of third segmentation image areas and a plurality of fourth segmentation image areas from the plurality of image areas, wherein the third segmentation image areas comprise pixels of even columns in the detection image, and the fourth segmentation image areas comprise pixels of odd columns in the detection image. And splicing the plurality of third segmentation image regions into a third segmentation image, and forming a fourth segmentation image by using the plurality of fourth segmentation image regions.

Specifically, the second direction is a column direction, and the detection image is segmented according to the first column direction, so that a plurality of image areas can be obtained, wherein each image area comprises a column of pixels in the detection image. And acquiring a plurality of third segmentation image areas and a plurality of fourth segmentation image areas from the plurality of image areas, wherein the third segmentation image areas refer to pixels of even columns in the detection image, and the fourth segmentation image areas refer to pixels of odd columns in the detection image. And sequentially splicing the plurality of third segmented image regions according to the positions in the detection image to obtain a third segmented image, and sequentially splicing the plurality of fourth segmented image regions according to the positions in the detection image to obtain a fourth segmented image.

In one embodiment, as shown in fig. 13, the step of obtaining a phase difference value in a first direction according to a phase relationship corresponding to the first sliced image and the second sliced image and obtaining a phase difference value in a second direction according to a phase relationship corresponding to the third sliced image and the fourth sliced image includes: step 1302 and step 1304.

Step 1302, determining a phase difference value of the matched pixels according to the position difference of the matched pixels in the first segmentation image and the second segmentation image. And determining the phase difference value of the first direction according to the phase difference values of the pixels matched with each other.

Specifically, when the first sliced image includes pixels in even-numbered lines, the second sliced image includes pixels in odd-numbered lines, and the pixel a in the first sliced image and the pixel b in the second sliced image are matched with each other, the phase difference value in the first direction can be determined according to the phase difference between the matched pixel a and pixel b.

And 1304, determining the phase difference value of the matched pixels according to the position difference of the matched pixels in the third segmentation image and the fourth segmentation image. And determining the phase difference value of the second direction according to the phase difference values of the pixels matched with each other.

Specifically, when the first sliced image includes pixels in even columns, the second sliced image includes pixels in odd columns, and the pixel a in the first sliced image and the pixel b in the second sliced image are matched with each other, the phase difference value in the second direction can be determined according to the phase difference between the matched pixel a and pixel b.

Here, "pixels matched with each other" means that pixel matrices composed of the pixels themselves and their surrounding pixels are similar to each other. For example, pixel a and its surrounding pixels in the first cut-out image form a pixel matrix with 3 rows and 3 columns, and the pixel values of the pixel matrix are:

2 15 70

1 35 60

0 100 1

the pixel b and its surrounding pixels in the second sliced image also form a pixel matrix with 3 rows and 3 columns, and the pixel value of the pixel matrix is:

1 15 70

1 36 60

0 100 2

as can be seen from the above, the two matrices are similar, and pixel a and pixel b can be considered to match each other. The pixel matrixes are judged to be similar in many ways, usually, the pixel values of each corresponding pixel in two pixel matrixes are subtracted, the absolute values of the obtained difference values are added, and the result of the addition is used for judging whether the pixel matrixes are similar, that is, if the result of the addition is smaller than a preset threshold, the pixel matrixes are considered to be similar, otherwise, the pixel matrixes are considered to be dissimilar.

For example, for the two pixel matrices of 3 rows and 3 columns, 1 and 2 are subtracted, 15 and 15 are subtracted, 70 and 70 are subtracted, … … are added, and the absolute values of the obtained differences are added to obtain an addition result of 3, and if the addition result of 3 is smaller than a preset threshold, the two pixel matrices of 3 rows and 3 columns are considered to be similar.

Another way to judge whether the pixel matrixes are similar is to extract the edge features of the pixel matrixes by using a sobel convolution kernel calculation way or a high laplacian calculation way, and the like, and judge whether the pixel matrixes are similar through the edge features.

In the present embodiment, "the positional difference of the pixels that match each other" refers to the difference between the position of the pixel located in the first sliced image and the position of the pixel located in the second sliced image among the pixels that match each other. As exemplified above, the positional difference of the pixel a and the pixel b that match each other refers to the difference in the position of the pixel a in the first sliced image and the position of the pixel b in the second sliced image.

The pixels matched with each other respectively correspond to different images formed in the image sensor by imaging light rays entering the lens from different directions. For example, a pixel a in the first sliced image and a pixel B in the second sliced image match each other, where the pixel a may correspond to the image formed at the a position in fig. 1 and the pixel B may correspond to the image formed at the B position in fig. 1.

Since the matched pixels respectively correspond to different images formed by imaging light rays entering the lens from different directions in the image sensor, the phase difference of the matched pixels can be determined according to the position difference of the matched pixels.

The brightness values of the pixels in the pixel group obtain a target image, the target image is divided into two segmentation images, and then the phase difference values of the pixels matched with each other can be rapidly determined through pixel matching, meanwhile, the phase difference values are rich, the phase difference value accuracy can be improved, and the focusing accuracy and stability are improved.

It should be understood that although the various steps in the flowcharts of fig. 7-13 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not limited to being performed in the exact order illustrated and, unless explicitly stated herein, may be performed in other orders. Moreover, at least some of the steps in fig. 7-13 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.

An embodiment of the present application provides a focus tracking apparatus applied to an electronic device, as shown in fig. 14, the focus tracking apparatus includes: a recognition module 1402, a prediction module 1404, an acquisition module 1406, and a focus tracking module 1408.

The identifying module 1402 is configured to obtain a target subject detection area where the target subject in the preview image is located.

Specifically, the recognition module 1402 is configured to perform focusing by using the electronic device to obtain a preview image, and perform subject detection on the preview image to obtain a target subject detection area including a target subject. Focusing refers to a process of imaging a photographed object clearly by adjusting a focal length. Where focal length refers to the distance from the optical center of a lens in a camera to the focal point of light collection. Subject detection (subject detection) refers to automatically processing regions of interest while selectively ignoring regions of no interest when facing a scene. The region of interest in this embodiment is referred to as a target subject detection region. In one embodiment, the subject detection model is obtained by collecting a large amount of training data in advance, and inputting the training data into the subject detection model including the initial network weight for training. The subject detection model may be trained to recognize and detect various subjects, such as people, flowers, cats, dogs, backgrounds, etc.

The predicting module 1404 is configured to, when the target subject moves, determine a target subject prediction region according to the target subject detection region and the movement data of the target subject, and acquire a detection image corresponding to the target subject prediction region.

The moving data of the target body is data such as the moving speed, the moving direction and the moving track of the target body. The movement data of the target subject can be obtained by using a trained neural network model. The target subject prediction region is a region where a target subject of the next time sequence is predicted. The detection image refers to an image including the target subject acquired by the electronic device with the target subject prediction area as the in-focus area.

Specifically, the prediction module 1404 is configured to detect that the target object moves and automatically perform focus tracking when the target object is a movable object. And predicting a target main body prediction area corresponding to the next time sequence according to the motion data of the target main body and the current target main body detection area. For example, a first image and a second image may be input into a trained neural network model, the first image and the second image include the same target subject, and the trained neural network model may predict movement data of a next time sequence target subject and a target subject prediction region according to different movement data of the target subject included in the first image and the second image; it is also possible to input a first image including a moving object, the first image including: the current time sequence corresponds to the motion data of the target subject detection area and the target subject, the corresponding network model can output a second image, and the second image carries the motion data of the target subject prediction area and the target subject corresponding to the next time sequence. Focusing the target main body prediction area, and acquiring a detection image by the electronic equipment according to pixel information of pixel points included in each pixel point group in the image sensor. The image sensor comprises a sub-pixel which is a photosensitive element capable of converting optical signals into electric signals, the intensity of the optical signals received by the sub-pixel can be obtained according to the electric signals output by the sub-pixel, and the pixel information of the sub-pixel can be obtained according to the intensity of the optical signals received by the sub-pixel.

The obtaining module 1406 is configured to obtain a phase difference value of the detection image, where the phase difference value includes a phase difference value in a first direction and a phase difference value in a second direction. The first direction and the second direction form a preset included angle.

Specifically, the obtaining module 1406 is configured to obtain a phase difference value of the detected image, where the phase difference value includes a phase difference value in a first direction and a phase difference value in a second direction. The first direction and the second direction form a preset included angle

The angle can be 30 degrees, 40 degrees, 45 degrees, 60 degrees and the like.

And a focusing module 1408, configured to control the lens to focus on the moving target object continuously according to the phase difference value in the first direction and the phase difference value in the second direction.

Specifically, the tracking module 1408 is configured to obtain a target phase difference value in a mapping relationship with the defocus distance value according to the phase difference value in the first direction and the phase difference value in the second direction, and obtain the phase difference value in the first direction, the phase difference value in the second direction, and the target phase difference value as a result of obtaining the correspondence between the target phase difference value and the target defocus distance value through calibration, so as to obtain the target defocus distance. And controlling the lens to continuously focus the moving target body according to the target defocusing distance value. The focusing refers to a process of keeping the target subject focused in the subsequent shooting process after the lens focuses the target subject, and the target subject in the acquired detection image keeps clearly imaged.

The focus tracking apparatus provided by the present embodiment acquires a target subject detection region in which a target subject is located in a preview image. When the target subject moves, determining a target subject prediction region according to the target subject detection region and the movement data of the target subject, and acquiring a detection image corresponding to the target subject prediction region. And acquiring a phase difference value of the detection image, wherein the phase difference value comprises a phase difference value in a first direction and a phase difference value in a second direction. The first direction and the second direction form a preset included angle. And controlling the lens to continuously focus the moving target body according to the phase difference value in the first direction and the phase difference value in the second direction. The scheme provided by the application can effectively utilize the phase difference value to perform focus tracking aiming at the scenes with horizontal textures or vertical textures, and improves the accuracy and stability of focus tracking.

In one embodiment, the prediction module is to control lens movement to bring focus into the center of the target subject prediction region. And acquiring a detection image corresponding to the target subject prediction region based on the focus.

In one embodiment, the prediction module is configured to input a first image to the prediction network model, the first image carrying information of the target subject detection area and motion data of the target subject. And acquiring a second image output by the prediction network model, wherein the second image carries the information of the target main body prediction region.

In one embodiment, the obtaining module is configured to obtain the defocus distance of the target according to the phase difference value in the first direction and the phase difference value in the second direction; and controlling the lens of the electronic equipment to move according to the target defocus distance to continuously focus the moving target body.

In one embodiment, the obtaining module is configured to obtain a target phase difference value according to the phase difference value in the first direction and the phase difference value in the second direction; and acquiring the defocusing distance of the target according to the target phase difference value.

In one embodiment, the obtaining module is configured to calculate the target defocus distance according to the calibrated defocus function and the target phase difference, and the calibrated defocus function is used to characterize a relationship between the target phase difference and the target defocus distance.

In one embodiment, the obtaining module is configured to obtain a first confidence degree corresponding to the phase difference value in the first direction; acquiring a second confidence corresponding to the phase difference value in the second direction; and determining the target phase difference value according to the magnitude relation of the first confidence coefficient and the second confidence coefficient.

In one embodiment, the obtaining module is configured to, when the first confidence degree is greater than the second confidence degree, take a phase difference value in a first direction corresponding to the first confidence degree as a target phase difference value; when the second confidence coefficient is larger than the first confidence coefficient, taking the phase difference value in the second direction corresponding to the second confidence coefficient as a target phase difference value; and when the first confidence coefficient is equal to the second confidence coefficient, both the first direction phase difference and the second direction phase difference are taken as target phase difference values.

In one embodiment, the acquisition module is configured to segment the inspection image into a first segmented image and a second segmented image according to a first direction. Obtaining a phase difference value in a first direction according to the corresponding phase relation of the first segmentation image and the second segmentation image; and segmenting the detection image into a third segmentation image and a fourth segmentation image according to the second direction. And obtaining a phase difference value in the second direction according to the corresponding phase relation of the third segmentation image and the fourth segmentation image.

In one embodiment, the obtaining module is configured to perform segmentation processing on a detection image according to a first direction to obtain a plurality of image regions, where each image region includes a row of pixels in the detection image, obtain a plurality of first segmentation image regions and a plurality of second segmentation image regions from the plurality of image regions, where the first segmentation image regions include pixels in an even row in the detection image, and the second segmentation image regions include pixels in an odd row in the detection image, splice the plurality of first segmentation image regions into a first segmentation image, and form a second segmentation image by using the plurality of second segmentation image regions; performing segmentation processing on the detection image according to a second direction to obtain a plurality of image areas, wherein each image area comprises a row of pixels in the detection image; acquiring a plurality of third segmentation image areas and a plurality of fourth segmentation image areas from the plurality of image areas, wherein the third segmentation image areas comprise pixels of even columns in the detection image, and the fourth segmentation image areas comprise pixels of odd columns in the detection image; and splicing the plurality of third segmentation image regions into a third segmentation image, and forming a fourth segmentation image by using the plurality of fourth segmentation image regions.

In one embodiment, the obtaining module is configured to determine the phase difference value of the matched pixels according to the position difference of the matched pixels in the first segmentation image and the second segmentation image. Determining a phase difference value in a first direction according to the phase difference values of the mutually matched pixels; and determining the phase difference value of the mutually matched pixels according to the position difference of the mutually matched pixels in the third segmentation image and the fourth segmentation image. And determining the phase difference value of the second direction according to the phase difference values of the pixels matched with each other.

The division of each module in the above-mentioned focus-following device is only used for illustration, in other embodiments, the focus-following device can be divided into different modules as required to complete all or part of the functions of the above-mentioned focus-following device.

For specific definition of the focus tracking device, reference may be made to the definition of the focus tracking method above, and details are not repeated here. The modules in the above-mentioned focus tracking device can be realized by software, hardware and their combination in whole or in part. The modules can be embedded in a hardware form or independent of a processor in the electronic device, or can be stored in a memory in the electronic device in a software form, so that the processor can call and execute operations corresponding to the modules.

An electronic device comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is enabled to execute steps such as a focus tracking method.

In one embodiment of the present application, an electronic device is provided, which may be an electronic device having a digital image capturing function, for example, a smart phone, a tablet computer, a camera, a video camera, or the like. The internal structure thereof may be as shown in fig. 15. The electronic device includes a processor and a memory connected by a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic equipment comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium may store an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The computer program is executed by a processor to implement a focus tracking method provided by the embodiment of the application.

In addition, although not shown in fig. 15, the electronic device may further include a lens and an image sensor, wherein the lens may be composed of a set of lenses, and the image sensor may be a Metal Oxide semiconductor (CMOS) image sensor, a Charge-coupled device (CCD), a quantum thin film sensor, an organic sensor, or the like. The image sensor can be connected with a processor through a bus, and the processor can realize the focus tracking method provided by the embodiment of the application through signals output by the image sensor.

Those skilled in the art will appreciate that the structure shown in fig. 15 is a block diagram of only a portion of the structure relevant to the present application, and does not constitute a limitation on the electronic device to which the present application is applied, and a particular electronic device may include more or less components than those shown in the drawings, or combine certain components, or have a different arrangement of components.

In one embodiment of the present application, an electronic device is provided, which may be an electronic device, and includes a memory and a processor, where the memory stores a computer program, and the processor executes the computer program to implement the following steps:

and acquiring a target body detection area where the target body in the preview image is located. When the target subject moves, determining a target subject prediction region according to the target subject detection region and the movement data of the target subject, and acquiring a detection image corresponding to the target subject prediction region. And acquiring a phase difference value of the detection image, wherein the phase difference value comprises a phase difference value in a first direction and a phase difference value in a second direction. The first direction and the second direction form a preset included angle. And controlling the lens to continuously focus the moving target body according to the phase difference value in the first direction and the phase difference value in the second direction.

The implementation principle and technical effect of the computer-readable storage medium provided in this embodiment are similar to those of the above method embodiments, and are not described herein again.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. The focus tracking method is applied to electronic equipment, wherein the electronic equipment comprises an image sensor and a lens, the image sensor comprises a plurality of pixel groups arranged in an array, and each pixel group comprises M × N pixels arranged in an array; each pixel point corresponds to a photosensitive unit, wherein M and N are both natural numbers greater than or equal to 2; the method comprises the following steps:

Segmenting, with the image sensor, the inspection image into a first segmented image and a second segmented image in a first direction, the first segmented image comprising a plurality of rows of even-numbered pixels in the inspection image, the second segmented image comprising a plurality of rows of odd-numbered pixels in the inspection image; obtaining a phase difference value in the first direction according to the phase relation corresponding to the first segmentation image and the second segmentation image; the first direction is a row direction;

utilizing the image sensor to segment the detection image into a third segmented image and a fourth segmented image according to a second direction, wherein the third segmented image comprises pixels of multiple columns of even columns in the detection image, and the fourth segmented image comprises pixels of multiple columns of odd columns in the detection image; obtaining a phase difference value in the second direction according to a phase relation corresponding to the third segmentation image and the fourth segmentation image; the second direction is a column direction;

2. The method according to claim 1, wherein the obtaining of the detection image corresponding to the target subject prediction region comprises:

And controlling the lens to move so as to enable the focus to be aligned with the center of the target subject prediction region and acquiring a detection image corresponding to the target subject prediction region.

3. The method of claim 1, wherein determining a target subject prediction region based on the target subject detection region and the movement data of the target subject comprises:

inputting a first image to a prediction network model, wherein the first image carries information of the target subject detection area and the movement data of the target subject;

and acquiring a second image output by the prediction network model, wherein the second image is marked with the target main body prediction region.

4. The method of claim 3, wherein the predictive network model is a network model built based on a recurrent neural network algorithm.

5. The method according to claim 1, wherein the controlling the lens to continuously focus on the moving target subject according to the phase difference value in the first direction and the phase difference value in the second direction comprises:

acquiring a target defocus distance according to the phase difference value in the first direction and the phase difference value in the second direction;

And controlling a lens of the electronic equipment to continuously focus the moving target main body according to the target defocus distance.

6. The method of claim 5, wherein obtaining the defocus distance of the target according to the phase difference value of the first direction and the phase difference value of the second direction comprises:

acquiring a target phase difference value according to the phase difference value in the first direction and the phase difference value in the second direction;

and acquiring the defocusing distance of the target according to the target phase difference value.

7. The method of claim 6, wherein the obtaining the target defocus distance according to the target phase difference value comprises:

and calculating the target defocus distance according to the calibrated defocus function and the target phase difference value, wherein the calibrated defocus function is used for representing the relation between the target phase difference value and the target defocus distance.

8. The method of claim 5, wherein obtaining the target phase difference value according to the phase difference value in the first direction and the phase difference value in the second direction comprises:

acquiring a first confidence corresponding to the phase difference value in the first direction;

Acquiring a second confidence corresponding to the phase difference value in the second direction;

and determining the target phase difference value according to the magnitude relation of the first confidence coefficient and the second confidence coefficient.

9. The method of claim 8, wherein determining the target phase difference value according to a magnitude relationship between the first confidence level and the second confidence level comprises:

when the first confidence coefficient is greater than the second confidence coefficient, taking the phase difference value of the first direction corresponding to the first confidence coefficient as the target phase difference value;

when the second confidence degree is greater than the first confidence degree, taking the phase difference value of the second direction corresponding to the second confidence degree as the target phase difference value;

and when the first confidence degree is equal to the second confidence degree, taking the first direction phase difference and the second direction phase difference as the target phase difference value.

10. The method of claim 1, wherein said segmenting the inspection image into a first segmented image and a second segmented image according to a first direction comprises:

segmenting the detection image according to the first direction to obtain a plurality of image areas, wherein each image area comprises a line of pixels in the detection image; acquiring a plurality of first segmentation image areas and a plurality of second segmentation image areas from the plurality of image areas, wherein the first segmentation image areas comprise pixels of even lines in the detection image, and the second segmentation image areas comprise pixels of odd lines in the detection image; splicing the plurality of first segmentation image regions into the first segmentation image, and forming the second segmentation image by using the plurality of second segmentation image regions;

The segmenting the detection image into a third segmented image and a fourth segmented image according to the second direction includes:

performing segmentation processing on the detection image according to the second direction to obtain a plurality of image areas, wherein each image area comprises a column of pixels in the detection image; acquiring a plurality of third split image areas and a plurality of fourth split image areas from the plurality of image areas, wherein the third split image areas comprise pixels of even columns in the detection image, and the fourth split image areas comprise pixels of odd columns in the detection image; and splicing the plurality of third segmentation image regions into a third segmentation image, and forming a fourth segmentation image by using the plurality of fourth segmentation image regions.

11. The method according to claim 1, wherein the obtaining the phase difference value in the first direction according to the corresponding phase relationship between the first sliced image and the second sliced image and the obtaining the phase difference value in the second direction according to the corresponding phase relationship between the third sliced image and the fourth sliced image comprises:

determining the phase difference value of the mutually matched pixels according to the position difference of the mutually matched pixels in the first segmentation image and the second segmentation image, and determining the phase difference value in the first direction according to the phase difference value of the mutually matched pixels;

And determining the phase difference value of the mutually matched pixels according to the position difference of the mutually matched pixels in the third segmentation image and the fourth segmentation image, and determining the phase difference value in the second direction according to the phase difference value of the mutually matched pixels.

12. The method according to claim 11, wherein the pixels matched with each other refer to that the pixel matrixes corresponding to the pixels are similar to each other, and the pixel matrix corresponding to the pixels is a pixel matrix composed of the pixels themselves and a preset number of pixels around the pixels.

13. The focus tracking device is applied to electronic equipment, wherein the electronic equipment comprises an image sensor and a lens, the image sensor comprises a plurality of pixel groups arranged in an array, and each pixel group comprises M × N pixels arranged in an array; every pixel corresponds a sensitization unit, and wherein, M and N are the natural number more than or equal to 2, include:

the identification module is used for acquiring a target body detection area where a target body in the preview image is located;

An obtaining module, configured to segment, by using the image sensor, the detection image into a first segmented image and a second segmented image according to a first direction, where the first segmented image includes pixels in a plurality of even rows in the detection image, and the second segmented image includes pixels in a plurality of odd rows in the detection image; obtaining a phase difference value in the first direction according to the phase relation corresponding to the first segmentation image and the second segmentation image; the first direction is a row direction;

the acquisition module is further configured to segment, by using the image sensor, the detection image into a third segmented image and a fourth segmented image according to a second direction, where the third segmented image includes pixels in multiple even columns in the detection image, and the fourth segmented image includes pixels in multiple odd columns in the detection image; obtaining a phase difference value in the second direction according to a phase relation corresponding to the third segmentation image and the fourth segmentation image; the second direction is a column direction;

14. An electronic device comprising a memory and a processor, the memory having stored therein a computer program that, when executed by the processor, causes the processor to perform the steps of the focus tracking method according to any one of claims 1 to 12.

15. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the focus tracking method according to any one of claims 1 to 12.