CN109089047A - Method and device for controlling focus, storage medium, and electronic device - Google Patents

Abstract

The application relates to a method and a device for controlling focusing, a storage medium and electronic equipment. And controlling the RGB camera to focus according to the distance from the shooting main body to the TOF camera, and finally shooting the image to be shot by the focused RGB camera to obtain a first target image. The TOF camera is used for simultaneously obtaining the depth data of the whole image by emitting infrared light, and the speed is very high. Therefore, when an image is shot, the distance from the shooting subject to the TOF camera is obtained from the depth data of the image to be shot obtained by the TOF camera very quickly, and the RGB camera is controlled to focus according to the distance from the shooting subject to the TOF camera. And finally, shooting the image to be shot by the focused RGB camera to obtain a first target image. The focusing speed at the time of photographing is finally improved.

Description

Method and device for controlling focus, storage medium, and electronic device

technical field

The present application relates to the field of computer technology, in particular to a method and device for controlling focus, a storage medium, and electronic equipment.

Background technique

With the rapid development of smart mobile terminals, users use mobile terminals with camera functions to take pictures more and more frequently. When using a mobile terminal to take pictures, it is generally necessary to perform a focusing operation, which may be performed automatically by the camera of the mobile terminal, or may be achieved by the user clicking on the screen. The traditional focusing method has a complex processing process, resulting in a slow focusing speed.

Contents of the invention

Embodiments of the present application provide a method and device for controlling focusing, a storage medium, and an electronic device, which can improve the focusing speed.

A method of controlling focus, comprising:

Obtain the depth data of the image to be captured through the TOF camera;

Obtain the distance from the subject to the TOF camera from the depth data of the image to be captured;

Control the RGB camera to focus according to the distance from the subject to the TOF camera;

The image to be captured is captured by the focused RGB camera to obtain the first target image.

A device for controlling focus, the device comprising:

The depth data acquisition module of the image to be captured is used to obtain the depth data of the image to be captured by the TOF camera;

A distance acquisition module, configured to acquire the distance from the subject to the TOF camera from the depth data of the image to be captured;

A focus module, used to control the RGB camera to focus according to the distance from the subject to the TOF camera;

The photographing module is configured to photograph the image to be photographed by the focused RGB camera to obtain the first target image.

A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for controlling focus as described above are realized.

An electronic device includes a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the computer program, it executes the steps of the method for controlling focus as described above.

The above method and device for controlling focus, storage medium, and electronic equipment obtain depth data of the image to be captured through the TOF camera, and obtain the distance from the subject to the TOF camera from the depth data of the image to be captured. According to the distance from the subject to the TOF camera, the RGB camera is controlled to focus, and finally the focused RGB camera shoots the image to be captured to obtain the first target image. The TOF camera obtains the depth data of the entire image at the same time by emitting infrared light, which is very fast. Therefore, when shooting an image, it is also very fast to obtain the distance from the subject to the TOF camera from the depth data of the image to be captured obtained by the TOF camera, and then control the RGB camera to focus according to the distance from the subject to the TOF camera. Finally, the focused RGB camera shoots the image to be shot to obtain the first target image. So in the end, the focus speed when shooting was improved.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present application. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is an internal structural diagram of an electronic device in an embodiment;

Fig. 2 is a flowchart of a method for controlling focus in an embodiment;

Fig. 3 is the flowchart of the method for obtaining the distance from the subject to the TOF camera from the depth data of the image to be captured in Fig. 2;

FIG. 4 is a flow chart of a method for acquiring a first focal point from an image to be captured in FIG. 3;

FIG. 5 is a flowchart of a method for controlling focus in another embodiment;

FIG. 6 is a schematic structural diagram of a device for controlling focus in an embodiment;

Fig. 7 is a schematic structural diagram of a device for controlling focus in another embodiment;

Fig. 8 is a schematic diagram of an image processing circuit in one embodiment.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

Fig. 1 is a schematic diagram of the internal structure of an electronic device in one embodiment. As shown in FIG. 1, the electronic device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory is used to store data, programs, etc., and at least one computer program is stored on the memory, and the computer program can be executed by the processor to implement the scene recognition method suitable for electronic devices provided in the embodiments of the present application. The memory may include a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random-access-memory (Random-Access-Memory, RAM). For example, in one embodiment, the memory includes non-volatile storage media and internal memory. Nonvolatile storage media store operating systems and computer programs. The computer program can be executed by a processor, so as to implement a method for controlling focus provided in the following embodiments. The internal memory provides a high-speed running environment for the operating system computer program in the non-volatile storage medium. The network interface can be an Ethernet card or a wireless network card, etc., and is used for communicating with external electronic devices. The electronic device may be a mobile phone, a tablet computer, a personal digital assistant, or a wearable device.

In one embodiment, as shown in FIG. 2 , a method for controlling focus is provided, and the application of the method to the electronic device in FIG. 1 is used as an example for illustration, including:

Step 220, acquire the depth data of the image to be captured through the TOF camera.

TOF is the abbreviation of Time of flight, literally translated as flight time. A TOF camera is a time-of-flight camera. The TOF camera continuously sends light pulses to the target, and then uses the sensor to receive the light returned from the object, and obtains the target object distance by detecting the flight (round-trip) time of the light pulse. This technology is different from the 3D laser sensor in principle. The 3D laser sensor scans point by point, while the TOF camera obtains the depth information of the entire image at the same time. The image to be captured here may refer to a preview image presented on the display screen of the electronic device when the camera is turned on.

Step 240, acquiring the distance from the subject to the TOF camera from the depth data of the image to be captured.

Generally, when shooting, there will be a subject. When auto-focusing, the camera generally selects the object that is closest to the lens and has the highest contrast as the subject. It often happens that the subject that the camera automatically finds is not the subject that the user needs, or the position of the object to be photographed does not meet the user’s preference. Composition, then it is necessary to manually select the subject. For example, when the image to be captured is a person in the foreground and a blue sky and white clouds in the background, the camera can automatically take the person as the subject. Of course, the person in the preview image can also be taken as the subject by manually tapping the screen by the user. However, when the proportion of the background in the image to be captured is very small, and the proportion of the figure included in the foreground is very high, and the number of figures is relatively large, it is generally difficult to automatically determine the subject through the camera at this time. By clicking on the screen, one or several characters in the preview image can be taken as the subject.

After the subject is determined, the distance from the subject to the TOF camera is obtained from the depth data of the image to be captured. Specifically, the first focal point is acquired from the image to be photographed, and the first focal point is located on the subject. The depth data corresponding to the first focal point is obtained from the depth data of the image to be captured, and the depth data corresponding to the first focal point is used as the distance from the subject to the TOF camera.

Step 260, controlling the RGB camera to focus according to the distance from the subject to the TOF camera.

After obtaining the distance from the subject to the TOF camera, calculate the motor parameters of the RGB camera according to the distance from the subject to the TOF camera, and then control the movement of the motor of the RGB camera according to the calculated motor parameters of the RGB camera to achieve focus. The number of RGB cameras on the electronic device is not limited.

In step 280, the focused RGB camera captures the image to be captured to obtain a first target image.

The depth data from the first focal point on the subject to the TOF camera (or the imaging plane) collected by the TOF camera is used to control the RGB camera to focus according to the depth data. Specifically, according to the depth data, the calibration data of the simultaneous calibration of the RGB camera and the TOF camera, and the motor parameters of the RGB camera in the current camera, the depth data from the first focal point to the TOF camera is converted into the motor of the RGB camera trying to focus Move steps. Then move the motor of the RGB camera by the corresponding number of steps to achieve focusing according to the first focal point. Then, the image to be captured is captured by the focused RGB camera to obtain the first target image.

In the embodiment of this application, firstly, compared with the binocular stereo camera or the triangulation system, the TOF camera is small in size, which is almost the same size as a general camera, and is very suitable for some occasions that require a light and small camera. TOF cameras can quickly calculate depth information in real time, reaching tens to 100fps. Secondly, binocular stereo cameras need to use complex correlation algorithms, and the processing speed is relatively slow. The depth calculation accuracy of TOF does not change with the distance, and can basically be stable at the cm level, which is very meaningful for some applications with large-scale motion. And the TOF camera can still measure the distance in the dark environment, which solves the problem of the RGB camera losing focus in the dark environment.

The depth data of the image to be captured is acquired through the TOF camera, and the distance from the subject to the TOF camera is acquired from the depth data of the image to be captured. Control the RGB camera to focus according to the distance from the subject to the TOF camera. The TOF camera obtains the depth data of the entire image at the same time by emitting infrared light, which is very fast. Therefore, when shooting an image, it is also very fast to obtain the distance from the subject to the TOF camera from the depth data of the image to be captured obtained by the TOF camera, and then control the RGB camera to focus according to the distance from the subject to the TOF camera. Therefore, the focusing speed during shooting is finally improved, and the focusing accuracy is higher, and accurate focusing can also be performed in low-light environments. Finally, the image to be captured is captured by the focused RGB camera to obtain the first target image. Obviously, the image quality of the first target image captured after the above focusing process will be greatly improved.

In one embodiment, as shown in FIG. 3, step 240, obtaining the distance from the subject to the TOF camera from the depth data of the image to be captured includes:

Step 242, acquiring a first focal point from the image to be captured, where the first focal point is located on the subject.

First, it is necessary to determine the subject from the image to be captured, and then determine the first focal point from the subject. The above-mentioned steps of determining the subject and determining the first focal point can be automatically set by the camera, of course, can also be determined manually by the user by clicking the preview image appearing on the electronic screen. The camera generally chooses to automatically select the object closest to the lens and with the highest contrast as the subject, and select a certain point on the subject as the first focus. For example, when the image to be captured is a person in the foreground and a blue sky and white clouds in the background, the camera can automatically take the foreground person closest to the lens and with the greatest contrast as the subject. Then select a certain point (such as the bridge of the nose) from the face of the foreground person as the first focus.

When the subject and the first focal point automatically determined by the camera do not meet the user's photographing needs, the user can manually click the preview image that appears on the electronic screen to determine the subject, and further determine the first focus from the subject .

Step 244, acquiring depth data corresponding to the first focal point from the depth data of the image to be captured.

After the subject and the first focal point are determined, the depth data corresponding to the first focal point can be obtained from the depth data of the image to be captured. The depth data is the depth data of the image to be captured obtained by the TOF camera emitting and receiving infrared light. The acquisition process is not only fast and accurate, but also suitable for environments with good light and dark light.

Step 246, use the depth data corresponding to the first focal point as the distance from the subject to the TOF camera.

The distance from the subject to the TOF camera can be understood as the focal length. The depth data corresponding to the first focal point is used as the distance from the subject to the TOF camera. Then you can control the RGB camera to focus according to the distance from the subject to the TOF camera.

In the embodiment of the present application, after the subject is determined from the image to be captured, and the first focal point is further determined from the subject. The depth data of the first focal point is correspondingly obtained directly from the depth data of the image to be captured obtained by the TOF camera. The depth data of the first focal point is the distance from the subject to the TOF camera. From the depth data of the image to be captured acquired by the TOF camera, the accuracy of the acquired distance from the subject to the TOF camera is very high. Therefore, the accuracy of subsequently controlling the focus of the RGB camera according to the acquired distance from the subject to the TOF camera is further improved.

In one embodiment, as shown in FIG. 4, step 242, acquiring the first focal point from the image to be captured, where the first focal point is located on the subject, includes:

Step 242a, acquiring the subject from the image to be captured;

Step 242b, determine multiple focus points to be selected from the subject, and the multiple focus points to be selected have different depth data;

Step 242c, perform shooting preview according to each focus point to be selected respectively, and obtain multiple preview images;

Step 242d, filter out the target preview image from the multiple preview images, acquire the candidate focus corresponding to the target preview image, and use the candidate focus as the first focus.

Specifically, after the subject is acquired from the image to be photographed according to the method in the above embodiment, multiple focus points to be selected are determined from the subject, and the multiple focus points to be selected have different depth data respectively. Wherein, the depth data of multiple focus points to be selected may be determined according to an arithmetic sequence. For example, when it is determined from the image to be captured that the subject is a person, it is assumed that there are multiple persons in the image to be captured, and the multiple persons stand together with front and back. At this point, multiple characters have different depth data. The depth data is screened from the depth data of these characters in the sequence of the arithmetic sequence, that is, the screened depth data is arranged in an arithmetic sequence. For example, filter out a piece of data (for example, 3 meters) from the depth data of the first person standing closest to the shooting camera, and then filter out a data from the depth data of the second person whose distance from the shooting camera is second only to the first person One data is selected from the depth data (for example, 3.05 meters, and the tolerance of the arithmetic series is 0.05 meters at this time, of course, it can be set to other values in other embodiments), and a set of arithmetic series is obtained by analogy. The focus points corresponding to these depth values in arithmetic progression are the focus points to be selected.

The shooting preview is performed according to each focus point to be selected respectively, and multiple preview images are obtained. For example, shooting the image to be captured according to the focus to be selected is to sequentially adjust the focus of the RGB camera according to the above-mentioned depth values in an arithmetic sequence, so as to obtain a group of preview images respectively. Each preview image corresponds to a different shooting focus, so the image quality of each preview image will vary. The image quality here may include sharpness, brightness, chroma, etc., which is not limited here.

After obtaining a group of preview images, a target preview image is selected from multiple preview images, and the target preview image is a preview image with the best image quality in the group of preview images. The preview image with the best image quality is selected from multiple preview images, which can be automatically selected by the processor inside the camera. After a preview image with the best image quality is screened out, the candidate focus corresponding to the target preview image is acquired, and the candidate focus is used as the first focus. Finally, when the image to be processed is formally photographed, the depth data of the first focus can be obtained according to the first focus, and the depth data corresponding to the first focus can be used as the distance from the subject to the TOF camera. According to the distance from the subject to the TOF camera, the RGB camera is controlled to focus, and the image to be captured is captured by the focused RGB camera to obtain the first target image.

In the embodiment of the present application, when determining the first focal point from the image to be processed, firstly, a plurality of candidate focal points are determined from the subject, and each candidate focal point has different depth data. Therefore, a group of preview images are shot according to multiple focus points to be selected, and these preview images are shot according to the focus points with different depth data respectively. Therefore, the image with the best image quality is selected from the plurality of preview images, so that the result after comparison and screening is more accurate, which is more conducive to improving the accuracy of the determined first focus, and then improving the accuracy of the images captured according to the first focus. The image quality of the first target image. And the depth data is obtained by the TOF camera. The TOF camera obtains the depth data of the entire image at the same time by emitting infrared light, and the speed is very fast. Therefore, while improving the focusing speed, the image quality of the captured first target image is also guaranteed.

In one embodiment, as shown in FIG. 5, in step 280, the image to be captured is captured by the focused RGB camera, and after the first target image is obtained, it includes:

Step 510, judging whether the image quality of the first target image meets a preset standard.

The preset standard can be pre-stored in the electronic device, for example, the definition of the image can reach a certain standard, or the color and brightness of the image can reach a certain standard.

Step 520, if not, acquire a point within a preset range centered on the first focal point from the image to be captured as a second focal point, and the second focal point is located on the subject.

When the first focus point is preliminarily determined, and the RGB camera is controlled to focus according to the obtained distance from the subject to the TOF camera, the captured first target image is judged whether the image quality reaches the preset standard, if the judgment result is If the preset standard is reached, there is no need to re-determine the second focal point.

If the judgment result is that the preset standard is not reached, then the second focus point needs to be re-determined again, and of course, the second focus point is still located on the subject. A point within a preset range centered on the first focal point is acquired from the image to be captured as the second focal point. For example, within the preset range centered on the first focal point, a specific number of pixels can be moved to the left, right, up, or down, respectively, so that the obtained pixels correspond to the points on the subject as Second focus. Specifically, which direction to move and how many pixels to move can be determined according to the image quality of the first target image. For example, when the sharpness of the first target image is lower than the preset standard, it means that the determined depth data of the first focus point is greater than the focal length when shooting clearly. Therefore, when the second focus point is determined, it can be A point near the first focal point whose depth data is slightly smaller than that of the first focal point is selected as the second focal point. Assuming that the depth data of the first focal point is 4 meters, a point near the first focal point with a depth data of 3.95 meters is selected as the second focal point at an interval of 0.05 meters. Of course, this interval can also be set to other reasonable values.

When the clarity of the first target image meets the preset standard, but the brightness is lower than the preset standard, it means that the determined depth data of the first focal point meets the preset standard, but the brightness at the first focal point is relatively low. If it is bright, it will cause the entire image taken to be dark. At this time, in order to make the brightness of the image meet the preset standard, you can select a point near the first focal point whose depth data is the same as the depth data of the first focal point, but the brightness is relatively dark as the second focal point. At this time, according to the second focal point The brightness of the image taken by the two focal points will be improved.

Step 530, acquiring depth data corresponding to the second focal point from the depth data of the image to be captured.

Step 540, use the depth data corresponding to the second focal point as the distance from the subject to the TOF camera.

According to the position of the second focus on the image to be captured, the depth data corresponding to the second focus is obtained from the depth data of the image to be captured. The depth data corresponding to the second focal point is used as the distance from the subject to the TOF camera. It can be considered that the depth data corresponding to the second focal point is the focal length.

Step 550, controlling the RGB camera to focus according to the distance from the subject to the TOF camera.

Step 560, the image to be captured is captured by the focused RGB camera to obtain a second target image.

According to the depth data corresponding to the second focus, the calibration data of the RGB camera and the TOF camera that are calibrated simultaneously, and the motor parameters of the RGB camera in the current camera, try to convert the depth data from the second focus to the TOF camera into the motor of the RGB camera The number of steps to focus on. Then move the motor of the RGB camera by the corresponding number of steps to achieve focusing according to the second focal point. Then, the image to be captured is captured by the refocused RGB camera to obtain the second target image.

In the embodiment of the present application, after the depth data of the first focal point is acquired for the first time according to the first focal point roughly acquired from the subject corresponding to the depth data collected by the TOF camera. The RGB camera is focused according to the depth data of the first focal point, and the first target image is obtained by shooting with the focused RGB camera. If it is judged that the image quality of the first target image does not meet the preset standard, it is necessary to fine-tune the focusing effect. You can try to move a specific number of pixels to the left or right or up or down within the preset range centered on the first focus, so that the acquired pixels correspond to the points on the subject as the second focus. Similarly, the RGB camera is focused according to the depth data of the second focal point, and the focused RGB camera is used to shoot to obtain the second target image. It is judged again whether the image quality of the second target image meets the preset standard. Try this until the resulting image quality meets the preset standards. After many attempts to change the focus position, this can avoid the inaccuracy of the first focus acquired at one time, resulting in that although the depth data of the first focus can be accurately obtained according to the TOF camera, it still cannot be obtained from the source (that is, the focus position) ) to improve the quality of captured images.

In one embodiment, the RGB camera is controlled to focus according to the distance from the subject to the TOF camera, including:

Calculate the motor parameters of the RGB camera according to the distance from the subject to the TOF camera;

The movement of the motor of the RGB camera is controlled according to the calculated motor parameters of the RGB camera to achieve focusing.

In the embodiment of the present application, according to the distance from the subject to the TOF camera, the calibration data of the simultaneous calibration of the RGB camera and the TOF camera, and the motor parameters of the RGB camera in the current camera, the first focal point on the subject to the TOF camera The depth data is converted to the number of steps the RGB camera's motors are moving to try to focus. Then move the motor of the RGB camera by the corresponding number of steps to achieve focusing according to the first focal point. According to the depth data of the TOF camera, the motor of the RGB camera can be moved quickly and accurately to achieve focus.

In one embodiment, the depth data of the image to be captured is acquired by a TOF camera, including:

Obtain multiple frames of TOF data of different phases of the image to be captured through the TOF camera, and each phase corresponds to one frame of TOF data;

Combine multiple frames of TOF data to obtain the depth data of the image to be captured.

In the embodiment of the present application, a frame of depth data of an image to be captured is generally synthesized from a frame of TOF data corresponding to 4 phases or 8 phases respectively. When the phase is selected as 4, a frame of TOF data corresponding to each of the 4 phases of the image to be captured is obtained through the TOF camera, and there are 4 frames of TOF data. The 4 frames of TOF data are synthesized to obtain the depth data of the image to be captured. Of course, when the selected phase is 8, the TOF camera obtains a frame of TOF data corresponding to the 8 phases of the image to be captured respectively, and there are 8 frames of TOF data. The 8 frames of TOF data are synthesized to obtain the depth data of the image to be captured.

In one embodiment, after obtaining the depth data of the image to be captured by the TOF camera, including:

Perform filtering and denoising operations on the depth data of the image to be captured.

In the embodiment of the present application, Poisson equation filtering, Gaussian filtering and bilateral filtering methods may be used to perform filtering and denoising operations on the depth data of the image to be captured. Therefore, the depth data of the image to be captured obtained after filtering and denoising processing is more accurate.

In one embodiment, as shown in FIG. 6 , an apparatus 600 for controlling focus is provided, including: a depth data acquisition module 610 of an image to be captured, a distance acquisition module 620 , a focusing module 630 and a photographing module 640 . in,

The depth data acquisition module 610 of the image to be captured is used to obtain the depth data of the image to be captured by the TOF camera;

A distance acquisition module 620, configured to acquire the distance from the subject to the TOF camera from the depth data of the image to be captured;

A focus module 630, configured to control the RGB camera to focus according to the distance from the subject to the TOF camera;

The photographing module 640 is configured to photograph the image to be photographed by the focused RGB camera to obtain the first target image.

In one embodiment, the distance acquiring module 620 is further configured to acquire the first focal point from the image to be captured, where the first focus is located on the subject; acquire the depth data corresponding to the first focus from the depth data of the image to be captured; The depth data corresponding to the first focal point is used as the distance from the subject to the TOF camera.

In one embodiment, the distance acquisition module 620 is also used to acquire the subject from the image to be photographed; determine a plurality of focus points to be selected from the subject, and the focus points to be selected have different depth data; Select the focal point for shooting preview to obtain multiple preview images; filter out the target preview image from the multiple preview images, obtain the candidate focus corresponding to the target preview image, and use the candidate focus as the first focus.

In one embodiment, as shown in FIG. 7 , an apparatus 600 for controlling focus is provided, which further includes:

A judging module 650, configured to judge whether the image quality of the first target image reaches a preset standard;

The second focus acquisition module 660 is configured to, if not, acquire a point within a preset range centered on the first focus from the image to be captured as a second focus, and the second focus is located on the subject;

The refocusing module 670 is used to obtain the depth data corresponding to the second focal point from the depth data of the image to be captured; use the depth data corresponding to the second focal point as the distance from the subject to the TOF camera; according to the distance from the subject to the TOF camera Control the RGB camera to focus;

The re-capturing module 680 is configured to capture the image to be captured by the focused RGB camera to obtain a second target image.

In one embodiment, the focus module 630 is also used to calculate the motor parameters of the RGB camera according to the distance from the subject to the TOF camera; and control the movement of the motors of the RGB camera according to the calculated motor parameters of the RGB camera to achieve focusing.

In one embodiment, the depth data acquisition module 610 of the image to be captured is also used to obtain multiple frames of TOF data of different phases of the image to be captured through the TOF camera, each phase corresponds to a frame of TOF data; the multiple frames of TOF data Synthesis is performed to obtain the depth data of the image to be captured.

In one embodiment, the depth data acquisition module 610 of the image to be captured is further configured to perform filtering and denoising operations on the depth data of the image to be captured.

The division of each module in the above-mentioned device for controlling focus is only for illustration. In other embodiments, the device for controlling focus can be divided into different modules according to needs, so as to complete all or part of the functions of the device for controlling focus.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for controlling focus provided in the foregoing embodiments are implemented.

In one embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and operable on the processor. When the processor executes the computer program, the focus control provided by the above-mentioned embodiments is realized. steps of the method.

An embodiment of the present application further provides a computer program product, which, when running on a computer, causes the computer to execute the steps of the method for controlling focus provided in the foregoing embodiments.

The embodiment of the present application also provides an electronic device. The electronic device can be any terminal device including mobile phone, tablet computer, PDA (Personal Digital Assistant, personal digital assistant), POS (Point of Sales, sales terminal), vehicle-mounted computer, wearable device, etc., taking the electronic device as a mobile phone as an example : The above-mentioned electronic device includes an image processing circuit, and the image processing circuit may be realized by hardware and/or software components, and may include various processing units defining an ISP (Image Signal Processing, image signal processing) pipeline. Fig. 8 is a schematic diagram of an image processing circuit in one embodiment. As shown in FIG. 8 , for ease of description, only various aspects of the image processing technology related to the embodiment of the present application are shown.

As shown in FIG. 8 , the image processing circuit includes a first ISP processor 830 , a second ISP processor 840 and a control logic 850 . The first camera 810 includes one or more first lenses 812 and a first image sensor 814 . The first image sensor 814 can include a color filter array (such as a Bayer filter), and the first image sensor 814 can obtain light intensity and wavelength information captured by each imaging pixel of the first image sensor 814, and provide information that can be obtained by the first ISP. A set of image data processed by the processor 830. The second camera 820 includes one or more second lenses 822 and a second image sensor 824 . The second image sensor 824 can include a color filter array (such as a Bayer filter), and the second image sensor 824 can obtain light intensity and wavelength information captured by each imaging pixel of the second image sensor 824, and provide information that can be obtained by the second ISP. A set of image data processed by processor 840.

The first image collected by the first camera 810 is transmitted to the first ISP processor 830 for processing. After the first ISP processor 830 processes the first image, the statistical data of the first image (such as the brightness of the image, the contrast value of the image , the color of the image, etc.) to the control logic 850, the control logic 850 can determine the control parameters of the first camera 810 according to the statistical data, so that the first camera 810 can perform operations such as auto focus and auto exposure according to the control parameters. The first image can be stored in the image memory 860 after being processed by the first ISP processor 830 , and the first ISP processor 830 can also read the image stored in the image memory 860 for processing. In addition, the first image can be directly sent to the display 870 for display after being processed by the ISP processor 830 , and the display 870 can also read the image in the image memory 860 for display.

Wherein, the first ISP processor 830 processes image data pixel by pixel in various formats. For example, each image pixel may have a bit depth of 8, 10, 12 or 14 bits, and the first ISP processor 830 may perform one or more image processing operations on the image data, collect statistical information about the image data. Among them, image processing operations can be performed with the same or different bit depth calculation precision.

The image memory 860 may be a part of a memory device, a storage device, or an independent dedicated memory in an electronic device, and may include a DMA (Direct Memory Access) feature.

Upon receiving an interface from the first image sensor 814, the first ISP processor 830 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 860 for additional processing before being displayed. The first ISP processor 830 receives the processed data from the image memory 860, and performs image data processing in RGB and YCbCr color spaces on the processed data. The image data processed by the first ISP processor 830 may be output to the display 870 for viewing by the user and/or for further processing by a graphics engine or a GPU (Graphics Processing Unit, graphics processor). In addition, the output of the first ISP processor 830 can also be sent to the image memory 860 , and the display 870 can read image data from the image memory 860 . In one embodiment, image memory 860 may be configured to implement one or more frame buffers.

The statistics determined by the first ISP processor 830 may be sent to the control logic 850 . For example, statistical data may include first image sensor 814 statistical information such as automatic exposure, automatic white balance, automatic focus, flicker detection, black level compensation, first lens 812 shading correction, and the like. The control logic 850 may include a processor and/or a microcontroller executing one or more routines (e.g., firmware) that determine control parameters of the first camera 810 and the second camera 810 based on received statistical data. A control parameter of the ISP processor 830. For example, the control parameters of the first camera 810 may include gain, integration time of exposure control, anti-shake parameters, flash control parameters, control parameters of the first lens 812 (such as focal length for focusing or zooming), or a combination of these parameters. ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (eg, during RGB processing), as well as first lens 812 shading correction parameters.

Similarly, the second image collected by the second camera 820 is transmitted to the second ISP processor 840 for processing. After the second ISP processor 840 processes the first image, the statistical data of the second image (such as the brightness of the image, the image The contrast value of the image, the color of the image, etc.) are sent to the control logic 850, and the control logic 850 can determine the control parameters of the second camera 820 according to the statistical data, so that the second camera 820 can perform operations such as autofocus and auto exposure according to the control parameters . The second image can be stored in the image memory 860 after being processed by the second ISP processor 840 , and the second ISP processor 840 can also read the image stored in the image memory 860 for processing. In addition, the second image can be directly sent to the display 870 for display after being processed by the ISP processor 840 , and the display 870 can also read the image in the image memory 860 for display. The second camera 820 and the second ISP processor 840 can also implement the processing described in the first camera 810 and the first ISP processor 830 .

The following are the steps of implementing the image processing method using the image processing technology in FIG. 8 .

Any reference to memory, storage, database, or other medium as used herein may include non-volatile and/or volatile memory. Suitable nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in many forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous Synchlink DRAM (SLDRAM), Memory Bus (Rambus) Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM).

The above-mentioned embodiments only express several implementation modes of the present application, and the description thereof is relatively specific and detailed, but should not be construed as limiting the patent scope of the present application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the scope of protection of the patent application should be based on the appended claims.

Claims (10)

1. A method of controlling focus, comprising:

acquiring depth data of an image to be shot through a TOF camera;

acquiring the distance from a shooting subject to the TOF camera from the depth data of the image to be shot;

controlling an RGB camera to focus according to the distance from the shooting main body to the TOF camera;

and shooting the image to be shot by the focused RGB camera to obtain a first target image.

2. The method according to claim 1, wherein the obtaining of the distance from the subject to the TOF camera from the depth data of the image to be captured comprises:

acquiring a first focus from the image to be shot, wherein the first focus is positioned on a shooting subject;

acquiring depth data corresponding to the first focus from the depth data of the image to be shot;

and taking the depth data corresponding to the first focus as the distance from the shooting subject to the TOF camera.

3. The method according to claim 2, wherein the obtaining a first focus from the image to be captured, the first focus being located on a subject, comprises:

acquiring a shooting subject from the image to be shot;

determining a plurality of to-be-selected focuses from the shooting subject, wherein the plurality of to-be-selected focuses have different depth data;

shooting and previewing according to each focus to be selected respectively to obtain a plurality of preview images;

and screening a target preview image from the plurality of preview images, acquiring a to-be-selected focus corresponding to the target preview image, and taking the to-be-selected focus as a first focus.

4. The method according to claim 1, wherein after the capturing the image to be captured by the focused RGB camera to obtain a first target image, the method comprises:

judging whether the image quality of the first target image reaches a preset standard or not;

if not, acquiring a point within a preset range taking the first focus as the center from the image to be shot as a second focus, wherein the second focus is positioned on the shooting main body;

acquiring depth data corresponding to the second focus from the depth data of the image to be shot;

taking the depth data corresponding to the second focus as the distance from the shooting subject to the TOF camera;

controlling an RGB camera to focus according to the distance from the shooting main body to the TOF camera;

and shooting the image to be shot by the focused RGB camera to obtain a second target image.

5. The method of claim 1, wherein the controlling the RGB camera to focus according to the distance from the subject to the TOF camera comprises:

calculating motor parameters of the RGB camera according to the distance from the shooting main body to the TOF camera;

and controlling the motor of the RGB camera to move according to the calculated motor parameters of the RGB camera so as to realize focusing.

6. The method according to claim 1, wherein the obtaining of depth data of the image to be captured by the TOF camera comprises:

multi-frame TOF data of different phases of an image to be shot are respectively obtained through the TOF camera, and each phase corresponds to one frame of TOF data;

and synthesizing the multi-frame TOF data to obtain depth data of the image to be shot.

7. The method according to claim 1, after said acquiring depth data of the image to be captured by the TOF camera, comprising:

and carrying out filtering and denoising operations on the depth data of the image to be shot.

8. An apparatus for controlling focusing, the apparatus comprising:

the device comprises a depth data acquisition module of an image to be shot, a depth data acquisition module of the image to be shot and a depth data acquisition module of the image to be shot, wherein the depth data acquisition module is used for acquiring the depth data of the image to be shot through a TOF camera;

the distance acquisition module is used for acquiring the distance from the shooting main body to the TOF camera from the depth data of the image to be shot;

the focusing module is used for controlling the RGB camera to focus according to the distance from the shooting main body to the TOF camera;

and the shooting module is used for shooting the image to be shot by the focused RGB camera to obtain a first target image.

9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of controlling focusing according to any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of controlling focus according to any one of claims 1 to 7 when executing the computer program.