WO2021134179A1

WO2021134179A1 - Focusing method and apparatus, photographing device, movable platform and storage medium

Info

Publication number: WO2021134179A1
Application number: PCT/CN2019/129852
Authority: WO
Inventors: 翁松伟; 韩守谦; 普贵翔
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-12-30
Filing date: 2019-12-30
Publication date: 2021-07-08
Also published as: CN112585941A

Abstract

Provided by the present application are a focusing method and apparatus, a photographing device, a movable platform, and a storage medium. The focusing method comprises: determining a first position region corresponding to a target object in a first image collected by a first photographing device; setting the image definition corresponding to the first position region in the first image to have a first weight, and the image definition corresponding to regions in the first image other than the first position region to have a second weight, wherein the first weight is greater than the second weight; acquiring a definition statistical value of the first image according to the first weight and the second weight; and if the definition statistical value of the first image meets a set focusing condition, then perform focusing on the target object so as to ensure that the target object is clearly imaged.

Description

Focusing method, device, shooting equipment, movable platform and storage medium

Technical field

The present invention relates to the field of cameras, in particular to a focusing method, device, photographing equipment, movable platform and storage medium.

Background technique

Camera functions are used in many application scenarios, and devices that provide camera functions include mobile phones, cameras, and so on. During the shooting process, in order to ensure a clear imaging picture, a focusing process is necessary. Currently, many shooting devices support an auto focus (Auto Focus, AF for short) function to complete focusing through the AF function.

At present, in the process of performing AF, the focus area is often selected as the center area of the image taken by the photographing device. At this time, the purpose of AF is to ensure that the image in the focus area is clearly imaged. With this AF processing method, it is difficult to ensure that the target object that the user pays attention to is clearly imaged among several objects being photographed.

Summary of the invention

The invention provides a focusing method, device, photographing equipment, movable platform and storage medium, which can realize rapid focusing of target objects.

The first aspect of the present invention provides a focusing method, including:

Determine a first location area corresponding to a target object in a first image captured by a first photographing device; wherein the first location area is an area that covers the target object and is smaller than the first image;

Set the image definition corresponding to the first position area in the first image to have a first weight; wherein, the image definition corresponding to the areas other than the first position area in the first image has A second weight, where the first weight is greater than the second weight;

Acquiring a sharpness statistical value of the first image according to the first weight and the second weight;

If the sharpness statistical value of the first image meets the set focus condition, then focus processing is performed on the target object.

A second aspect of the present invention provides a focusing device, which is provided in a first photographing device. The focusing device includes: a memory and a processor; wherein the memory stores executable code, when the executable code is When the processor executes, the processor realizes:

Determining a first location area corresponding to a target object in a first image captured by the first photographing device; wherein the first location area is an area that covers the target object and is smaller than the first image;

A third aspect of the present invention provides a photographing device, including:

The lens assembly is arranged inside the housing of the photographing device;

The sensor module is arranged inside the housing and at the rear end of the lens assembly, the sensor module includes a circuit board and an imaging sensor, and the imaging sensor is arranged on the front surface of the circuit board facing the lens assembly ；

The focusing device according to the second aspect is arranged inside the housing.

The fourth aspect of the present invention provides a movable platform, including:

Body

The power system is arranged on the body and used to provide power for the movable platform;

The photographing device according to the third aspect is arranged on the body and used to photograph a first image and perform focusing processing on a target object in the first image.

A fifth aspect of the present invention provides a computer-readable storage medium having executable code stored in the computer-readable storage medium, and the executable code is used to implement the focusing method described in the first aspect.

In the focusing method provided by the present invention, firstly, the position area (referred to as the first position area) corresponding to the target object being focused in the image currently collected by the shooting device performing the focusing method is determined, and then the image can be calculated. The image definition of the image, and the image definition corresponding to the first location area containing the target object is set to have the first weight, and the image definition corresponding to the areas other than the first location area in the image has the first weight. Two weights, the first weight is greater than the second weight, so that the shooting device can quickly and accurately identify the focused subject-the target object. Afterwards, the statistical value of the sharpness of the image is calculated according to the first weight and the second weight. When the statistical value of the sharpness of the image meets the set focusing conditions, it means that the image of the target object is blurred, so as to focus on the target object Processing to ensure clear imaging of the target object.

Description of the drawings

The drawings described here are used to provide a further understanding of the application and constitute a part of the application. The exemplary embodiments and descriptions of the application are used to explain the application, and do not constitute an improper limitation of the application. In the attached picture:

FIG. 1 is a schematic flowchart of a focusing method according to an embodiment of the present invention;

FIG. 2 is a schematic flowchart of a focusing process provided by an embodiment of the present invention;

3 is a schematic flowchart of another focusing method provided by an embodiment of the present invention;

4 is a schematic flowchart of another focusing method provided by an embodiment of the present invention;

FIG. 5 is a schematic diagram of an application scenario of a focusing method provided by an embodiment of the present invention;

6 is a schematic structural diagram of a focusing device provided by an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of a photographing device provided by an embodiment of the present invention;

FIG. 8 is a schematic structural diagram of a movable platform provided by an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments, and are not intended to limit the present invention.

FIG. 1 is a schematic flowchart of a focusing method provided by an embodiment of the present invention. As shown in FIG. 1, the focusing method may include the following steps:

101. Determine a first location area corresponding to a target object in a first image captured by a first photographing device, where the first location area is an area that covers the target object and is smaller than the first image.

102. Set the image definition corresponding to the first position area in the first image to have the first weight, and the image definition corresponding to the areas other than the first position area in the first image have the second weight, and the first weight is greater than The second weight.

103. Acquire a sharpness statistical value of the first image according to the first weight and the second weight.

104. If the sharpness statistical value of the first image meets the set focusing condition, perform focusing processing on the target object.

In the embodiment of the present invention, the first photographing device may be a visible light zoom camera.

In practical applications, the first photographing device can be integrated and used in other devices, or can be used independently. For example, the first photographing device may be implemented as a camera on a terminal device such as a mobile phone, a notebook computer, and the like. For another example, the first photographing device may be a camera mounted on a drone for use.

The target object mentioned in the embodiment of the present invention may be set by the user according to requirements. Specifically, before the user uses the above-mentioned first photographing device to photograph the target object, the user can set the target object on the first photographing device to inform the first photographing device of what the target object is, so that the first photographing device finally completes Focusing on the target object is to ensure that the image of the target object is clear.

Among them, the process of focusing is to continuously adjust the object distance of the first photographing device to ensure that the target object photographed by the first photographing device is always clearly imaged.

Optionally, the user's setting of the target object may be implemented as: the user inputs a category corresponding to the target object. Therefore, the first photographing device considers the object corresponding to the category in the first image collected as the target object.

Affected by the angle of view of the first photographing device, in addition to the target object, other objects may be included in the first captured image. For example, the target object is a human body, and other objects may be trees, buildings, flowers, vehicles, etc. that exist around the person. After capturing the first image, the first camera can identify whether the first image contains the target object set by the user and the first location area of the target object in the first image, where the first location area covers the target object And is smaller than the area of the first image.

In practical applications, the first imaging device can identify whether the first image contains the target object based on the visible light characteristics of the target object. The so-called visible light feature refers to the optical feature of the target object. For example, when the target object is a human body, the corresponding visible light feature can be the contour shape of the human body, facial features, and so on. The implementation of human body recognition based on these features can be implemented with reference to the existing related technologies, which will not be repeated here. When it is recognized that the first image contains the target object, the boundary contour of the target object can be determined as the first location area of the target object, or the smallest rectangular frame surrounding the target object can also be determined as the first position area of the target object. Location area.

Specifically, in practical applications, the first image can be divided into N grids according to the set grid size. Therefore, assuming that the M grids contain part of the target object, the M grids can be determined. The area covered by each grid is the first location area, or it can be determined that the area covered by the M grids and the surrounding K grids is the first location area, where the M grids and the surrounding K grids are the first location area. K grids will form the smallest rectangular frame covering the target object.

In traditional focusing schemes, the focus area is often the central area of the image, that is, to ensure that the central area of the image is clear. However, if the target object in the first image is not in the central area of the first image, and the central area of the first image is the background, the traditional focusing scheme will make the background appear clear and the target object as the foreground subject is not clearly imaged , Resulting in blurring of the foreground subject in auto focus.

In order to overcome this problem, in the embodiment of the present invention, if the first photographing device can automatically select the focus area as the first location area where the target object is located, and then execute the auto-focus algorithm to make the first location area The target object achieves the clearest imaging purpose, thereby completing the focus processing for the target object.

The first imaging device selects the focus area as the first position area where the target object is located. The method is to set the image definition corresponding to the first position area in the first image to have the first weight, and the first image is divided by the first weight. The image definition corresponding to other areas outside the location area has a second weight, and the first weight is greater than the second weight.

Optionally, for example, the second weight may be set to 0, and the first weight may be set to a value greater than 0, for example, set to 1.

In practical applications, the image sharpness can be represented by the image gradient value, that is, the image sharpness of the first image is obtained by calculating the gradient of each pixel in the first image. Assuming that the above first location area contains 100 pixels, it can be considered that the gradient values of these 100 pixels are all set with the first weight. Assume that the 100 pixels contained in the first image except for the first location area There are 500 pixels in addition, then these 500 pixels are set to have the second weight.

Through the above-mentioned setting of the first weight and the second weight, the target object is highlighted, that is, the focus area is positioned on the target object.

After that, the sharpness statistical value of the first image can be calculated according to the setting results of the first weight and the second weight, so as to determine whether it is necessary to perform focus processing on the target object based on the sharpness statistical value. Because if the image of the target object is clear at this time, there is no need to perform focus processing.

In the case where the image definition is represented by the image gradient value, the statistical value of the definition of the first image is obtained according to the first weight and the second weight, which can be implemented as:

According to the first weight and the second weight, a weighted sum calculation is performed on the image gradient value corresponding to the first location area and the image gradient value corresponding to other location areas to obtain the sharpness statistical value of the first image. Taking the above 100 pixels with the first weight, the above 500 pixels with the second weight, and the first weight is 1 and the second weight is 0 as an example, the statistical value of the sharpness of the first image is 100 The sum of the gradient values of each pixel.

Because the larger the gradient value, the clearer the image. Therefore, optionally, if the statistical value of the sharpness of the first image is greater than the set threshold, it is considered that the target object is clearly imaged and does not need to be focused. On the contrary, if the first image is If the sharpness statistical value is less than the set threshold, it is considered that the target object is not sharply imaged and needs to be focused.

In addition, in some practical application scenarios, it is necessary to track and record the target object. At this time, the collected video can be segmented to obtain an image sequence composed of multiple frames of images. Assuming that the first image is the last frame of the image in the image sequence, it can also be determined according to a certain frame of image that has been focused before the first image is captured, whether it is necessary to perform focus processing on the target object when the first image is currently captured. Optionally, if the difference between the sharpness statistical value of the first image and the sharpness statistical value of the reference image is greater than a set threshold, focus processing is performed on the target object, where the reference image has been focused on the target object before Of an image.

At this time, after the focus processing is completed, the reference image may be updated with the image taken after the focus is completed, so as to perform focus processing on the image subsequently collected by the first photographing device.

In addition, in practical applications, the focusing process (adjusting the object distance) and the zooming process (focusing) always exist at the same time. In some embodiments, before performing the above focusing process, the following zooming process may also be performed:

If the first location area is not located in the central area of the screen of the first photographing device, the first location area is moved to the central area of the screen according to the relative position of the first location area and the main optical axis of the first photographing device.

During the zooming process, the image is often enlarged. Therefore, after the first image is moved so that the first position area is located in the center area of the screen, the first image can be enlarged.

Take the first camera device mounted on the drone as an example. The first camera device is mounted on the drone's pan/tilt. The first camera device can determine the relative position of the first location area to the main optical axis. Report to the pan/tilt, and the pan/tilt adjusts its pose so that during the zooming process, the first location area containing the target object in the first image is always located in the center area of the screen.

To sum up, in the embodiment of the present invention, by identifying the location area of the target object in the image, the weight of the image clarity between the location area of the target object and other areas in the image is set differently, so as to achieve focusing on the target object. Automatic selection of the area, and then when the target object needs to be focused, the object distance and image distance are adjusted to ensure that the target object in the focus area is clearly imaged to complete the accurate focus of the target object.

In the following, in combination with commonly used application scenarios of the first photographing device, the process of executing the focusing solutions provided by the embodiments of the present invention in different application scenarios will be briefly introduced. Among them, the commonly used application scenarios of the first shooting device include photographing scenes and video recording scenes.

For the photographing scene, at this time, the first image collected by the first photographing device mentioned above should be understood as the preview that the user can preview through the first photographing device before triggering the actual photographing operation on the first photographing device Each frame of the multi-frame image. For example, suppose the first shooting device is a mobile phone. When the user activates the shooting function and points the camera toward the target object (ie, the object being photographed), the image of the target object will be displayed on the screen of the mobile phone, that is, in the preview box At this time, the image presented in the preview frame can also be regarded as a kind of video, and multiple frames of images can be obtained by sampling the video, which are assumed to be three frames of images denoted as F1, F2, and F3, respectively. After the user adjusts the shooting angle, click the shooting button on the screen, and at this time, a real photo of the target object is triggered, and a photo of the target object will be taken, which is assumed to be Z1. However, the focusing process in the embodiment of the present invention occurs before the user clicks the shooting button, through the multiple frames of images obtained in the preview process-F1, F2, and F3, respectively, corresponding to the definition statistical values (the calculation method is as described above) First, focus on the target object, and then, when the user clicks the shooting button, a photo with good focus effect, namely Z1, will be obtained. Among them, if F1, F2, and F3 are the first three frames of images obtained after the user starts the shooting function, then F1 can be selected as the initial reference image, and the focus processing will be completed based on F1, F2, and F3 to obtain a suitable object distance Later, the reference image can be updated with the focused photo Z1 obtained by shooting. After the photo Z1 is taken, if the user continues to shoot the target object, then the preview video will be seen on the screen at this time, and multiple frames of images can be sampled, such as F4, F5, F6, through this preview process The multi-frame images obtained in the image-F4, F5, F6 focus again on the target object to ensure that the user clicks the shooting button again to take the photo Z2 with a good focus effect, that is, the target object is clearly imaged in the photo Z2. When the time, the reference image can be updated to the photo Z2.

In summary, in a photographing scene, a set number of images can be sampled from the previewed video stream for focusing processing, such as 3 frames, 5 frames, or even 1 frame. The processing process for each frame of the image is the same, as shown in steps 101-104. At this time, the target object will be focused on the target object in combination with the corresponding sharpness statistics of the multiple frames of images. The specific process will be described below .

For a video recording scene, at this time, the first image collected by the first shooting device mentioned above is each frame image obtained after sampling the recorded video. At this time, while recording the video, the video can be sampled to obtain a frame of image, and the statistical value of the definition of each frame of image can be calculated by the method described above. In this scene, optionally, multiple frames of images obtained by sequential sampling can also be used as a group, such as 3 frames, 5 frames, and the target object is focused on the target object with the corresponding sharpness statistics of the multiple frames in each group. . Specifically, in this scene, the reference image can be initialized to the first frame of image obtained by sampling, and it is set that the first group of images needs to be focused, so that after the first group of images is focused, you can A certain appropriate object distance is obtained. Under the condition that the position of the target object relative to the first shooting device does not change, the object distance can make the target object in the next video image captured by the first shooting device clear. Based on this, the reference image can be updated to the first image in the second group of images. When the difference between the sharpness statistics of the other images in the second group and the reference image is large, it means that it needs to be done at this time. Once the focus is processed again, a new object distance can be determined based on the second set of images, and so on.

Regardless of the above application scenarios, in summary, the execution principle of focus processing is as follows:

Determine the object distance adjustment direction of the first photographing device, where the object distance is adjusted according to the object distance adjustment direction so that the statistical value of the sharpness of the multi-frame image becomes larger. The multi-frame image includes the first image and the first image captured by the first image. At least one frame of image adjacent to the first image collected by the device. Furthermore, the object distance of the first photographing device is adjusted to the target object distance, and the target object distance corresponds to the maximum sharpness statistical value in the multiple frames of images. Wherein, the first image here may refer to any frame of image obtained by segmenting the preview video in the photographing scene, or may refer to any frame of image sampled in the recording scene.

According to the introduction of the above focusing principle, the purpose of auto focusing is to find the target object distance position. Compared with the non-target object distance position, the captured image has the largest statistical value of sharpness at the target object distance position.

For ease of understanding, the focusing process is illustrated as an example: assuming that the currently collected image is F1, calculate the sharpness statistical value corresponding to F1, and reduce the object distance of the first photographing device by a set step. Assuming that the next frame of image F2 is collected at this object distance, the sharpness statistical value of F2 is calculated. If the sharpness statistical value of F2 is greater than the sharpness statistical value of F1, it means that the adjustment direction of the smaller object distance is correct. Otherwise, it is necessary to determine the adjustment direction to increase the object distance. In this way, the object distance is gradually reduced, and a frame of image is collected at each object distance position. According to the introduction of combining multi-frame images for focus processing mentioned above, suppose that the multi-frame images collected in sequence are F1, F2, F3, F4, and that the object distances corresponding to these four frames are W1, W2, W3, respectively , W4, it can be determined that the target object distance is the maximum of these four object distances. Suppose it is W2. At this time, in fact, the sharpness statistical value corresponding to image F2 will also be the largest of the four images. Taking the above-mentioned photographing scene as an example, these four frames of images can be understood as images obtained by segmenting the preview video. Therefore, the first image in the foregoing may include these four frames of images.

The principle of the focusing process has been described above, and a specific implementation manner of performing the focusing process is provided below in conjunction with FIG. 2. As shown in FIG. 2, the focusing process may include the following steps:

201. Determine that the difference between the sharpness statistical value of the collected image P1 and the sharpness statistical value of the reference image is greater than a set threshold.

202. Drive the focus motor to move the lens in the direction of the target object to reduce the object distance.

203. Determine whether the sharpness statistical value of the K1 frame image collected during the movement of K1 set steps shows a tendency to increase, if yes, perform step 204; otherwise, perform step 205.

Among them, K1 is an integer greater than or equal to 1.

Wherein, the execution of step 202 can be understood as: the focus motor gradually reduces the object distance with a set step length, and each time it moves by a step length, one frame of image is collected. Assuming K1=2, the images collected by moving two steps are: image P2 and image P3. The sharpness statistical values of the two frames of images are respectively calculated to determine whether the sharpness statistical values of the image P1, the image P2, and the image P3 show a tendency to become larger.

If the sharpness statistics of image P1, image P2, and image P3 show a tendency to increase, it indicates that the current adjustment direction of the object distance is correct, and further adjustment of the object distance in this direction can make the image clearer. On the contrary, if the sharpness statistical values of the image P1, the image P2, and the image P3 do not show a tendency to increase, it indicates that the current adjustment direction of the object distance is incorrect, and the object distance needs to be adjusted in the opposite direction to make the image clearer.

204. Continue to drive the focus motor to move the lens K2 set steps in the direction of the target object to reduce the object distance.

205. Drive the focus motor to move the lens K2 set steps in a direction away from the target object to increase the object distance.

Among them, K2 is an integer greater than or equal to 1.

206. Determine whether the sharpness statistics of the K2 frame images and the K1 frame images collected during the movement of K2 set steps show a mountain-like change feature, if yes, perform step 207, otherwise repeat step 203.

207. Drive the focus motor to move to the object distance position corresponding to the sharpness statistical value of the peak top.

Among them, the purpose of steps 204 to 207 is to continue to adjust the object distance according to the previously determined object distance adjustment direction, so as to find the object distance position that makes the image of the target object clearest.

Specifically, one frame of image can be collected every time a set step is moved, assuming K2=3, so that the images collected by moving three steps are: image P4, image P5, and image P6. Calculate the sharpness statistics of these three frames respectively. It is determined whether the sharpness statistical values of the image P1, the image P2, the image P3, the image P4, the image P5, and the image P6 show a mountain-like characteristic, that is, whether they show a trend of gradually increasing and then gradually decreasing. If there is this trend, and assuming that the maximum value, that is, the peak top, is the sharpness statistical value of the image P3, it means that the object distance position when the image P3 is taken is the best, and shooting at this object distance position will make the image of the target object clearest , So control the focus motor to move to the object distance position.

It is worth noting that if you try to move K2 steps according to the previously determined moving direction several times, the above object distance position cannot be determined. At this time, the above image P1 can be updated with another frame of image collected after a delay of the set time. , Re-execute the above steps 201-207.

To sum up the examples of the two application scenarios, it can be seen that in actual applications, in the process of shooting the target object, because the position of the first shooting device relative to the target object is constantly changing, it is necessary to continuously focus on the target object. , In order to ensure that the target object always maintains a clear image in the captured image. Therefore, it is necessary to perform the calculation processing of the image sharpness statistical value in the embodiment shown in FIG. 1 for each frame of images collected during the shooting process, and to combine the image sharpness statistical value of the set number of images continuously collected to complete the image acquisition. The adjustment of the object distance of the first photographing device over time is the focusing process.

FIG. 3 is a schematic flowchart of another focusing method provided by an embodiment of the present invention. As shown in FIG. 3, the focusing method may include the following steps:

301. Initialize the auto focus state to focus, and initialize the statistical value of the sharpness of the reference image to a preset value.

In this embodiment, two parameters are set: auto focus state and reference image.

Among them, the auto-focus state is used to indicate whether the currently collected image needs to be subjected to auto-focus processing. Initially, this state is set to need to be focused, which means that the first captured image needs to be used for focus processing.

Wherein, in the initial situation, the reference image may be an automatically generated image, and the size of the image is equal to the size of each image actually collected subsequently. In practical applications, the weight of the gradient value of each pixel in the initial reference image (assuming that the gradient value is used as a measure of image sharpness) can be set to a set value, such as 0, so that the weight of the gradient value of all pixels The sum is the sharpness statistical value of the initial reference image. Of course, it is also possible to use the acquired first frame of image as the initial reference image, and calculate the image definition of the image, and use the image definition as the statistical value of the image definition of the reference image.

302. Determine whether there is a target object in the first image captured by the first photographing device, if it exists, execute step 303, and if it does not exist, execute step 304.

303. Set the image definition corresponding to the first location area containing the target object in the first image to have a first weight, and image definitions corresponding to other areas have a second weight, and the first weight is greater than the second weight.

For the execution process of locating the focus area to the target object through step 303, refer to the description in the foregoing embodiment.

304. Set the image definition corresponding to the central position area of the first image to have the first weight, and set the image definition corresponding to the non-central position area to have the second weight.

When the target object is not included in the first image, the focus area can still be positioned to the center position area of the first image. In practical applications, the range of the central location area can be set.

305. Obtain the sharpness statistical value of the first image according to the first weight and the second weight, and determine whether the difference between the sharpness statistical value of the first image and the sharpness statistical value of the reference image is greater than a set threshold; if it is greater, then Step 306 is executed, otherwise, step 307 is executed.

306. Determine that the auto-focus state is in need of focusing.

307. Determine that the auto focus state is that no focus is required.

When it is determined that the auto-focusing state does not require focusing, it means that the first image does not need to be subjected to focusing processing. At this time, the next image is taken, and the processing logic of steps 302 to 307 is continued to be executed for the next image. Conversely, if it is determined that the auto-focus state is in need of focusing, it means that the first image needs to be subjected to focusing processing. At this time, the focusing processing procedure described above is executed.

The description in the above embodiments is based on a single first photographing device (such as a visible light zoom camera) to realize the recognition and focus processing of the target object. In some practical application scenarios, when there is only one visible light zoom camera, if the target object to be photographed is far away from the visible light zoom camera, the visual field angle of the visible light zoom camera is generally small, so the image obtained is visually It will feel very small and the visual effect is poor. At this time, it is not conducive to quickly identifying the target object, so as to achieve fast and accurate focusing processing for the target object. To this end, the embodiment of the present invention also provides a focusing method as shown in FIG. 3.

FIG. 4 is a schematic flowchart of another focusing method provided by an embodiment of the present invention. The focusing method is executed by the first photographing device described above. As shown in FIG. 4, the focusing method may include the following steps:

401. Identify a target object in a second image collected by a second photographing device.

402. According to the second location area corresponding to the target object in the second image and the coordinate system mapping relationship between the first shooting device and the second shooting device, determine the first image corresponding to the target object in the first image captured by the first shooting device. Location area, where the shooting time of the first image and the second image are the same.

403. Set the image definition corresponding to the first position area in the first image to have the first weight, and the image definition corresponding to the areas other than the first position area in the first image has the second weight, and the first weight is greater than The second weight.

404. Acquire a sharpness statistical value of the first image according to the first weight and the second weight.

405. If the sharpness statistical value of the first image meets the set focusing condition, perform focusing processing on the target object.

In this embodiment, the angle of view of the first photographing device is smaller than the angle of view of the second photographing device. In practical applications, the second photographing device may be an infrared camera, or the second photographing device may also be a visible light wide-angle camera.

For ease of understanding, an application scenario shown in FIG. 5 is taken as an example for illustration.

In FIG. 5, it is assumed that the first photographing device is a visible light zoom camera with a field of view angle of FOV1, and the second photographing device is an infrared camera with a field of view angle of FOV2, and FOV1 is smaller than FOV2. In practical applications, both the first photographing device and the second photographing device can be mounted on the drone. In addition, the image coordinate mapping relationship of the two shooting devices has been determined in advance according to the shooting parameters of the first shooting device and the shooting parameters of the second shooting device.

Therefore, at the same moment, both the first shooting device and the second shooting device shoot an image of the target object, the image collected by the first shooting device is called the first image, and the image collected by the second shooting device is called the second image . Since the second photographing device has a larger field of view, the data content contained in the second image is much more than that of the first image, so that the target object can be captured more quickly by the second photographing device.

Different features can be adopted according to the different types of the second photographing device to identify the target object in the second image.

For example, when the second photographing device is an infrared camera, the target object can be identified in the second image according to the temperature characteristic of the target object. The temperature value ranges corresponding to different types of target objects are different, according to which the target objects can be identified.

For another example, when the second photographing device is a visible light wide-angle camera, the target object can be identified in the second image according to the visible light feature of the target object. The meaning of the visible light feature can be referred to the above description.

When it is recognized that the target object is contained in the second image, the second location area corresponding to the target object in the second image is determined, and then, according to the image coordinate mapping relationship between the two shooting devices, it can be determined that the target object is in the first image For the first position area of the first image, the subsequent focus processing steps are performed on the first image. Refer to the introduction in the foregoing other embodiments.

FIG. 6 is a schematic structural diagram of a focusing device provided by an embodiment of the present invention. The focusing device may be provided in the first photographing device mentioned above. As shown in FIG. 6, the focusing device includes: a memory 11 and a processor 12; The memory 11 stores executable code, and when the executable code is executed by the processor 12, the processor 12 is enabled to implement:

Optionally, the image definition includes an image gradient value. Therefore, in the process of obtaining the definition statistical value of the first image, the processor 12 is specifically configured to: according to the first weight and the second weight, the image corresponding to the first location area A weighted sum calculation is performed between the gradient value and the image gradient value corresponding to the other location area to obtain the sharpness statistical value of the first image.

Optionally, the processor 12 is specifically configured to: if the difference between the sharpness statistical value of the first image and the sharpness statistical value of the reference image is greater than a set threshold, perform focusing processing on the target object, so The reference image is an image that has been in focus.

Optionally, the processor 12 is further configured to update the reference image with an image taken after focusing.

Optionally, the second weight is set to zero.

Optionally, the processor 12 is further configured to: if the first location area is not located in the central area of the frame of the first photographing device, then according to the first location area and the main image of the first photographing device The relative position of the optical axis moves the first position area to the center area of the screen.

Optionally, in the process of determining the first location area corresponding to the target object in the first image captured by the first photographing device, the processor 12 is specifically configured to: The target object is identified in the image; the target is determined according to the second location area corresponding to the target object in the second image and the image coordinate system mapping relationship between the first photographing device and the second photographing device The first location area corresponding to the object in the first image; wherein the shooting time of the first image and the second image are the same.

Optionally, the angle of view of the first photographing device is smaller than that of the second photographing device.

Optionally, the first photographing device is a visible light zoom camera, and the second photographing device is an infrared camera or a visible light wide-angle camera.

Optionally, when the second photographing device is an infrared camera, the processor 12 is specifically configured to: identify the target object in the second image according to the temperature characteristic of the target object.

Optionally, when the second photographing device is a visible light wide-angle camera, the processor 12 is specifically configured to: identify the target object in the second image according to the visible light characteristic of the target object.

Optionally, in the process of focusing processing on the target object, the processor 12 is specifically configured to: determine the object distance adjustment direction of the first photographing device, wherein the object distance is adjusted according to the object distance adjustment direction such that The sharpness statistical value of a multi-frame image shows an increasing trend, and the multi-frame image includes the first image and at least one frame of image adjacent to the first image collected by the first photographing device; The object distance of the first photographing device is adjusted to a target object distance, and the target object distance corresponds to the maximum sharpness statistical value in the multiple frames of images.

FIG. 7 is a schematic structural diagram of a photographing device provided by an embodiment of the present invention. As shown in FIG. 7, the photographing device includes:

The lens assembly 21, the sensor module 22, and the focusing device 23 as shown in FIG. 6.

Among them, the lens assembly 21 is arranged inside the housing of the photographing device. The focusing device 23 is arranged inside the housing. The sensor module 22 is arranged inside the housing and at the rear end of the lens assembly 21. The sensor module 22 includes a circuit board and an imaging sensor. The imaging sensor is arranged on the front surface of the circuit board facing the lens assembly 21.

It is understandable that the image collected by the photographing device will be imaged by the aforementioned imaging sensor, and the focusing device 23 is used for focusing processing on the target object contained in the image collected by the photographing device. The photographing device corresponds to the first photographing device described above. For the execution process of the focusing device 23, refer to the introduction in the foregoing embodiment, which is not repeated here.

FIG. 8 is a schematic structural diagram of a movable platform provided by an embodiment of the present invention. In FIG. 8, the movable platform is implemented as a drone as an example. Of course, the movable platform can also be implemented as a handheld PTZ, PTZ vehicles, electric cars, electric bicycles, etc.

As shown in FIG. 8, the movable platform includes: a body 31, a power system 32 provided on the body 31, and a first photographing device 33 provided on the body 31.

Among them, the power system 32 is used to provide power for the movable platform.

Wherein, the first photographing device 33 is a photographing device as shown in FIG. 7, which is used to photograph a first image and perform focusing processing on a target object in the first image.

Optionally, the movable platform may further include: a second photographing device 34 arranged on the body 31. The angle of view of the first photographing device 33 is smaller than that of the second photographing device 34.

Wherein, the second photographing device 34 is used for photographing a second image containing the target object, and transmitting the second image to the first photographing device 33, so that the first photographing device 33 can use the target object in the second image according to the target object. The corresponding location area in the image determines the location area corresponding to the target object in the first image.

For the functions and working processes of the first photographing device and the second photographing device, reference may be made to the introduction in the foregoing other embodiments, which will not be repeated here.

In addition, when the movable platform is implemented as an unmanned aerial vehicle, as shown in FIG. 8, the unmanned aerial vehicle may also include a pan/tilt 35 arranged on the body 31, so that the first photographing device 33 and the second photographing device 34 may be set On the pan-tilt 35, the first photographing device 33 and the second photographing device 34 can move relative to the body through the pan-tilt 35.

The power system 32 of the drone may include an electronic governor, one or more rotors, and one or more motors corresponding to the one or more rotors.

Other devices such as an inertial measurement unit (not shown in the figure) can also be provided on the drone, which will not be listed here.

In addition, an embodiment of the present invention also provides a computer-readable storage medium having executable code stored in the computer-readable storage medium, and the executable code is used to implement the focusing method provided in the foregoing embodiments.

The technical solutions and technical features in each of the above embodiments can be singly or combined without conflict, as long as they do not exceed the cognitive scope of those skilled in the art, they all belong to equivalent embodiments within the protection scope of the present application.

The above are only the embodiments of the present invention, which do not limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the description and drawings of the present invention, or directly or indirectly applied to other related technologies In the same way, all fields are included in the scope of patent protection of the present invention.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be equivalently replaced; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention. range.

Claims

A focusing method, characterized in that the method includes:

Determine a first location area corresponding to a target object in a first image captured by a first photographing device; wherein the first location area is an area that covers the target object and is smaller than the first image;

Set the image definition corresponding to the first position area in the first image to have a first weight; wherein, the image definition corresponding to the areas other than the first position area in the first image has A second weight, where the first weight is greater than the second weight;

Acquiring a sharpness statistical value of the first image according to the first weight and the second weight;

If the sharpness statistical value of the first image meets the set focus condition, then focus processing is performed on the target object.
The method according to claim 1, wherein the image definition includes an image gradient value;

The acquiring the sharpness statistical value of the first image according to the first weight and the second weight includes:

Perform a weighted sum calculation on the image gradient value corresponding to the first location area and the image gradient value corresponding to the other location area according to the first weight and the second weight to obtain the first image The clarity statistics.
The method according to claim 1, wherein if the statistical value of the sharpness of the first image meets a set focusing condition, performing focusing processing on the target object comprises:

If the difference between the sharpness statistical value of the first image and the sharpness statistical value of the reference image is greater than the set threshold, then focus processing is performed on the target object, and the reference image is an image that has been focused.
The method according to claim 3, wherein the method further comprises:

The reference image is updated with the image taken after focusing.
The method according to claim 1, wherein the second weight is set to zero.
The method according to claim 1, wherein the method further comprises:

If the first position area is not located in the central area of the screen of the first photographing device, move the first position area according to the relative position of the first position area and the main optical axis of the first photographing device To the central area of the screen.
The method according to any one of claims 1 to 6, wherein the determining the first location area corresponding to the target object in the first image captured by the first photographing device comprises:

Identifying the target object in the second image collected by the second photographing device;

According to the second location area corresponding to the target object in the second image and the image coordinate system mapping relationship between the first photographing device and the second photographing device, it is determined that the target object corresponds to the first image The first location area;

Wherein, the shooting time of the first image and the second image are the same.
The method according to claim 7, wherein the angle of view of the first photographing device is smaller than that of the second photographing device.
The method according to claim 7, wherein the first photographing device is a visible light zoom camera, and the second photographing device is an infrared camera or a visible light wide-angle camera.
The method according to claim 9, wherein the second photographing device is an infrared camera, and the recognizing the target object in the second image collected by the second photographing device comprises:

The target object is identified in the second image according to the temperature characteristic of the target object.
The method of claim 9, wherein the second photographing device is a visible light wide-angle camera, and the recognizing the target object in the second image collected by the second photographing device comprises:

The target object is identified in the second image according to the visible light characteristic of the target object.
The method according to claim 1, wherein the performing focus processing on the target object comprises:

The object distance adjustment direction of the first photographing device is determined, wherein the object distance is adjusted according to the object distance adjustment direction so that the statistical value of the sharpness of a multi-frame image tends to become larger, and the multi-frame image includes the first An image and at least one frame of image adjacent to the first image collected by the first photographing device;

The object distance of the first photographing device is adjusted to a target object distance, and the target object distance corresponds to the maximum sharpness statistical value in the multiple frames of images.
A focusing device, characterized in that it is provided in a first photographing device, the focusing device includes: a memory and a processor; wherein executable code is stored in the memory, and when the executable code is used by the processor When executed, the processor is made to realize:

Determining a first location area corresponding to a target object in a first image captured by the first photographing device; wherein the first location area is an area that covers the target object and is smaller than the first image;

Set the image definition corresponding to the first position area in the first image to have a first weight; wherein, the image definition corresponding to the areas other than the first position area in the first image has A second weight, where the first weight is greater than the second weight;

Acquiring a sharpness statistical value of the first image according to the first weight and the second weight;

If the sharpness statistical value of the first image meets the set focus condition, then focus processing is performed on the target object.
The focusing device according to claim 13, wherein the image sharpness comprises an image gradient value;

The processor is specifically configured to: perform a weighted sum calculation on the image gradient value corresponding to the first location area and the image gradient value corresponding to the other location areas according to the first weight and the second weight , In order to obtain the sharpness statistical value of the first image.
The focusing device according to claim 13, wherein the processor is specifically configured to: if the difference between the sharpness statistical value of the first image and the sharpness statistical value of the reference image is greater than a set threshold, The target object is subjected to focusing processing, and the reference image is an image that has been focused.
The focusing device according to claim 15, wherein the processor is further configured to update the reference image with an image taken after focusing.
The focusing device according to claim 13, wherein the second weight is set to zero.
The focusing device according to claim 13, wherein the processor is further configured to: if the first location area is not located in the central area of the frame of the first photographing device, then according to the first location area The relative position with the main optical axis of the first photographing device moves the first position area to the center area of the screen.
The focusing device according to any one of claims 13 to 18, wherein the processor is specifically configured to:

Identify the target object in the second image captured by the second photographing device; according to the second location area corresponding to the target object in the second image and the first photographing device and the second photographing device The mapping relationship of the image coordinate system is determined to determine the first location area corresponding to the target object in the first image; wherein the shooting time of the first image and the second image are the same.
The focusing device according to claim 19, wherein the angle of view of the first photographing device is smaller than that of the second photographing device.
The focusing device according to claim 19, wherein the first photographing device is a visible light zoom camera, and the second photographing device is an infrared camera or a visible light wide-angle camera.
The focusing device according to claim 21, wherein the second photographing device is an infrared camera, and the processor is specifically configured to:

The target object is identified in the second image according to the temperature characteristic of the target object.
The focusing device according to claim 21, wherein the second photographing device is a visible light wide-angle camera, and the processor is specifically configured to:

The target object is identified in the second image according to the visible light characteristic of the target object.
The focusing device according to claim 13, wherein the processor is specifically configured to: determine an object distance adjustment direction of the first photographing device, wherein the object distance is adjusted according to the object distance adjustment direction so that multiple frames The sharpness statistical value of the image has a tendency to increase, and the multi-frame image includes the first image and at least one frame of image adjacent to the first image collected by the first photographing device; The object distance of the first photographing device is adjusted to the target object distance, and the target object distance corresponds to the maximum sharpness statistical value in the multiple frames of images.
A photographing equipment, characterized in that it comprises:

The lens assembly is arranged inside the housing of the photographing device;

The sensor module is arranged inside the housing and at the rear end of the lens assembly, the sensor module includes a circuit board and an imaging sensor, and the imaging sensor is arranged on the front surface of the circuit board facing the lens assembly ；

The focusing device according to any one of claims 13 to 24, which is provided inside the housing.
A movable platform, characterized in that it comprises:

Body

The power system is arranged on the body and used to provide power for the movable platform;

The photographing device according to claim 25, which is arranged on the body and is used to photograph a first image and perform focus processing on a target object in the first image.
The movable platform according to claim 26, further comprising:

Another photographing device is arranged on the body and used to photograph a second image containing the target object, and transmit the second image to the photographing device, so that the photographing device can use it according to the target object The corresponding location area in the second image determines the location area corresponding to the target object in the first image.
The movable platform according to claim 27, wherein the angle of view of the photographing device is smaller than that of the another photographing device.
A computer-readable storage medium, wherein executable code is stored in the computer-readable storage medium, and the executable code is used to implement the focusing method according to any one of claims 1 to 12.