WO2021026754A1

WO2021026754A1 - Focus control method and apparatus for photography apparatus, and unmanned aircraft

Info

Publication number: WO2021026754A1
Application number: PCT/CN2019/100344
Authority: WO
Inventors: 吴博; 钱杰
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-08-13
Filing date: 2019-08-13
Publication date: 2021-02-18
Also published as: CN112154650A

Abstract

Provided are a focus control method for a photography apparatus (101), a focus control apparatus (800) for the photography apparatus (101), and an unmanned aircraft. The photography apparatus (101) is mounted on the unmanned aircraft (10); the focus control apparatus (800) for the photography apparatus (101) comprises a memory (802), a processor (801), and a computer program stored in the memory (802) and capable of being run on the processor (801). The focus control method for the photography apparatus (101) comprises: acquiring a three-dimensional image of the surrounding environment of the unmanned aircraft (10); determining a focus target and focus parameters of the photography apparatus (101) at least partially on the basis of the three-dimensional image, the focus parameters comprising: a target distance between the focus target and the photography apparatus (101); and, on the basis of the focus parameters, adjusting the focal distance of the photography apparatus (101). The present focus control method can accurately and rapidly calculate the focus target and focus parameters and implement automatic focusing.

Description

Focus control method, device and unmanned aerial vehicle of photographing device

Technical field

This application relates to the field of imaging technology, and in particular to a focus control method and device of a photographing device, and an unmanned aerial vehicle.

Background technique

At present, shooting devices generally have an auto focus (Auto Focus, referred to as “AF”) function. In related technologies, SLR cameras usually use a phase difference detection method for focusing. However, in actual shooting, repeated focusing and The situation where it is difficult to focus results in unclear imaging of the shooting target and poor user experience.

Summary of the invention

In view of this, one of the objectives of the present invention is to provide a focus control method, device and unmanned aerial vehicle for a shooting device, so as to at least achieve the purpose of accurately determining the focus parameter of the shooting device and using the focus parameter to perform autofocus.

In a first aspect, an embodiment of the present invention provides a focus control method of a photographing device installed on an unmanned aerial vehicle, and the method includes:

Acquiring a three-dimensional map of the surrounding environment where the unmanned aerial vehicle is located;

Determining a focus target and focus parameters of the shooting device at least partly according to the three-dimensional map, the focus parameters including: a target distance between the focus target and the shooting device;

The focal length of the photographing device is adjusted according to the focus parameter.

In the second aspect, an embodiment of the present invention provides a focus control device for a photographing device, including a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor executes the program when the program is executed. :

In a third aspect, an embodiment of the present invention provides an unmanned aerial vehicle, the unmanned aerial vehicle comprising: a photographing device and the focus control device of the photographing device as described in the second aspect.

The focus control method, device, and unmanned aerial vehicle of a photographing device provided by the embodiments of the present invention obtain a three-dimensional map of the surrounding environment of the unmanned aerial vehicle, and determine the focus target and focus of the photographing device at least partially based on the three-dimensional map. Parameters, and then adjust the focal length of the shooting device according to the focus parameters to control the shooting device to perform auto-focusing. The present invention can accurately calculate the focus parameters and realize auto-focusing, and has simple and efficient positive effects.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the present invention;

2 is a schematic flowchart of a focus control method of a photographing device according to an embodiment of the present invention;

3 is a schematic diagram of a process of obtaining a three-dimensional map of the surrounding environment where the camera is located according to an embodiment of the present invention;

4 is a schematic diagram of a process for determining the focus parameters of a shooting device according to a three-dimensional map according to an embodiment of the present invention;

Fig. 5 is a schematic diagram of posture information of a photographing device provided by an embodiment of the present invention;

Figure 6 is a schematic diagram of a designated area provided by an embodiment of the present invention;

FIG. 7 is a schematic diagram of a three-dimensional map provided by an embodiment of the present invention;

8 is a schematic structural diagram of a focus control device of a photographing device according to an embodiment of the present invention;

Figure 9 is a schematic structural diagram of an unmanned aerial vehicle provided by an embodiment of the present invention;

Fig. 10 is a schematic structural diagram of another unmanned aerial vehicle provided by an embodiment of the present invention.

detailed description

In order to make the objectives, technical solutions and advantages of the present invention more obvious, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments of the present invention. It should be understood that the present invention is not limited by the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the protection scope of the present invention.

In the following description, a lot of specific details are given in order to provide a more thorough understanding of the present invention. However, it is obvious to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features known in the art are not described.

It should be understood that the present invention can be implemented in different forms and should not be interpreted as being limited to the embodiments presented here. On the contrary, the provision of these embodiments will make the disclosure thorough and complete, and will fully convey the scope of the present invention to those skilled in the art.

The purpose of the terms used here is only to describe specific embodiments and not as a limitation of the present invention. When used herein, the singular forms of "a", "an" and "the/the" are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "composition" and/or "including", when used in this specification, determine the existence of the described features, integers, steps, operations, elements and/or components, but do not exclude one or more other The existence or addition of features, integers, steps, operations, elements, parts, and/or groups. As used herein, the term "and/or" includes any and all combinations of related listed items.

Focusing refers to adjusting the distance of the focus to make the photo clear. The current focus methods include manual focus and auto focus. Manual focus is a focusing method that adjusts the distance between the lens groups of the camera lens by manually turning the focusing ring on the lens to make the image clear. This focusing method largely depends on the photographer's clarity of the image in the viewfinder Judgment. Current SLR cameras usually use a phase difference detection method of focusing scheme. However, in the case of low light and insufficient light, the focusing speed and performance of this scheme will be significantly reduced. In actual shooting, there will be repeated focusing and difficult focusing conditions, resulting in shooting The image of the target is not clear, and the user experience is poor.

Based on this, the embodiments of the present invention provide a focus control method, device, and unmanned aerial vehicle for a photographing device to combine the application scenarios of the unmanned aerial vehicle to achieve a simple and efficient autofocus effect.

The focus control method, device, and movable platform of the photographing device of the present application will be described in detail below with reference to the accompanying drawings. In the case of no conflict, the following embodiments and features in the implementation can be combined with each other.

Please refer to FIG. 1, which is a schematic diagram of an application scenario provided by an embodiment of the present invention. The UAV 10 includes a photographing device 101 and a depth sensor. The depth sensor may be a binocular vision sensor (including a first camera 102A and a second camera). Second camera 102B). The binocular vision sensor can obtain the depth map of the surrounding environment of the unmanned aerial vehicle 10, and the positioning device and the attitude sensor (not shown) of the unmanned aerial vehicle 10 can obtain the position information and attitude information of the binocular vision sensor. According to the depth map and the position information and attitude information of the binocular vision sensor, a three-dimensional map of the surrounding environment of the unmanned aerial vehicle 10 is obtained. When determining the focus target and focus parameters of the imaging device 101, the optical center of the imaging device can be used as the starting point to form one or more direction vectors in the central area of the field of view (FOV) of the imaging device. Counting the area passed by one or more direction vectors in the map can obtain the focus target and the depth information corresponding to the focus target. Thus, the focal length of the photographing device 101 can be automatically adjusted according to the depth information corresponding to the focus target to achieve automatic focusing. In this embodiment, an object located in the center area of the screen in the image captured by the camera can always be clearly imaged.

Optionally, the number of binocular vision sensors is one or more, so as to generate a three-dimensional map within a certain radius around the unmanned aerial vehicle in real time during the movement of the unmanned aerial vehicle. When the number of binocular vision sensors is multiple, multiple binocular vision sensors can be installed on the front, rear, left, and right sides of the UAV, respectively.

Since existing unmanned aerial vehicles are generally equipped with binocular vision sensors to realize automatic obstacle avoidance and other functions, the embodiment of the present invention uses binocular vision sensors to generate three-dimensional maps without adding additional hardware structures such as other sensors. The focus target is accurately determined, and the focal length of the shooting device is adjusted according to the depth information of the focus target to achieve automatic focusing, which has simple and efficient positive effects.

2 is a schematic flowchart of a focus control method of a photographing device according to an embodiment of the present invention. The camera is installed in an unmanned aerial vehicle. The focus control method of the shooting device is applied to the focus control device of the shooting device. The focus control device of the shooting device may be installed inside the shooting device, or may be independently installed outside the shooting device, or partly installed inside the shooting device , A part is independently installed outside the camera, for example, a part is installed in the body of an unmanned aerial vehicle, and a part is installed in the camera. Referring to FIG. 2, the method includes the following steps S201-S203:

S201. Acquire a three-dimensional map of the surrounding environment where the unmanned aerial vehicle is located.

In an embodiment, the three-dimensional map of the surrounding environment where the photographing device is located may be pre-established, and the three-dimensional map contains three-dimensional information of the surrounding environment where the photographing device is located. For example, the three-dimensional information may be expressed in longitude, latitude, and altitude. Optionally, the three-dimensional map can be stored in a cloud processor. During the execution of this step, the focus control device of the camera can acquire a three-dimensional map within a certain radius centered on the unmanned aerial vehicle in real time.

In an embodiment, the three-dimensional map of the surrounding environment where the aforementioned camera is located may be updated in real time based on the depth sensor on the unmanned aerial vehicle. The depth sensor can be a binocular vision sensor, an ultrasonic sensor, a millimeter wave radar sensor, and a lidar sensor.

Fig. 3 is a schematic flowchart of obtaining a three-dimensional map of the surrounding environment of an unmanned aerial vehicle according to an embodiment of the present invention. As shown in FIG. 3, in the embodiment of the present invention, the foregoing acquiring a three-dimensional map of the surrounding environment where the photographing device is located specifically includes the following steps S301-S303:

S301. Obtain a depth map of the surrounding environment where the UAV is located, where the depth map is detected by a depth sensor on the UAV.

Optionally, the aforementioned depth sensor is a binocular vision sensor. By using the binocular vision sensor for distance measurement, the three-dimensional coordinates of the object can be quickly calculated and the three-dimensional space can be reconstructed. It has the advantages of high flexibility and high accuracy.

S302. Acquire the position and posture of the depth sensor.

Optionally, in this embodiment, the position of the depth sensor may be acquired through a positioning device, such as a GPS module, and the attitude of the depth sensor may be acquired through an attitude sensor, such as an inertial measurement unit IMU. The GPS module and the IMU are installed in an unmanned aerial vehicle.

S303. Obtain a three-dimensional map of the surrounding environment where the unmanned aerial vehicle is located according to the depth map and the position and attitude of the depth sensor.

Optionally, multiple depth sensors are installed on the unmanned aerial vehicle, and depth images can be generated in real time through each depth sensor. After obtaining the position and attitude of each depth sensor, the relative position relationship between the depth sensors can be further obtained. The coordinate conversion relationship between the depth images generated by each depth sensor can be determined. Then, according to the coordinate conversion relationship, the depth images generated by the depth sensors are image-spliced to obtain a three-dimensional map of the surrounding environment where the camera is located.

Optionally, a single depth sensor is installed on the unmanned aerial vehicle. The depth sensor can generate multiple depth images in real time during the movement of the unmanned aerial vehicle. After obtaining the positions and attitudes of the depth sensors corresponding to the multiple depth images, it can Determine the coordinate conversion relationship between the multiple depth images generated by the depth sensor. Then, according to the coordinate conversion relationship, multiple depth images generated by the depth sensor are image-spliced to obtain a three-dimensional map of the surrounding environment where the camera is located.

S202: Determine a focus target and focus parameters of the shooting device at least partially according to the three-dimensional map, where the focus parameters at least include: a target distance between the focus target and the shooting device.

The focus target can be determined from the above-mentioned map, and the target distance between the focus target and the shooting device can be obtained according to the depth information of the focus target.

S203. Adjust the focal length of the photographing device according to the focus parameter.

In this embodiment, after determining the target distance between the focus target and the shooting device, the focal length of the shooting device can be adjusted according to the target distance to control the shooting device to perform automatic focusing, so that the focus target can form a clear image.

The focus control method of the shooting device provided in the above-mentioned embodiment of the present invention obtains a three-dimensional map of the surrounding environment of the shooting device, determines the focus target and focus parameters of the shooting device from the three-dimensional map, and determines the focus target and focus parameters of the shooting device according to the determined focus parameters Control the camera to auto focus, which has a positive effect of accuracy and efficiency.

In an embodiment of the present invention, the foregoing acquiring a three-dimensional map of the surrounding environment where the photographing device is located includes:

Acquire a three-dimensional map within a preset range in the surrounding environment where the camera is located.

For the existing shooting device, when the distance between the shooting target and the shooting device is greater than a certain distance, the focal length of the shooting device can be set to infinity; for example, when the focus of the shooting scene is between the focus target and the shooting device When the distance is greater than 16 meters, the focal length of the camera can be set to infinity.

Therefore, in this embodiment, the acquired three-dimensional map in the surrounding environment of the camera is a three-dimensional map within a preset range. The preset range may be centered on the carrier of the camera with a radius of 16 meters or 32 meters. Or other distance ranges. It is understandable that setting the range of the three-dimensional map can reduce the amount of data storage and increase the processing speed.

Furthermore, in an optional embodiment of the present invention, when the focus target is not determined from the three-dimensional map, the camera can be controlled to adjust the focus to infinity.

Fig. 4 is a schematic diagram of a process for determining the focus parameters of a shooting device according to a three-dimensional map according to an embodiment of the present invention. As shown in FIG. 4, in the embodiment of the present invention, determining the focus parameter of the shooting device according to the three-dimensional map includes the following steps S401-S403:

S401. Acquire position information and posture information of the photographing device.

Optionally, a positioning device, such as a GPS module, is installed on the unmanned aerial vehicle. The positioning device can obtain the position information of the unmanned aerial vehicle in real time. Since the relative position relationship between the camera and the positioning device can be obtained in advance, The location information of the shooting device is obtained according to the location information obtained by the positioning device. Optionally, the position information of the photographing device is position information of the optical center of the photographing device.

Referring to FIG. 5, in an embodiment of the present invention, the posture information of the aforementioned camera includes: the exit direction of the FOV center line (the dotted line shown in the figure) of the field of view of the camera 1011.

It should be noted that the posture information of the camera is not limited to the above representation. In an optional embodiment of the present invention, the camera is installed on the unmanned aerial vehicle through a three-axis platform, and the posture angle of the three-axis platform can also be used. Indicates the posture of the camera.

Furthermore, according to the acquired position information and posture information, one or more direction vectors within the field of view of the camera can be determined.

S402: Determine a focus target according to the position information and posture information of the shooting device and the three-dimensional map.

In the embodiment of the present invention, based on the above-mentioned position information and posture information, the focus target can be located from the three-dimensional map.

S403. Determine the depth information of the focus target as the target distance between the focus target and the shooting device.

After the focus target is determined from the three-dimensional map, the depth information of the focus target can be further obtained, and the depth information is determined as the target distance between the focus target and the shooting device.

In an embodiment of the present invention, determining the focus target according to the position information and the posture information and the three-dimensional map includes the following steps A10-A20:

Step A10: Determine one or more direction vectors according to the position information and posture information of the photographing device. After obtaining the position information and posture information of the shooting device, the optical center of the shooting device can be used as the starting point, and one or more direction vectors are formed in the central area of the field of view of the shooting device to search for the focus target in the central area . Please refer to FIG. 6, the central area may be a cylindrical area 600 extending in the direction of the central axis of the photographing device with the optical center of the photographing device as the center and the predetermined value as the radius. The center area of the image.

The above-mentioned direction vector is a vector in the cylindrical region 600 that represents the ray emitted from the optical center of the camera under the designated coordinate system, and the vector may be a unit vector. The above-mentioned designated coordinate system may be a three-dimensional coordinate system formed by the optical center of the camera as the origin O, the true east direction as the X axis direction, the true north direction as the Y axis direction, and the sky pointing direction as the Z axis direction. Optionally, the designated coordinate system may also be another coordinate system.

Please refer to FIG. 7. FIG. 7 is a schematic diagram of a three-dimensional map provided by an embodiment of the present invention. Optionally, the three-dimensional map and the above-mentioned direction vector are in the same coordinate system, or in different coordinate systems and have certain Coordinate conversion relationship.

Step A20: Determine a focus target of the shooting device according to the direction vector and the three-dimensional map.

In an embodiment of the present invention, a target location point is determined from the three-dimensional map according to the above-mentioned direction vector and a three-dimensional map, the target area where the target location point is located is identified, and when the target area corresponding to the target location point satisfies When the conditions are preset, the target position point is determined as the focus target.

In an embodiment of the present invention, the target area corresponding to the target location point meets a preset condition, including:

The area occupied by the target area is greater than a preset threshold and/or the location of the target area is located in the preset location area.

Optionally, the area occupied by the target area may be the area occupied by the target area in the image after imaging. Optionally, the area occupied by the target area may be the ratio of the area occupied by the target area in the image after imaging to the entire image area. Optionally, the area occupied by the target area may be a physical area corresponding to the target area. Optionally, the area occupied by the target area may be a ratio of the physical area corresponding to the target area to the preset area. The predetermined area may be the area of the circular cross-section of the cylindrical region 600.

Noise points may exist in the three-dimensional map of the surrounding environment where the camera is located. These noise points are usually small. These noise points can be eliminated by determining whether the area occupied by the target area is greater than a preset threshold, and the accuracy of focusing can be improved.

Optionally, the area where the target area is located at the preset position may be the central area of the image after the target area is imaged, and the central area may be a rectangular area, a square area or a circular area. Optionally, the location where the target area is located at a preset location area may be a central area of the target area within the field of view of the camera in the three-dimensional space, and the central area may be a rectangular parallelepiped area, a cubic area, or a cylindrical area. Area or spherical area. Please refer to FIG. 6, the predetermined location area may be a cylindrical area 601 with a certain distance from the camera. Optionally, the radius of the cylindrical area 601 may be the same as or different from that of the cylindrical area 600. The cylindrical area 601 is coaxial with the cylindrical area 600. The cylindrical area 601 may also be replaced with a cuboid, a cube, a sphere, etc. Other shapes.

Generally, the focus target is in the center area of the screen, and the target area at the edge of the screen can be filtered out by determining whether the location of the target area is located in the preset location area. In addition, the focusing distance of the camera is usually limited. For example, the closest focusing distance of some camera is 1.5 meters, that is, when the target area is too close to the camera, the camera cannot focus. In this way, you can change the preset The location area is set at a certain distance from the camera to avoid inability to focus.

In the embodiment of the present invention, the target location that satisfies that the area occupied by the target area is greater than the preset threshold and/or the location of the target area is located in the preset location area is used as the focus target, which can improve the accuracy of focus and avoid inability The focus situation.

In an embodiment of the present invention, when the number of focus targets is multiple, adjusting the focal length of the photographing device according to the focus parameter includes the following steps B10-B30:

Step B10: For each focus target, respectively acquire images collected by the photographing device according to the focus parameters corresponding to the focus target.

In this embodiment, for each focus target, the distance between the focus target and the shooting device is obtained from the map, and the shooting device is controlled to use the distance corresponding to each focus target to perform automatic focusing and perform image acquisition.

Step B20: Compare all images to determine the target image.

Compare all the images collected above to obtain the target image with the best imaging quality. Optionally, the target image with the best imaging quality can be obtained according to the contrast information of all images. Optionally, all images can be input to a pre-trained model to obtain a target image with the best imaging quality.

Step B30: Control the photographing device to perform automatic focusing according to the focusing parameter corresponding to the target image.

The distance corresponding to the target image with the best imaging quality obtained above is determined as the target distance for controlling the camera to perform autofocus, and the focal length of the camera is adjusted according to the target distance to achieve autofocus. Furthermore, the focus of the camera is more accurate.

In another embodiment of the present invention, the above-mentioned focusing parameters further include: the target distance corresponds to state information of the photographing device, and the state information of the photographing device includes posture information and position information.

In the embodiment of the present invention, after the above-mentioned determination of the focus target and focus parameters of the shooting device at least partially based on the three-dimensional map, the above-mentioned method further includes the following step C10:

Step C10: Associate and store the target distance and the state information of the photographing device corresponding to the target distance.

The above-mentioned associated storage method may be corresponding storage in a table manner.

Then, the state information of the camera is acquired. When it is determined that the state of the camera is the state represented by the state information of the camera, the stored target distance corresponding to the state information is acquired, and the camera is controlled according to the target distance. auto focus.

For example, in a scene where a drone is used for power inspection work or a scene where a surveying drone is used for surveying and mapping work, the flight trajectory of the drone is a pre-set fixed trajectory. Select multiple designated position points in the, and set the posture of the camera at each designated position point, and then determine in advance according to the posture information of the camera at one or more positions and the position information of the position point. The focus parameter of the camera for focusing when the human-machine flies to the position point, so that the camera can achieve rapid focusing when the position point is reached.

Furthermore, in the embodiment of the present invention, it is possible to obtain in advance the state information that characterizes the shooting device at a later time, and to determine the focus target of the shooting device at a later time from the three-dimensional map of the surrounding environment where the shooting device is located, and then obtain the The target distance between the focus target and the image capturing device is associated and stored with the above-mentioned state information. After determining that the state of the capturing device is consistent with the state represented by the above-mentioned state information, the target corresponding to the state information is obtained Distance, adjust the focal length of the photographing device according to the target distance to control the photographing device to perform auto-focusing; further, it is possible to pre-determine the focus parameters before the state of the photographing device reaches the state at a certain moment, which can improve the focal length of the photographing device The efficiency of regulation.

FIG. 8 is a schematic structural diagram of a focus control device of a photographing device provided in an embodiment of the present invention. Referring to FIG. 8, the focus control device 800 of the photographing device includes a memory 802, a processor 801, and a computer program stored on the memory 802 and running on the processor 801. The processor 801 implements :

Acquiring a three-dimensional map of the surrounding environment where the photographing device is located;

Optionally, the foregoing processor 801 implements the following when executing the program:

Acquiring a depth map of the surrounding environment where the unmanned aerial vehicle is located, the depth map being detected by a depth sensor on the unmanned aerial vehicle;

Acquiring the position and attitude of the depth sensor;

According to the depth map and the position and attitude of the depth sensor, a three-dimensional map of the surrounding environment of the unmanned aerial vehicle is obtained.

Optionally, the above-mentioned depth sensor includes a binocular vision sensor; when the processor executes the program:

Acquire a three-dimensional map within a preset range in the surrounding environment where the unmanned aerial vehicle is located.

Acquiring position information and posture information of the photographing device;

Determine the focus target according to the position information and posture information of the photographing device and the three-dimensional map;

The depth information of the focus target is determined as the target distance between the focus target and the photographing device.

Determine one or more direction vectors according to the position information and posture information of the photographing device;

According to the direction vector and the three-dimensional map, a focus target of the photographing device is determined.

According to the position information and posture information of the photographing device, one or more direction vectors corresponding to the central position area within the field of view of the photographing device are determined.

Determine one or more target location points according to the direction vector and the three-dimensional map;

When the target area corresponding to the target position point meets a preset condition, the target position point is determined as the focus target.

Optionally, the foregoing preset condition includes: the area occupied by the target area is greater than a preset threshold and/or the location of the target area is located in a preset location area.

For each of the focus targets, respectively acquiring images collected by the shooting device according to the focus parameters corresponding to the focus targets;

Compare all images to determine the target image;

The focal length of the photographing device is adjusted according to the focus parameter corresponding to the target image.

Optionally, when the processor 801 executes the application program:

Storing the target distance and the state information of the photographing device corresponding to the target distance in association, and the state information of the photographing device includes posture information and position information;

And, when it is determined that the state of the photographing device is the state of the photographing device represented by the state information of the photographing device, the stored target distance corresponding to the state information of the photographing device is acquired, and the photographing is controlled according to the target distance. The device performs automatic focusing.

The device shown in FIG. 8 can execute the methods of the embodiments shown in FIG. 1 to FIG. 7. For parts that are not described in detail in this embodiment, please refer to the related description of the embodiment shown in FIG. 1 to FIG. 7, which will not be repeated here.

An embodiment of the present invention also provides an unmanned aerial vehicle. FIG. 9 is a schematic structural diagram of an unmanned aerial vehicle according to an embodiment of the present invention. Referring to FIG. 9, the UAV 1000 includes: a photographing device 1002 and a focusing control device 1001 of the photographing device described in any of the above embodiments. The focusing control device 1001 of the photographing device may be set inside the photographing device 1002 It may also be installed outside the imaging device 1002, or partly installed inside the imaging device 1002, and part independently installed outside the imaging device 1002, for example, one part is installed in the body of the UAV 1000, and another part is installed in the imaging device 1002.

Wherein, the focus control device 1001 of the shooting device is used to obtain a three-dimensional map of the surrounding environment where the shooting device 1002 is located; the focus target and focus parameters of the shooting device are determined at least partially according to the three-dimensional map, and the focus The parameters include: the target distance between the focus target and the shooting device; and adjusting the focal length of the shooting device according to the focus parameter.

Optionally, referring to FIG. 10, the above-mentioned unmanned aerial vehicle 1000 further includes: a depth sensor 1003. The above-mentioned depth sensor 1003 is a binocular vision sensor. By using the binocular vision sensor for distance measurement, the object's position can be quickly calculated. Three-dimensional coordinates, reconstruction of three-dimensional space. It has the advantages of high flexibility and high accuracy. And because existing unmanned aerial vehicles are generally installed with binocular vision sensors to realize automatic obstacle avoidance and other functions, the embodiment of the present invention generates a three-dimensional map by using binocular vision sensors without adding additional hardware structures such as other sensors. It can accurately determine the focus target and realize automatic focus based on the depth information of the focus target, which has simple and efficient positive effects.

Although the exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above-described exemplary embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. A person of ordinary skill in the art can make various changes and modifications therein without departing from the scope and spirit of the present invention. All these changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

A person of ordinary skill in the art may be aware that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered as going beyond the scope of the present invention.

In the several embodiments provided in this application, it should be understood that the disclosed device and method may be implemented in other ways. For example, the device embodiments described above are merely illustrative, for example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another device, or some features can be ignored or not implemented.

In the instructions provided here, a lot of specific details are explained. However, it can be understood that the embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and technologies are not shown in detail, so as not to obscure the understanding of this specification.

Similarly, it should be understood that in order to simplify the present invention and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure , Or in its description. However, the method of the present invention should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly stated in each claim. To be more precise, as reflected in the corresponding claims, the point of the invention is that the corresponding technical problems can be solved with features less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are thus explicitly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present invention.

Those skilled in the art can understand that in addition to mutual exclusion between the features, any combination of all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and any method or device disclosed in this manner can be used. Processes or units are combined. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

In addition, those skilled in the art can understand that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments means that they are within the scope of the present invention. Within and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

The various component embodiments of the present invention may be implemented by hardware, or by software modules running on one or more processors, or by their combination. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some modules according to the embodiments of the present invention. The present invention can also be implemented as a device program (for example, a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for realizing the present invention may be stored on a computer-readable medium, or may have the form of one or more signals. Such signals can be downloaded from Internet websites, or provided on carrier signals, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate the present invention rather than limit the present invention, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be constructed as a limitation to the claims. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several devices, several of these devices may be embodied in the same hardware item. The use of the words first, second, and third does not indicate any order. These words can be interpreted as names.

Claims

A focus control method of a photographing device, wherein the photographing device is installed on an unmanned aerial vehicle, and the method includes:

Acquiring a three-dimensional map of the surrounding environment where the unmanned aerial vehicle is located;

Determining a focus target and focus parameters of the shooting device at least partly according to the three-dimensional map, the focus parameters including: a target distance between the focus target and the shooting device;

The focal length of the photographing device is adjusted according to the focus parameter.
The method according to claim 1, wherein the obtaining a three-dimensional map of the surrounding environment of the unmanned aerial vehicle comprises:

Acquiring a depth map of the surrounding environment where the unmanned aerial vehicle is located, the depth map being detected by a depth sensor on the unmanned aerial vehicle;

Acquiring the position and attitude of the depth sensor;

According to the depth map and the position and attitude of the depth sensor, a three-dimensional map of the surrounding environment of the unmanned aerial vehicle is obtained.
The method according to claim 2, wherein the depth sensor comprises a binocular vision sensor, and the acquiring a three-dimensional map of the surrounding environment of the unmanned aerial vehicle comprises:

Acquire a three-dimensional map within a preset range in the surrounding environment where the unmanned aerial vehicle is located.
The method according to claim 1, wherein the determining a focus target and focus parameters of the shooting device at least partly according to the three-dimensional map comprises:

Acquiring position information and posture information of the photographing device;

Determine the focus target according to the position information and posture information of the photographing device and the three-dimensional map;

The depth information of the focus target is determined as the target distance between the focus target and the photographing device.
The method of claim 4, wherein the determining the focus target according to the position information and posture information of the photographing device and the three-dimensional map comprises:

Determine one or more direction vectors according to the position information and posture information of the photographing device;

According to the direction vector and the three-dimensional map, a focus target of the photographing device is determined.
The method according to claim 5, wherein the determining one or more direction vectors according to the position information and posture information of the photographing device comprises:

According to the position information and posture information of the photographing device, one or more direction vectors corresponding to the central position area within the field of view of the photographing device are determined.
The method of claim 5, wherein the determining the focus target of the photographing device according to the direction vector and the three-dimensional map comprises:

Determine one or more target location points according to the direction vector and the three-dimensional map;

When the target area corresponding to the target position point meets a preset condition, the target position point is determined as the focus target.
The method according to claim 7, wherein the preset condition comprises: the area occupied by the target area is greater than a preset threshold and/or the location of the target area is located in a preset location area.
The method according to claim 1, wherein when the number of the focus targets is multiple, the adjusting the focal length of the photographing device according to the focus parameter comprises:

For each of the focus targets, respectively acquiring images collected by the shooting device according to the focus parameters corresponding to the focus targets;

Compare all images to determine the target image;

The focal length of the photographing device is adjusted according to the focus parameter corresponding to the target image.
The method according to claim 1, wherein the focus parameter further comprises: state information of the shooting device corresponding to the target distance, and the state information of the shooting device includes posture information and position information;

After the focusing target and focusing parameters of the photographing device are determined at least partially according to the three-dimensional map, the method further includes:

Storing the target distance and the state information of the shooting device corresponding to the target distance in association;

When it is determined that the state of the shooting device is the state of the shooting device represented by the state information of the shooting device, the stored target distance corresponding to the state information of the shooting device is acquired, and the shooting device is controlled according to the target distance. auto focus.
A focus control device for a photographing device, which is characterized in that it comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and is characterized in that when the processor executes the program:

Acquiring a three-dimensional map of the surrounding environment where the unmanned aerial vehicle is located;

Determining a focus target and focus parameters of the shooting device at least partly according to the three-dimensional map, the focus parameters including: a target distance between the focus target and the shooting device;

The focal length of the photographing device is adjusted according to the focus parameter.
The device according to claim 11, wherein the processor implements:

Acquiring a depth map of the surrounding environment where the unmanned aerial vehicle is located, the depth map being detected by a depth sensor on the unmanned aerial vehicle;

Acquiring the position and attitude of the depth sensor;

According to the depth map and the position and attitude of the depth sensor, a three-dimensional map of the surrounding environment of the unmanned aerial vehicle is obtained.
The device according to claim 12, wherein the depth sensor comprises a binocular vision sensor; when the processor executes the program:

Acquire a three-dimensional map within a preset range in the surrounding environment where the unmanned aerial vehicle is located.
The device according to claim 11, wherein the processor implements:

Acquiring position information and posture information of the photographing device;

Determine the focus target according to the position information and posture information of the photographing device and the three-dimensional map;

The depth information of the focus target is determined as the target distance between the focus target and the photographing device.
The apparatus according to claim 14, wherein the processor implements:

Determine one or more direction vectors according to the position information and posture information of the photographing device;

According to the direction vector and the three-dimensional map, a focus target of the photographing device is determined.
The device according to claim 15, wherein when the processor executes the program:

According to the position information and posture information of the photographing device, one or more direction vectors corresponding to the central position area within the field of view of the photographing device are determined.
The device according to claim 15, wherein when the processor executes the program:

Determine one or more target location points according to the direction vector and the three-dimensional map;

When the target area corresponding to the target position point meets a preset condition, the target position point is determined as the focus target.
The device according to claim 17, wherein the preset condition comprises: the area occupied by the target area is greater than a preset threshold and/or the location of the target area is in a preset location area.
The device according to claim 11, wherein the processor implements:

For each of the focus targets, respectively acquiring images collected by the shooting device according to the focus parameters corresponding to the focus targets;

Compare all images to determine the target image;

The focal length of the photographing device is adjusted according to the focus parameter corresponding to the target image.
The device according to claim 1, wherein the processor implements when the application program is executed:

Storing the target distance and the state information of the photographing device corresponding to the target distance in association, and the state information of the photographing device includes posture information and position information;

And, when it is determined that the state of the photographing device is the state of the photographing device represented by the state information of the photographing device, the stored target distance corresponding to the state information of the photographing device is acquired, and the photographing is controlled according to the target distance. The device performs automatic focusing.
An unmanned aerial vehicle, characterized in that the unmanned aerial vehicle comprises: a photographing device and a focusing control device of the photographing device according to claim 11 to claim 20.