WO2023097494A1

WO2023097494A1 - Panoramic image photographing method and apparatus, unmanned aerial vehicle, system, and storage medium

Info

Publication number: WO2023097494A1
Application number: PCT/CN2021/134568
Authority: WO
Inventors: 贾向华; 刘昂; 王璐
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2021-11-30
Filing date: 2021-11-30
Publication date: 2023-06-08

Abstract

A panoramic image photographing method and apparatus, an unmanned aerial vehicle, a system, and a storage medium, the panoramic image photographing method being applied to an unmanned aerial vehicle provided with a photographing apparatus, and the method comprising: acquiring position information and photographing angle information of at least two selected photographing waypoints, the position information and the photographing angle information of the at least two selected photographing waypoints being determined on the basis of a photographing waypoint setting operation of a user; planning a photographing route according to the position information and the photographing angle information of the at least two selected photographing waypoints so as to determine position information and photographing angle information of one or more planned photographing waypoints in the photographing route; and during the flight of the unmanned aerial vehicle according to the photographing route, according to the position information and the photographing angle information of the at least two selected photographing waypoints and the position information and the photographing angle information of the one or more planned photographing waypoints controlling the photographing apparatus to photograph a plurality of images, the plurality of images being used for generating a panoramic image. A panoramic image in a wider field of view is acquired.

Description

Panoramic image shooting method, device, unmanned aerial vehicle, system and storage medium

technical field

The present application relates to the technical field of drone aerial photography, in particular, to a panoramic image shooting method, device, drone, system and storage medium.

Background technique

In related technologies, the process of obtaining a panoramic image through a UAV is usually: the UAV hovers in the air, automatically controls the heading or the angle of the gimbal of the UAV to change the orientation of the shooting device, at an angle of 180° or 360° Take multiple images within the range, and then combine the multiple images into a panoramic image. This method of obtaining a panoramic image cannot flexibly set the shooting range, and since the UAV hovers in the air for shooting, the content contained in the obtained panoramic image or the field of view involved is limited.

Contents of the invention

In view of this, one of the objectives of the present application is to provide a panoramic image shooting method, device, drone, system and storage medium.

In the first aspect, the embodiment of the present application provides a panoramic image shooting method, the method is applied to a drone, and the drone is equipped with a shooting device, and the method includes:

Acquiring position information and shooting angle information of at least two selected shooting waypoints, the position information and shooting angle information of the at least two selected shooting waypoints are determined based on the user's shooting waypoint setting operation;

Plan the shooting route according to the position information and shooting angle information of the at least two selected shooting waypoints, so as to determine the position information and shooting angle information of one or more planned shooting waypoints in the shooting route;

During the flight of the UAV according to the shooting route, according to the position information and shooting angle information of the at least two selected shooting waypoints, and the positions of the one or more planned shooting waypoints Information and shooting angle information, control the shooting device to take multiple images, and the multiple images are used to generate a panoramic image.

In a second aspect, an embodiment of the present application provides a panoramic image shooting device, the device comprising:

memory for storing executable instructions;

one or more processors;

Wherein, when the one or more processors execute the executable instructions, they are individually or collectively configured to execute the method described in the first aspect.

In a third aspect, the embodiment of the present application provides a drone, including:

body;

A power system, installed on the body, is used to provide power for the unmanned aerial vehicle;

a photographing device installed on the body for photographing images; and,

The panoramic image shooting device according to the second aspect installed on the body.

In the fourth aspect, the embodiment of the present application provides a panoramic image shooting system, including the drone and the remote control device described in the third aspect;

The unmanned aerial vehicle is also used to send a plurality of captured images to the remote control device;

The remote control device is used to stitch the multiple images into a panoramic image.

In the fifth aspect, the embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores executable instructions, and when the executable instructions are executed by a processor, the method as described in the first aspect is implemented .

A panoramic image shooting method, device, unmanned aerial vehicle, system, and storage medium provided in the embodiments of the present application can plan the shooting route according to the position information and shooting angle information of at least two shooting waypoints selected by the user, so as to Determine the position information and shooting angle information of one or more planned shooting waypoints in the shooting route, and then complete the shooting at multiple shooting waypoints to obtain multiple images during the flight of the drone according to the shooting route , and then based on multiple images, a panoramic image covering the scene desired by the user can be generated. The embodiment of the present application can support the flexible setting of the panoramic shooting range, and a panoramic image with a wider field of view and more content can be obtained by using the movement characteristics of the drone in space.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

Fig. 1 is a product schematic diagram of an unmanned aerial system provided by an embodiment of the present application;

FIG. 2 is a schematic structural diagram of a drone provided by an embodiment of the present application;

FIG. 3 is a schematic flowchart of a panoramic image shooting method provided by an embodiment of the present application;

Fig. 4 is a schematic diagram of an imaging range of a panoramic image provided by an embodiment of the present application;

Figure 5 and Figure 6 are two different schematic diagrams of the intersection of the plane where the shooting route is located and the horizontal plane provided by an embodiment of the present application;

Fig. 7 is a schematic diagram of the first type of route provided by an embodiment of the present application;

Fig. 8A, Fig. 8B, Fig. 9A and Fig. 9B are different schematic diagrams of the second type of routes provided by an embodiment of the present application;

Fig. 10 is a schematic structural diagram of a panoramic image capturing device provided by an embodiment of the present application.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some of the embodiments of the application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

Aiming at the problem that the method of generating panoramic images in the related art cannot flexibly set the shooting range, the content of the panoramic image or the field of view is limited, the embodiment of the present application provides a panoramic image shooting method, device, unmanned aerial vehicle, system and storage medium.

In some embodiments, the panoramic image shooting method provided in the embodiment of the present application can be applied to a drone. It will be apparent to those skilled in the art that any type of drone can be used without limitation, and the embodiments of the present application can be applied to various types of drones. For example, an unmanned aerial vehicle (UAV) can be a small or large drone. In some embodiments, the unmanned aerial vehicle may be a rotorcraft, for example, a multi-rotor unmanned aerial vehicle propelled by a plurality of propulsion devices through the air, the embodiments of the present application are not limited thereto, the unmanned aerial vehicle Other types of drones are also possible.

Please refer to FIG. 1 and FIG. 2 , FIG. 1 shows a product schematic diagram of an unmanned aerial system, and FIG. 2 shows an exemplary architecture diagram of an unmanned aerial vehicle. In this embodiment, a rotor drone is taken as an example for illustration.

The unmanned aerial system may include a drone 110 and a remote control device 140 . Wherein, the unmanned aerial vehicle 110 may include a power system 150, a flight control system 160, a frame and a pan-tilt 120 carried on the frame. The remote control device 140 can include a display device 130, and the display device 130 can be integrated into the remote control device 140, such as a remote controller with a screen; the display device 130 can also be set independently, for example, the remote control device includes a remote control and a mobile phone, and the remote control can communicate wirelessly with the mobile phone Or wired communication. The drone 110 can communicate wirelessly with the remote control device 140 . The UAV 110 can be an agricultural UAV or an industrial application UAV, and there is a need for cyclic operations.

The frame may include the fuselage and undercarriage (also known as landing gear). The fuselage may include a center frame and one or more arms connected to the center frame, and one or more arms extend radially from the center frame. The tripod is connected with the fuselage and is used for supporting the UAV 110 when it lands.

The power system 150 may include one or more electronic governors (abbreviated as ESCs) 151, one or more propellers 153 and one or more motors 152 corresponding to the one or more propellers 153, wherein the motors 152 are connected to Between the electronic governor 151 and the propeller 153, the motor 152 and the propeller 153 are arranged on the machine arm of the UAV 110; the electronic governor 151 is used to receive the driving signal generated by the flight control system 160, and provide driving according to the driving signal Current is supplied to the motor 152 to control the speed of the motor 152 . The motor 152 is used to drive the propeller to rotate, so as to provide power for the flight of the UAV 110 , and the power enables the UAV 110 to realize movement of one or more degrees of freedom. In some embodiments, drone 110 may rotate about one or more axes of rotation. For example, the rotation axis may include a roll axis (Roll), a yaw axis (Yaw) and a pitch axis (pitch). It should be understood that the motor 152 may be a DC motor or an AC motor. In addition, the motor 152 can be a brushless motor or a brushed motor.

Flight control system 160 may include flight controller 161 and sensing system 162 . The sensing system 162 is used to measure the attitude information of the UAV, that is, the position information and state information of the UAV 110 in space, such as three-dimensional position, three-dimensional angle, three-dimensional velocity, three-dimensional acceleration and three-dimensional angular velocity. The sensing system 162 may include, for example, at least one of sensors such as a gyroscope, an ultrasonic sensor, an electronic compass, an inertial measurement unit (Inertial Measurement Unit, IMU), a visual sensor, a global navigation satellite system, and a barometer. For example, the global navigation satellite system may be the Global Positioning System (GPS). The flight controller 161 is used to control the flight of the UAV 110 , for example, the flight of the UAV 110 can be controlled according to the attitude information measured by the sensing system 162 . It should be understood that the flight controller 161 can control the UAV 110 according to pre-programmed instructions, or can control the UAV 110 by responding to one or more remote control signals from the remote control device 140 .

The gimbal 120 may include a motor 122 . The pan-tilt is used to carry the photographing device 123 . The flight controller 161 can control the movement of the gimbal 120 through the motor 122 . Optionally, as another embodiment, the pan-tilt 120 may further include a controller for controlling the movement of the pan-tilt 120 by controlling the motor 122 . It should be understood that the gimbal 120 may be independent of the UAV 110 or be a part of the UAV 110 . It should be understood that the motor 122 may be a DC motor or an AC motor. In addition, the motor 122 may be a brushless motor or a brushed motor. It should also be understood that the gimbal can be located on top of the drone or on the bottom of the drone.

The photographing device 123 can be, for example, a camera or a video camera or other equipment for capturing images. The photographing device 123 can communicate with the flight controller and take pictures under the control of the flight controller. The photographing device 123 in this embodiment includes at least a photosensitive element, such as a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS) sensor or a charge-coupled device (Charge-coupled Device, CCD) sensor. It can be understood that the camera device 123 can also be directly fixed on the drone 110, so that the pan-tilt 120 can be omitted.

The remote control device 140 is located at the ground end of the unmanned aerial system, and can communicate with the UAV 110 in a wireless manner for remote control of the UAV 110 .

It should be understood that the above naming of the various components of the unmanned aerial vehicle system is only for the purpose of identification, and should not be construed as limiting the embodiments of the present application.

In an exemplary application scenario, in the embodiment shown in FIG. 1 or FIG. 2 , the UAV can send the collected images back to the remote control device 140 for display, and the user can operate the remote control device 140 to Control the proper position of the UAV movement, control the gimbal to drive the shooting device to rotate to a suitable angle, so as to determine the position information and shooting angle information of the selected shooting waypoint. The location information and shooting angle information of at least two selected shooting waypoints can indicate the flight range and shooting range of the drone, and the drone can Information for shooting route planning to determine the position information and shooting angle information of one or more planned shooting waypoints in the shooting route, and then during the flight of the drone according to the shooting route, at multiple shooting waypoints respectively Complete the photo taking to obtain multiple images; the drone can stitch multiple images into a panoramic image, or the drone can send multiple images back to the remote control device 140, and the remote control device 140 stitches multiple images into a panoramic image image.

Next, the panoramic image shooting method provided by the embodiment of the present application will be described. Please refer to FIG. 3 . FIG. 3 shows a schematic flowchart of a panoramic image shooting method, and the method can be executed by a panoramic image shooting device. Exemplarily, the panoramic image shooting device may be a flight controller in the unmanned aerial vehicle shown in FIG. 2 , and the unmanned aerial vehicle is also equipped with a shooting device. The methods include:

In step S101, the location information and shooting angle information of at least two selected shooting waypoints are obtained, and the position information and shooting angle information of the at least two selected shooting waypoints are based on the user's shooting waypoint setting operation definite.

In step S102, plan the shooting route according to the position information and shooting angle information of the at least two selected shooting waypoints, so as to determine the position information and shooting angle of one or more planned shooting waypoints in the shooting route information.

In step S103, during the flight of the drone according to the shooting route, according to the position information and shooting angle information of the at least two selected shooting waypoints, and the one or more planned The location information and shooting angle information of the waypoint are photographed, and the photographing device is controlled to photograph a plurality of images, and the plurality of images are used to generate a panoramic image.

In some embodiments, during the flight of the drone, the drone can send the images captured by the shooting device to the remote control device for display in real time; the at least two selected shooting waypoints The location information and shooting angle information are determined based on the user's shooting waypoint setting operation on the remote control device during the flight of the drone.

The user can control the UAV to fly through the remote control device. While the UAV is flying, it collects the picture through the shooting device and sends it back to the remote control device for display, that is, the user can see the picture returned by the UAV in real time, and then shoot During the waypoint setting operation, the scene covered by the panoramic image and the imaging effect of the panoramic image can be expected based on the real-time returned picture. Therefore, during the flight of the drone, the user can obtain the scene covered by the panoramic image and To expect the imaging effect, perform the shooting waypoint setting operation on the remote control device to select the shooting waypoint.

Among them, the user can perform shooting waypoint setting operation when the UAV is in a hovering state, or perform shooting waypoint setting operation when the UAV is in a moving state.

Wherein, the position information may include longitude, latitude and altitude; when the shooting device is directly fixed on the drone, the shooting angle information may include the nose direction of the drone and/or the field of view of the shooting device Direction; when the shooting device is set on the drone through the cloud platform, the shooting angle information may include at least one of the following: the direction of the nose of the drone, the orientation of the cloud platform or the orientation of the shooting device Field of view direction. Wherein, the shooting angle information of at least two selected shooting waypoints may be the same or different.

In some embodiments, the user can adjust the position of the drone through the remote control device, as well as the direction of the nose, the direction of the field of view of the shooting device, or the direction of the gimbal, etc., and the drone will use the real-time position information and real-time shooting angle information to preset The frequency is sent to the remote control device. If the remote control device detects the user's shooting waypoint setting operation (for example, the user triggers the waypoint setting control in the remote control device), the latest received real-time position information and real-time shooting angle information of the drone can be sent to the remote control device. Record the location information and shooting angle information of the shooting waypoint selected by the user this time. After finishing the shooting waypoint setting process, the remote control device can transmit the position information and shooting angle information of at least two selected shooting waypoints to the drone.

In some embodiments, the remote controller is provided with a control for triggering waypoint setting. When the user operates the control, the remote controller will send a shooting waypoint setting instruction to the UAV, and the UAV can send the unmanned The real-time location information and real-time shooting angle information of the man-machine are recorded as the location information and shooting angle information of the shooting waypoint selected by the user this time.

Of course, the user can also set the position first and then set the shooting angle, or set the shooting angle first and then set the position. This embodiment does not impose any limitation on the specific interaction manner.

In some embodiments, after the UAV acquires the position information and shooting angle information of at least two selected waypoints for shooting, the UAV can and shooting angle information to determine whether a new shooting waypoint needs to be planned, and if necessary, add one or more planned shooting waypoints.

In an exemplary embodiment, the UAV can determine the imaging of the panoramic image according to the field of view of the shooting device, the position information and shooting angle information of the at least two selected shooting waypoints Range; the imaging range of the panoramic image can reflect the flight range of the UAV to a certain extent, so the shooting route planning can be carried out according to the imaging range of the panoramic image to determine one or more planned routes in the shooting route Capture the position information and shooting angle information of the waypoint. Wherein, the size of the field of view of the photographing device may be determined according to parameters such as focal length, aperture or shutter of the photographing device.

Exemplarily, the shooting area corresponding to the selected shooting waypoint can be determined based on the field of view of the shooting device, the position information of the selected shooting waypoint, and the shooting angle information, and the shooting area refers to the physical In the environmental area, different selected shooting waypoints correspond to different shooting areas. Then, the imaging range of the panoramic image may be a minimum circumscribed figure including the shooting areas respectively corresponding to at least two selected shooting waypoints. The minimum circumscribed figure may be, for example, a minimum circumscribed rectangle or a minimum circumscribed circle.

In one example, please refer to FIG. 4. Taking two selected shooting waypoints as an example, the shooting areas corresponding to the two selected shooting waypoints include shooting area 11 and shooting area 12, and shooting area 11 and shooting area 12 may partially overlap or not overlap at all, and the imaging range 13 of the panoramic image may be the smallest circumscribed rectangle including the shooting area 11 and the shooting area 12 . The shooting areas corresponding to the two selected shooting waypoints can be located at the diagonal positions of the minimum circumscribed rectangle, so that the user can intuitively expect the imaging effect of the panoramic image through the shooting areas corresponding to the two selected shooting waypoints, and then This makes it easier for the finally stitched panoramic image to meet user expectations.

In another example, the user can also select 3 or more non-collinear shooting waypoints according to actual needs, and the imaging range of the panoramic image can include 3 or more non-collinear shooting waypoints. The minimum circumscribed rectangle of the shooting area corresponding to the points respectively.

In an exemplary embodiment, in order to improve the stitching efficiency and stitching effect of panoramic images, in the shooting route, the images taken by two adjacent shooting waypoints need to have a preset overlap ratio. It can be understood that the The specific value of the overlap rate can be set according to the actual application scenario, and this embodiment does not impose any limitation on it, for example, the overlap rate is 20%, 25%, or 30%. Then the UAV can carry out shooting route planning according to the position information and shooting angle information of the at least two selected shooting waypoints, as well as the preset overlap rate and the field of view of the shooting device, so as to determine The position information and shooting angle information of one or more planned shooting waypoints, so that the field of view of the shooting device corresponding to two adjacent shooting waypoints overlaps, thereby ensuring that the images taken by two adjacent shooting waypoints The image has a preset overlap rate, which facilitates the stitching process of multiple images, improves the stitching efficiency and stitching effect when multiple images are stitched into a panoramic image, and makes the stitched panoramic image naturally unobtrusive.

Exemplarily, the UAV can determine the imaging range of the panoramic image according to the field of view of the shooting device, the position information and shooting angle information of the at least two selected shooting waypoints, and according to the panoramic image determine whether a new shooting waypoint needs to be planned, for example, the shooting of the selected shooting waypoint can be determined according to the field of view of the shooting device, the position information and the shooting angle information of the selected shooting waypoint area, if the shooting areas of the at least two selected shooting waypoints do not overlap or the overlapping ratio of the shooting regions of the at least two selected shooting waypoints does not meet the preset overlapping ratio, it is determined that planning is required A new shooting waypoint, the UAV can according to the difference between the position information of the at least two selected shooting waypoints, the difference between the shooting angle information of the at least two selected shooting waypoints The difference, the preset overlap rate and the field of view of the shooting device are used to plan a shooting route to determine the position information and shooting angle information of one or more planned shooting waypoints in the shooting route.

In some embodiments, the location information of the at least two selected shooting waypoints includes altitude, longitude and latitude, and the at least two selected shooting waypoints include a first selected waypoint and a second selected waypoint A waypoint, in the case that the altitude of the first selected waypoint is different from that of the second selected waypoint, the plane where the shooting route is located intersects with the horizontal plane. Exemplarily, if the longitude of the first selected waypoint is approximately the same as that of the second selected waypoint and/or the latitude of the first selected waypoint is approximately the same as that of the second selected waypoint , please refer to FIG. 5 , the plane 10 where the shooting route is located is approximately perpendicular to the horizontal plane 20 . In this embodiment, the movement characteristics of the UAV can be used to obtain a panoramic image covering a plane approximately perpendicular to the horizontal plane.

Among them, "substantially the same" includes two situations, one is that the longitude and/or latitude of the first selected waypoint and the second selected waypoint are exactly the same, and the other is that the first selected waypoint There is a preset difference in longitude and/or latitude between the fixed waypoint and the second selected waypoint, and the preset difference is relatively small, such as less than 10 degrees. Correspondingly, "approximately vertical" also includes two situations. In the case where the longitude and/or latitude of the first selected waypoint and the second selected waypoint are exactly the same, the location where the shooting route is located The plane is completely perpendicular to the horizontal plane; when there is a preset difference in longitude and/or latitude between the first selected waypoint and the second selected waypoint, the plane where the shooting route is located and the horizontal plane There is a deviation angle between the angle and 90°, and the deviation angle is small, for example, the deviation angle is less than 5°.

Exemplarily, if the longitude of the first selected waypoint is different from that of the second selected waypoint and the latitude of the first selected waypoint is different from that of the second selected waypoint, please refer to FIG. 6. The plane 10 where the shooting route is located intersects with the horizontal plane 20 and is not perpendicular, that is, the angle between the plane 10 where the shooting route is located and the horizontal plane 20 and 90° have a difference greater than the above deviation angle. In this embodiment, the movement characteristics of the UAV can be used to obtain a panoramic image covering a plane that intersects with the horizontal plane and is not vertical.

In some embodiments, the location information of the at least two selected shooting waypoints includes altitude, longitude and latitude. If the altitude of the first selected waypoint is different from that of the second selected waypoint, the longitude of the first selected waypoint and the second selected waypoint are approximately the same and the first selected waypoint The latitude of the waypoint is approximately the same as that of the second selected waypoint. Please refer to FIG. 7, the plane 10 where the shooting route is located is approximately perpendicular to the horizontal plane 20, and the shooting route is a first-type route, and the first Class routes are straight lines. When the drone is flying according to the shooting route, the user can see that the drone is flying straight up or down straight down.

In Fig. 7, the first type of route includes photographing waypoint A, photographing waypoint B and photographing waypoint C, wherein photographing waypoint A and photographing waypoint B are the selected photographing waypoints, and photographing waypoint C is the planned , and Fig. 7 shows the field of view 21 of the photographing device at waypoint A, the field of view 22 of waypoint B and the field of view 23 of waypoint C, two adjacent waypoints The fields of view of the respective corresponding shooting devices overlap, so that the images captured by two adjacent shooting waypoints have a preset overlap rate.

In the first type of route, the longitude and latitude of the planned shooting waypoint are approximately the same as the longitude and latitude of the selected shooting waypoint. The height of the planned shooting waypoint is between the height of the first selected waypoint and the height of the second selected waypoint, for example, the UAV can The height difference between the height of the selected waypoint and the height of the second selected waypoint, the field of view of the shooting device, and the preset overlapping ratio that the images taken by two adjacent shooting waypoints need to have, determine the one that needs to be added or the altitude of multiple planned shooting waypoints.

Wherein, the greater the altitude difference between the altitude of the first selected waypoint and the altitude of the second selected waypoint, the more the number of the planned shooting waypoints needs to be increased, that is, the The height difference is positively correlated with the number of planned shooting waypoints; the larger the field of view of the shooting device, the larger the distance between adjacent waypoints can be set, and the increased number of planned shooting waypoints is required. The fewer the number of points, that is, the field of view of the shooting device has a negative correlation with the number of the planned shooting waypoints; the larger the preset overlap rate, the further the distance between adjacent waypoints needs to be set. is smaller, the more the number of the planned shooting waypoints needs to be increased, that is, the preset overlap ratio is positively correlated with the number of the planned shooting waypoints.

The shooting angle information of the planned shooting waypoint may be determined based on the shooting angle information of the first selected waypoint and/or the shooting angle information of the second selected waypoint; the first selected waypoint The shooting angle information of the second selected waypoint may be the same as or different from the shooting angle information of the second selected waypoint.

In the case that the shooting angle information of the first selected waypoint is the same as the shooting angle information of the second selected waypoint, the shooting angle information of the planned shooting waypoint is also the same as that of the first selected waypoint above. The camera angle information is the same for the waypoint and the second selected waypoint.

In the case where the shooting angle information of the first selected waypoint is different from the shooting angle information of the second selected waypoint, in an example, for example, the shooting angle information of the planned shooting waypoint may be the same as The shooting angle information of the first selected waypoint is the same; in another example, the shooting angle information of the planned shooting waypoint may also be the same as the shooting angle information of the second selected waypoint. In yet another example, the shooting angle information of the planned shooting waypoint may be determined according to the difference between the shooting angle information of the first selected waypoint and the shooting angle information of the second selected waypoint, For example, in order to realize that the shooting angle of the shooting waypoint in the first type of route presents a gradual and smooth transition from the shooting angle of one of the first selected waypoint and the second selected waypoint to the shooting of the other The effect of the angle, the planned shooting waypoint can be superimposed on the basis of the shooting angle information of the first selected waypoint or the shooting angle information of the second selected waypoint (the first selected waypoint Partial differences between the shooting angle information of the selected waypoint and the shooting angle information of the second selected waypoint) are obtained to realize the smooth transition of the shooting angles of the shooting waypoints in the first type of route, ensuring that two adjacent The transition of images captured at each shooting waypoint is naturally unobtrusive.

If the shooting angle information of the first selected waypoint is different from the shooting angle information of the second selected waypoint, according to the shooting angle information of the first selected waypoint and the second selected waypoint In the process of determining the shooting angle information of the planned shooting waypoint by the difference between the shooting angle information of the points, it can be satisfied that the sum of the shooting angle differences of all two adjacent shooting waypoints in the first type of route is equal to A camera angle difference between the first selected waypoint and the second selected waypoint.

Exemplarily, such as in FIG. 7 , the first type of route includes photographing waypoint A, photographing waypoint B and photographing waypoint C, wherein photographing waypoint A and photographing waypoint B are selected photographing waypoints, and photographing Waypoint C is the planned shooting waypoint, and the shooting angle of waypoint C can be superimposed on the shooting angle of waypoint A or the shooting angle of waypoint B (shooting waypoint A and shooting waypoint Part of the shooting angle difference in B) is obtained, and the sum of the shooting angle difference between shooting waypoint A and shooting waypoint C, and the shooting angle difference between shooting waypoint C and shooting waypoint B can be set to be equal to shooting waypoint A and shooting The shooting angle difference between waypoints B, so that all shooting waypoints in the first type of route can present one of the first selected waypoint and the second selected waypoint (for example, shooting waypoint A) The shooting angle of one is gradually changed to the shooting angle of the other (such as shooting waypoint B), so as to realize the smooth transition of shooting angles and ensure that the transition of images taken by two adjacent shooting waypoints is natural and not abrupt.

In some other embodiments, if the altitude of the first selected waypoint is different from that of the second selected waypoint, and the longitude of the first selected waypoint is different from that of the second selected waypoint And/or the latitude of the first selected waypoint is different from that of the second selected waypoint, and the plane where the shooting route is located intersects with the horizontal plane, that is, including the above two cases, the plane where the shooting route is located and The horizontal plane is approximately vertical, or the plane where the shooting route is located intersects the horizontal plane and is not vertical; wherein, the shooting route is a second-type route, and the second-type route includes a first sub-route and a second sub-route, and the The direction of the first sub-route is perpendicular to the direction of the second sub-route.

For an example, please refer to FIG. 8A or FIG. 8B , the second type of sub-route may be a bow-shaped route, or, please refer to FIG. 9A or FIG. 9B , the second type of sub-route may be an unclosed rectangular route. In Fig. 8A, Fig. 8B, Fig. 9A and Fig. 9B, "●" indicates the shooting waypoint, wherein, shooting waypoint A and shooting waypoint B are selected shooting waypoints, and others are planned shooting waypoints, assuming The shooting waypoint A is the starting point of the shooting route, and the dotted line connecting each shooting waypoint indicates the shooting route. The box at the shooting waypoint indicates the field of view of the shooting device corresponding to the shooting waypoint, and the fields of view of the shooting devices corresponding to two adjacent shooting waypoints overlap, so that the images taken by two adjacent shooting waypoints Has a preset overlap rate.

In the second type of route, the photographing direction of the photographing device at the photographing waypoint is approximately perpendicular to the plane 10 where the photographing route is located.

In the second type of route, the second type of route includes at least one first sub-route and at least one second sub-route. Assuming that the first sub-course is roughly parallel to the horizontal plane, the second sub-course intersects the horizontal plane; in one example, in Figure 8A, the first sub-course includes the AC course, the EF course and the DB course, and the second sub-course includes the CE course and FD routes; in another example, in Fig. 8B, the first sub-route includes CE route and FD route, and the second sub-route includes AC route, EF route and DB route; in yet another example, in Fig. 9A , the first sub-route includes the AC route and the BD route, and the second sub-route includes the CB route and the DF route. Wherein, "approximately parallel" includes two situations, one is that the first sub-route is completely parallel to the horizontal plane, and the other is that the angle between the first sub-waypoint and the horizontal plane is smaller than a preset deviation, and the preset deviation is relatively small.

Assuming that the first sub-route is roughly parallel to the horizontal plane, the altitudes of all shot waypoints belonging to the same first sub-route are approximately the same; in one example, in FIG. 8A , the six shots of the first sub-route AC The heights of the waypoints are approximately the same, and the altitudes of the six photographed routes belonging to another first sub-route BD are also approximately the same; in another example, in FIG. 8B, the heights of the two photographed waypoints belonging to the first sub-route CE The altitudes are approximately the same, and the altitudes of the two shooting waypoints belonging to the first sub-route FD are approximately the same.

Assuming that the first sub-route is approximately parallel to the horizontal plane, one of the longitudes and latitudes of all shooting waypoints belonging to the same first sub-route is approximately the same.

In the case where the first sub-route is approximately parallel to the horizontal plane, correspondingly, the second sub-route intersects the horizontal plane, including two cases, as shown in Figure 8A or Figure 9A, the first case is that the second sub-route is approximately parallel to the horizontal plane Vertical; as shown in FIG. 8B or FIG. 9B , the second case is that the second sub-route intersects the horizontal plane and is not vertical.

In some embodiments, the shooting angle information of the first selected waypoint and the shooting angle information of the second selected waypoint may be the same or different. In the case that the shooting angle information of the first selected waypoint is the same as the shooting angle information of the second selected waypoint, the shooting angle information of the planned shooting waypoint may also be the same as that of the first selected waypoint above. The shooting angle information of the fixed waypoint and the second selected waypoint are the same. In the second type of route, the second type of route includes a plurality of sub-routes, and the plurality of sub-routes includes a first sub-route and a second sub-route. In the case where the shooting angle information of the first selected waypoint is different from the shooting angle information of the second selected waypoint, the planned shooting waypoints include the first planned waypoint and the second planned waypoint , the first planned waypoint and the second planned waypoint are the starting point or end point of the sub-route, and the shooting angle information of the first planned waypoint is the same as that in the first selected waypoint and the second selected waypoint The shooting angle information of one of them is the same, and the shooting angle information of the second planned waypoint is the same as the shooting angle information of the other one of the first selected waypoint and the second selected waypoint.

As an example, in FIG. 8A , it is assumed that the selected waypoint A is the first selected waypoint and the starting point of the route, the selected waypoint B is the second selected waypoint, and the first planned waypoint Including photographing waypoint C and photographing waypoint E, the second planning waypoint includes photographing waypoint D and photographing waypoint F, wherein the photographing angle information of photographing waypoint C and photographing waypoint E is the same as the photographing angle of photographing waypoint B The information is the same, the shooting waypoint C is the end point of the sub-route AC and the starting point of the sub-route CD, the shooting waypoint E is the end point of the sub-route CE and the starting point of the sub-route EF; the shooting angle information of the shooting waypoint D and the shooting waypoint F is the same as that of the shooting The photographing angle information of waypoint A is the same, photographing waypoint D is the end point of sub-route FD and the starting point of sub-route DB, and photographing waypoint F is the end point of sub-waypoint EF and the starting point of sub-route FD.

As an example, in FIG. 9A , it is assumed that the selected waypoint A is the first selected waypoint and the starting point of the route, the selected waypoint B is the second selected waypoint, and the first planned waypoint Including shooting waypoint C, the second planned waypoint includes shooting waypoint D, then the shooting angle information of shooting waypoint C can be the same as the shooting angle information of shooting waypoint A, and shooting waypoint C is the end point and sub-waypoint of sub-route AC The starting point of point CB, the shooting angle information of shooting waypoint D can be the same as the shooting angle information of shooting waypoint B, and shooting waypoint D is the end point of sub-route BD and the starting point of sub-waypoint DF.

In the second type of route, the planned shooting waypoints further include a third planned waypoint, and the third planned waypoint may be a shooting waypoint in the sub-route except the starting point and the ending point. As an example, in FIG. 8A , the third planned waypoint is a photographed waypoint other than photographed waypoint A, photographed waypoint B, photographed waypoint C, photographed waypoint E, photographed waypoint D and photographed waypoint F . As an example, in FIG. 9A , the third planned waypoint is a photographed waypoint other than photographed waypoint A, photographed waypoint B, photographed waypoint C, and photographed waypoint D.

In the case that the shooting angle information of the first selected waypoint is different from the shooting angle information of the second selected waypoint, the shooting angle information of the third planned waypoint may be based on the first selected The shooting angle information of the waypoint and the shooting angle information of the second selected waypoint are calculated; for example, the shooting angle information of the third planned waypoint can be the shooting angle information of the first selected waypoint or the shooting angle information of the second selected waypoint is superimposed on the basis of the partial difference between the two (the shooting angle information of the first selected waypoint and the shooting angle information of the second selected waypoint) to obtain , so as to realize the smooth transition of the shooting angles of the shooting waypoints in the second type of route, and ensure that the transition of images taken by two adjacent shooting waypoints is natural and not abrupt.

Exemplarily, the second type of route includes a plurality of sub-routes, and the plurality of sub-routes includes a first sub-route and a second sub-route. The shooting angle information of the shooting waypoints is different. For example, the shooting angles of the starting point and the end point of the sub-route AC in FIG. 8A and FIG. The difference between the shooting angle information of the shooting waypoint at the starting point and the shooting angle information of the shooting waypoint as the end point is determined. For example, the sum of the shooting angle differences of all two adjacent shooting waypoints in the sub-route can be set equal to the starting point and The difference in the shooting angle of the end point realizes the smooth transition of the shooting angle of the shooting waypoint in the sub-route from the shooting angle of the shooting waypoint as the starting point to the shooting angle of the shooting waypoint as the ending point, ensuring that two adjacent shooting waypoints are taken The image transitions are natural and unobtrusive.

Various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features, so any combination of various technical features in the above embodiments also falls within the scope of the disclosure of this specification.

Correspondingly, please refer to FIG. 10 , the embodiment of the present application also provides a panoramic image shooting device 30, including:

memory 31 for storing executable instructions;

one or more processors 32;

Wherein, when the one or more processors 32 execute the executable instructions, they are individually or collectively configured to perform any one of the methods described above.

The processor 32 executes the executable instructions included in the memory 31, the processor 32 can be a central processing unit (Central Processing Unit, CPU), and can also be other general-purpose processors, digital signal processors (Digital Signal Processor) Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

The memory 31 stores the executable instructions of the method for returning to the voyage of the unmanned aerial vehicle, and the memory 31 can include at least one type of storage medium, and the storage medium includes a flash memory, a hard disk, a multimedia card, a card type memory (for example, SD or DX memory etc.), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), Magnetic Memory , Disk, CD, etc. Also, the device may cooperate with a web storage which performs a storage function of the memory through a network connection. The storage 31 may be an internal storage unit, such as a hard disk or a memory. The memory 31 can also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card (Flash Card) and the like. Further, the memory 31 may also include both an internal storage unit and an external storage device. The memory 31 can also be used to temporarily store data that has been output or will be output.

In some embodiments, when the processor 32 executes the executable instructions, it is individually or jointly configured to:

In some embodiments, the location information includes an altitude, and the at least two selected waypoints for photographing include a first selected waypoint and a second selected waypoint, the first selected waypoint is in relation to the The altitude of the second selected waypoint is different, and the plane where the shooting route is located intersects the horizontal plane.

In some embodiments, the location information also includes longitude and latitude;

If the longitude of the first selected waypoint is approximately the same as that of the second selected waypoint and/or the latitude of the first selected waypoint is approximately the same as that of the second selected waypoint, the photographing The plane on which the route is located is roughly perpendicular to the horizontal plane;

If the longitude of the first selected waypoint is different from that of the second selected waypoint and the latitude of the first selected waypoint is different from that of the second selected waypoint, the plane where the shooting route is located Intersects the horizontal plane and is not perpendicular.

If the longitude of the first selected waypoint is approximately the same as that of the second selected waypoint and the latitude of the first selected waypoint is approximately the same as that of the second selected waypoint, the shooting route is The first type of route, the first type of route is a straight line;

If the longitude of the first selected waypoint is different from that of the second selected waypoint and/or the latitude of the first selected waypoint is different from that of the second selected waypoint, the shooting route is A second type of route, the second type of route includes a first sub-route and a second sub-route, the direction of the first sub-route is perpendicular to the direction of the second sub-route.

In some embodiments, the shooting route is a second-type route, the first sub-route is approximately parallel to the horizontal plane, and the second sub-route intersects the horizontal plane;

Wherein, the heights of all shooting waypoints belonging to the same first sub-route are approximately the same.

In some embodiments, the planned shooting waypoints include a first planned waypoint and a second planned waypoint, and the shooting angle information of the first planned waypoint is related to the first selected waypoint and the second selected waypoint. The shooting angle information of one of the points is the same, and the shooting angle information of the second planned waypoint is the same as that of the other one of the first selected waypoint and the second selected waypoint.

In some embodiments, the shooting route is a second-type route, and the second-type route includes a plurality of sub-routes, and the plurality of sub-routes includes the first sub-route and the second sub-route;

Wherein, the first selected waypoint and the second planned waypoint are the starting point or the ending point of the sub-route.

In some embodiments, the planned shooting waypoint includes a third shooting waypoint, and the shooting angle information of the third shooting waypoint is based on the shooting angle information of the first selected waypoint and the second selected waypoint. The camera angle information of the selected waypoint is calculated.

In some embodiments, the photographing route is a second-type route, the second-type route includes multiple sub-routes, and the third photographing waypoint is a photographing waypoint in the sub-route except the start point and the end point .

In some embodiments, in the shooting route, images taken by two adjacent shooting waypoints have a preset overlap rate, and the processor 32 is further configured to: according to the at least two selected shooting waypoints The location information and shooting angle information of the shooting route planning, as well as the preset overlapping ratio and the field of view of the shooting device, to determine the location information and shooting angle information of one or more planned shooting waypoints in the shooting route .

In some embodiments, the processor 32 is further configured to: determine the imaging of the panoramic image according to the field of view of the shooting device, the position information and the shooting angle information of the at least two selected shooting waypoints Range: Plan the shooting route according to the imaging range of the panoramic image, so as to determine the position information and shooting angle information of one or more planned shooting waypoints in the shooting route.

In some embodiments, the location information includes longitude, latitude and altitude.

In some embodiments, the shooting angle information includes at least one of the following: the direction of the nose of the drone or the direction of the field of view of the shooting device; or

The shooting device is set on the drone through a cloud platform, and the shooting angle information includes at least one of the following: the direction of the nose of the drone, the orientation of the cloud platform, or the field of view of the shooting device direction.

Various implementations described herein can be implemented using a computer readable medium such as computer software, hardware, or any combination thereof. For hardware implementation, the embodiments described herein can be implemented by using Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays ( FPGA), processors, controllers, microcontrollers, microprocessors, electronic units designed to perform the functions described herein. For software implementation, an embodiment such as a procedure or a function may be implemented with a separate software module that allows at least one function or operation to be performed. The software codes can be implemented by a software application (or program) written in any suitable programming language, which can be stored in memory and executed by a controller.

For the implementation process of the functions and functions of each unit in the above device, please refer to the implementation process of the corresponding steps in the above method for details, and details will not be repeated here.

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium including instructions, such as a memory including instructions, which are executable by a processor of an apparatus to perform the above method. For example, the non-transitory computer readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

A non-transitory computer-readable storage medium, enabling the terminal to execute the above method when instructions in the storage medium are executed by a processor of the terminal.

In some embodiments, the embodiment of the present application also provides a drone, including:

body;

a power system installed in the body for powering the unmanned aerial vehicle;

a photographing device installed on the body for photographing images; and,

The above-mentioned panoramic image shooting device installed in the body.

For example, please refer to FIG. 2 , the panoramic image capturing device may be a flight controller in an unmanned aerial vehicle.

In some embodiments, please refer to FIG. 1 , the embodiment of the present application also provides a panoramic image shooting system, including the above-mentioned unmanned aerial vehicle and a remote control device.

Exemplarily, the panoramic image may be displayed on the display device shown in FIG. 1 .

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is a relationship between these entities or operations. There is no such actual relationship or order between them. The term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements but also other elements not expressly listed elements, or also elements inherent in such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

The methods and devices provided by the embodiments of the present application have been described in detail above. The principles and implementation methods of the present application have been explained by using specific examples in this paper. The descriptions of the above embodiments are only used to help understand the methods and methods of the present application. core idea; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation and application scope. In summary, the content of this specification should not be construed as limiting the application .

Claims

A panoramic image shooting method, characterized in that the method is applied to an unmanned aerial vehicle, the unmanned aerial vehicle is equipped with a photographing device, and the method comprises:

Acquiring position information and shooting angle information of at least two selected shooting waypoints, the position information and shooting angle information of the at least two selected shooting waypoints are determined based on the user's shooting waypoint setting operation;

Plan the shooting route according to the position information and shooting angle information of the at least two selected shooting waypoints, so as to determine the position information and shooting angle information of one or more planned shooting waypoints in the shooting route;

During the flight of the UAV according to the shooting route, according to the position information and shooting angle information of the at least two selected shooting waypoints, and the positions of the one or more planned shooting waypoints Information and shooting angle information, control the shooting device to take multiple images, and the multiple images are used to generate a panoramic image.
The method according to claim 1, characterized in that, during the flight of the drone, the drone can send the image captured by the shooting device to a remote control device for display in real time;

The position information and shooting angle information of the at least two selected shooting waypoints are determined based on the user's shooting waypoint setting operation on the remote control device during the flight of the drone.
The method according to claim 1 or 2, wherein the location information includes altitude, and the at least two selected shooting waypoints include a first selected waypoint and a second selected waypoint, the The height of the first selected waypoint is different from that of the second selected waypoint, and the plane where the shooting route is located intersects the horizontal plane.
The method according to claim 3, wherein the location information also includes longitude and latitude;

If the longitude of the first selected waypoint is approximately the same as that of the second selected waypoint and/or the latitude of the first selected waypoint is approximately the same as that of the second selected waypoint, the photographing The plane on which the route is located is roughly perpendicular to the horizontal plane;

If the longitude of the first selected waypoint is different from that of the second selected waypoint and the latitude of the first selected waypoint is different from that of the second selected waypoint, the plane where the shooting route is located Intersects the horizontal plane and is not perpendicular.
The method according to claim 3, wherein the location information also includes longitude and latitude;

If the longitude of the first selected waypoint is approximately the same as that of the second selected waypoint and the latitude of the first selected waypoint is approximately the same as that of the second selected waypoint, the shooting route is The first type of route, the first type of route is a straight line;

If the longitude of the first selected waypoint is different from that of the second selected waypoint and/or the latitude of the first selected waypoint is different from that of the second selected waypoint, the shooting route is A second type of route, the second type of route includes a first sub-route and a second sub-route, the direction of the first sub-route is perpendicular to the direction of the second sub-route.
The method according to claim 5, wherein the shooting route is a second-type route, the first sub-route is roughly parallel to the horizontal plane, and the second sub-route intersects the horizontal plane;

Wherein, the heights of the shooting waypoints belonging to the same first sub-route are approximately the same.
The method according to claim 5, wherein the planned shooting waypoints include a first planned waypoint and a second planned waypoint, and the shooting angle information of the first planned waypoint is consistent with the first selected waypoint. The shooting angle information of one of the point and the second selected waypoint is the same, and the shooting angle information of the second planning waypoint is the same as that of the other one of the first selected waypoint and the second selected waypoint. The camera angle information is the same.
The method according to claim 7, wherein the shooting route is a second-type route, and the second-type route includes a plurality of sub-routes;

Wherein, the first planned waypoint and the second planned waypoint are the starting point or the ending point of the sub-route.
The method according to claim 7 or 8, wherein the planned shooting waypoint further includes a third planned waypoint, and the shooting angle information of the third planned waypoint is based on the first selected waypoint The shooting angle information of the point and the shooting angle information of the second selected waypoint are calculated.
The method according to claim 9, wherein the shooting route is a second-type route, and the second-type route includes a plurality of sub-routes;

Wherein, the third planned waypoint is a shooting waypoint in the sub-route except the starting point and the ending point.
The method according to claim 1, characterized in that, in the shooting route, the images taken by two adjacent shooting waypoints have a preset overlap rate, and according to the at least two selected shooting waypoints location information and shooting angle information for shooting route planning, including:

According to the position information and shooting angle information of the at least two selected shooting waypoints, as well as the preset overlapping ratio and the field of view of the shooting device, the shooting route planning is performed.
The method according to claim 1, wherein the planning of the shooting route according to the position information and shooting angle information of the at least two selected shooting waypoints includes:

determining the imaging range of the panoramic image according to the field of view of the photographing device, the position information and the photographing angle information of the at least two selected photographing waypoints;

The shooting route planning is performed according to the imaging range of the panoramic image.
The method according to claim 1, wherein the location information includes longitude, latitude and altitude.
The method according to claim 1, wherein the shooting angle information includes at least one of the following: the direction of the nose of the drone or the direction of the field of view of the shooting device; or

The shooting device is set on the drone through a cloud platform, and the shooting angle information includes at least one of the following: the direction of the nose of the drone, the orientation of the cloud platform, or the field of view of the shooting device direction.
A panoramic image capture device, characterized in that the device comprises:

memory for storing executable instructions;

one or more processors;

Wherein, when the one or more processors execute the executable instructions, they are individually or jointly configured to perform the method described in any one of claims 1 to 14.
A kind of unmanned aerial vehicle, is characterized in that, comprises:

body;

A power system, installed on the body, is used to provide power for the unmanned aerial vehicle;

a photographing device installed on the body for photographing images; and,

The panoramic image shooting device according to claim 15 installed on the body.
A panoramic image shooting system, characterized in that, comprising the unmanned aerial vehicle and remote control device as claimed in claim 16;

The unmanned aerial vehicle is also used to send a plurality of captured images to the remote control device;

The remote control device is used to stitch the multiple images into a panoramic image.
A computer-readable storage medium, wherein the computer-readable storage medium stores executable instructions, and when the executable instructions are executed by a processor, the method according to any one of claims 1 to 14 is implemented.