AT510729A1 - SYSTEM AND METHOD FOR CREATING AIR PICTURES - Google Patents

Abstract

An aerial imaging system adapted to provide aerial images of a particular area of the earth's surface selected via a geoportal by means of a client and a communication network and electronically transmitted to a server calculating aircraft flight paths comprising a plurality take photographs of aerial photographs, whereby the server ultimately creates an overall image that is sent via client and communication network.

Description

'ON / 22 / NOV / 2010 16:43 FAX No: +49 89 54403080 p. 034 AEP005

≫ 7

description

The present invention relates to a system for generating aerial images, in particular a method which provides the user with aerial images of a selectable area of the earth's surface. 10 Aerial photography is usually high

Effort and expensive technical equipment. In particular, aerial photographs with georeferencing, ie the assignment of geographical reference information z, B. via GPS, require extensive technical equipment and know-15 how, which is not accessible to everyone, especially for cost reasons.

The object underlying the invention is to provide a system and a method for creating aerial photographs 20, which automates and simplifies the planning and execution of aerial photographs in such a way and thus also makes it more economical that basically everyone has the opportunity is to plan aerial photographs from the desk and to arrange their automated execution.

The invention achieves the object by a system according to claim 1. Exemplary embodiments of the system are the subject of the dependent claims. 30

In the course of the ongoing global networking, especially through the Internet and the ever-increasing efficiency of the corresponding network connections, there is the possibility of making electronic geoportals available. According to §3 Abs. 6 GeoZG (Spatial Data Access Law), an electronic geoportal is an electronic communication, transaction and interaction platform which 22/1 1 2010 MO 1B: 42 [SE / EM NR 7193] @ 004 .ON / 22 / NOV / 2010 16:43 AEP005 FAX no.: +43 89 54403080 ············· · P. 005 4 · * · # ··· ······································································································································································································································································· Maps under which the user can choose according to the individual requirements.

It describes a system for the automated creation of aerial photographs. In particular, the system includes a client that provides the user access to a geoportal 10 to enable him to select a desired area of the earth's surface.

The system further comprises a server which receives the information about the user-defined area of the earth's surface to be scanned from the client and calculates the flight path for at least one aircraft flying over the area. A communication network is required to implement the object of the invention electronically transferring the data identifying the selected area 20 from the client to the server. Usually, but not necessarily, the network is the Internet. The system additionally requires at least one suitable aircraft, for example an unmanned aerial drone, which is equipped with a camera arrangement 25 in order to be able to photograph the area desired by the user. When the area is completely scanned, the aerial images are sent to the server, which connects to the client, making the data easily accessible to the user.

The present invention also includes a method of creating aerial images using the system described above, the method comprising at least the following steps: selecting a region of the earth's surface using a geoportal on a client, electronically transmitting data to identify the selected 22/1 1 2010 MO 16:42 [SE / EM HR 7193] @ 005 10N / 22 / N0Y / 2Q1Q 16:43 AEPC05 • · «· · FAX Kr,: +49 89 54403080 S, 006 ►« «i *« · * · · · «III»

The area of the earth's surface from the client to a server via a communications network, which is, for example, the Internet, calculating a flight path for a plug device covering at least a portion of the selected area of the earth's surface, transmitting the flight route to the aircraft, scanning at least a portion of the selected area of the earth's surface using a suitable camera arrangement. The invention is explained in more detail below with reference to exemplary embodiments illustrated in the drawings. It shows:

FIG. 1A shows schematically the structure according to the invention

Implementation of the explained system main components;

FIG. 1B is a block diagram of a model for determining a position and a speed; and

2 shows schematically the essential components of a drone flight. 25 The measurement of geographical areas and the creation of special aerial photographs can be realized by using suitable aircraft (20), such as unmanned aerial vehicles or autonomous aerial drones. For example, such aircraft (20) can be used to document geographical changes after a natural event. Another application of aerial imagery taken using aerial drones is the delivery of raw data for virtual modeling of three-dimensional urban landscapes. There are quite a variety of other applications conceivable, such as agricultural or governmental 35 purposes. 22/1 1 2010 MO 16:42 [SE / EM NR 7193] @ 006 «0N / 22 / NÖV / 2010 16:43 FAX Nr.: +49 89 54403080 S. 007 AEP005

It is conceivable to use a geoportal which, inter alia, provides geodata services by electronic means for purposes of aerial photography of geographical areas by means of suitable aircraft. 5

FIG. 1A schematically shows by way of example how the components of the system for aerial photography can preferably cooperate according to the invention. The client (1) allows the user access to a geoportal (2) so that it can define the area of the earth's surface to be scanned by one or more aircraft (20) by means of corresponding edge markings. Via a communication network (4), preferably via the internet, the client (1) transfers the corresponding data defining the desired area to a server (3), which ultimately ensures that the selected area is completely occupied by at least one aircraft (20 ) is covered.

In an exemplary embodiment of the present invention, at least two aircraft (20) are used. The server (3) calculates the flight paths of the aircraft in such a way that each of the aircraft is assigned a subarea of the selected area of the earth's surface, wherein the subregions do not overlap each other in order to avoid 25 redundancies as far as possible.

As a payload carry the aircraft (20) according to the invention a camera assembly (10), which usually has a plurality of cameras, the lenses of some cameras are arranged obliquely 30, so that even from geographical slopes

Aerial images can be created without the aircraft (20) would have to tilt from the horizontal.

FIG. 2 is a schematic block diagram of an exemplary aircraft (20). The electric motor (5) seated in the drive train is shown with a battery (6) for power supply, the battery (6) also covering the components outside the drive train 22/1 1 2010 M0 16:42 [SE / EM HR 7193] @ 007 • ON / 22. / NOV / 2010 16:44 • «FAX No .: +49 89 54403080 ΑΕΡ005 of the aircraft. Furthermore, a suspension (7) is shown, which is preferably used for mounting the battery (6) and in particular the motor (7). The suspension (7) can be executed by the restliehen components of the aircraft (20) by 5 using a suitable damping (9) mechanically decoupled to intercept from the engine (7) caused vibrations. Thus the aerial photographs are not influenced by unwanted mechanical vibrations. Figure 2 further schematically shows a camera arrangement (10), a control unit 10 (11) and an autopilot unit (12) arranged and coupled in such a way as to communicate with each other. The control unit (11) also has a memory according to the invention, which, however, is not shown for the sake of clarity. 15

The aircraft used (20) can be, for example, unmanned aerial drones. The control unit (11) of a flying drone is designed to process GPS data of a GPS receiver arranged in the aircraft drone, to deposit aerial images recorded by the camera arrangement (10) in the memory unit, and to read out data from inertial sensors (IMU) and to process.

Inertial Measurement Units are known 25 and are used primarily for measuring translational and rotational acceleration behavior.

In particular, based on GPS data, the control unit (li) can assign geographic reference information 30 to the aerial images and provide them with a time stamp in the

Store storage unit. Including geographic reference information on a dataset, in this case the dataset corresponds to the aerial images, is often referred to in computer cartography as "georeferencing". Designated 35. According to the invention, the control unit synchronizes the aerial images by means of the time stamp with the inertial sensors and the GPS data

22/1 1 2010 MO 16:42 [SE / EM NR 7193] ® 0 0 S ION / 22 / NGV / 2010 16:44 FAX No.: +49 89 544030SC ♦ «· p. C09 AEP005 • ft * ·« «• · · · 6 stores in the memory unit. The measurements of the inertial sensors, the GPS data of the GPS receiver as well as the aerial images of the camera are accordingly provided with a time stamp. Typically, this time stamp is a reference time dictated by the GPS receiver. The integration of the inertial sensors into the system can be done via a known strapdown navigation algorithm. The inertial sensors provide corresponding sensor signals which can be fed as 10 system inputs to a Kalman filter. Via this Kalman filter, the sensor signals of the inertial sensors can then be linked to the navigation algorithm in a suitable manner. In FIG. 1C, a model or a method, in particular a state mode 11, for position and velocity determination, for example for use in the aircraft (20) according to the invention, is shown schematically in a block diagram. Since such models are known in principle, 20 At this point, only the essential aspects of the present invention will be explained.

A three-dimensional velocity vector ω = [ωχ, ωγ, ωζ] τ contains all the necessary information about the speed of a body, for example the aircraft (20), in a three-dimensional space. According to the invention, this velocity vector ω can be determined, for example, by means of a gyroscope (110) which is arranged in the aircraft (20). It can be arranged in the aircraft (20) and several 30 such gyroscopes (110). The use of other devices and devices suitable for measuring speeds is also conceivable. For the present state model, information 35 about the acceleration vector a = [ax, ay, a] T of the aircraft (20) is also desirable. To determine the acceleration vector 22 / Π 2010 MO 18:42 [SE / EM MR 7193] © 003 νίΟΝ / 2 2 / ΝΟ V / 2 010 16:44 FAX No.: +49 89 54403080 p. 010 ΑΕΡ005 ι · · * It is possible to provide a suitable acceleration measuring unit {130) which is arranged in the aircraft (20).

In the model shown in FIG. 2B, a suitable function block (120), in which corresponding model equations are stored, can be derived from FIG

Velocity vector ω and optionally, for example, an angular momentum or torque of the gyroscope (110) representing signal (160), for example, the altitude 10 calculated and a coordinate transformation (140) for the acceleration vector a are supplied. The coordinate transformation (140) can be used to calculate a transformed acceleration vector a. * ~ [Ax, ay, az] T 15, which depends on the altitude of the flight

Aircraft (120). A central functional block (150) contains the essential model equations necessary to implement the present model for position and velocity determination. Based on such 20 (known) model equations, both the speed and the exact position of the aircraft, including GPS data, can be determined. State variables that can underlie the present model can be represented as follows in vector notation: 25 χν = ίχ, y, z, vx, vy, νί (θ, φ, P = EpiT, .--, PiNlT, and x = [xv, p] T, 30 where the vector xv may include position, velocity B and altitude information, and the vector p may include in particular the positions of landmarks on the surface of the earth The states of the model may be taken together in vector x 35. We speak 22/1 1 2010 HO 16:42 [SE / EM NR 7193] @ 010 5ON / 22 / NOV / 2010 16:44 ΑΕΡ005 * · · »♦ • · FA): Nt. : +49 89 544C3080 * · · < * · • 4 · «p. 01

According to one embodiment (e.g., Slam algorithm) of the present invention, the state of the model (state map) is given by the vectors xv and p. If an aerial photograph is taken with the camera, then characteristic points will be searched (automatically) in the resulting aerial photograph. When a second aerial image is taken, these characteristic points, if present, are also searched for in the resulting second aerial photograph. However, this approach involves the risk of data association problems (e.g.

Assignment of the aerial images to each other). Once the corresponding characteristic points have been determined, the state vector p is dynamically expanded or reduced (i.e., entries added or removed) and a measurement update performed (i.e., new measurements 110, 130 performed) using appropriate measurement equations. These new measurements can be, for example, angle measurements. Advantages of the Slam algorithm are, for example, the fact that the aerial photographs can be created fully automatically on the one hand and joined together on the other fully automatically.

However, there are disadvantages associated with the Slam algorithm, that data association problems can occur, inaccuracies in aerial photography may arise, and that computational effort is significant. 25

According to another embodiment of the present invention, the characteristic points p may also be determined by landmarks applied to this area prior to scanning the desired geographic area of the earth's surface. Such markers can be provided with clearly identifiable patterns and thus easily recognized by the image processing in aerial photography. This eliminates the data association problem associated with using the Slam algorithm (see 35 above). The reconstruction of an aerial photograph from individual aerial photographs or aerial photographs will thus become £ 2/11 2010 MO 18:42 [SE / EM NR 7193] @ 011 * 9 * ··· · + · * ♦ iON / 22 / t! CV / 2GlO 16:44 AEP005 FAX No.: +49 89 54403080 £ .012

improved over the Slam algorithm. In addition, the computational effort is lower.

At least one aircraft (20) makes a plurality of aerial images from an area selected by the user 5, and when using a plurality of aircraft, each of them photographs a partial area of the selected area. Since the user generally desires a contiguous aerial image of the desired area, the aerial images of Fig. 10 are to be assembled into an overall image.

For this purpose, the plurality of aerial images are photographed in a manner and stored in memory, so that adjacent images overlap at least partially. From the memory of the aircraft (20) the stored plurality of individual aerial images is transmitted for the purpose of further processing. The server (3) can use image processing to combine the individual images into a seamless overall picture. Preferably, the known Image 20 stitching method is used here,

Ea is conceivable in one embodiment of the invention to distribute markers over the desired area of the earth's surface, which are recognizable for the cameras and appear visible on the aerial images 25.

If the overall picture is present, the user receives it electronically via the communication network (4) by means of the client S (1), without he himself having to become active.

Preferably, unmanned aerial drones according to the invention with motor (5), battery (6), suspension (7), outer shell (ö), damper (9), camera assembly (io), memory unit (11) and 35 autopilot unit (12) are used. 22/1 1 2010 MO 16:42 [SE / EM NR 7193] @ 012 EON / 22 / KOV / 2010 16:44 FAX No .: +49 89 54403080 p. G13 ΑΞΡ005 • · ttt · * * φ «» * «· ♦» ψ · · ψ · * i II «« io

Each motor, including an electric motor (5) as it is preferably used in a drone according to the invention, causes mechanical vibrations that are transmitted to all rigidly connected parts. Since the vibrations of the motor S (5) would thus also be transmitted to the cameras, a suitable damping device (9) is mounted between the suspension (7) of the drive train and the shell (8) of the aircraft drone. This therefore contributes to a high aerial quality, which is not mitigated by vibrational resonances originating from the 10 drivetrain.

There may be instances when a flying drone crashes due to a malfunction, for example. In order to increase the chances that the components installed in the aircraft drone will not be damaged in the event of a crash, it is advantageous to manufacture the outer shell (8) of the aircraft drone from materials which are capable of sudden mechanical load peaks, such as impacts, absorb and absorb. It is conceivable that materials such as EPP 20 (Expanding Polypropylene) and / or Styrofoam can realize these special properties with respect to the outer shell (8).

22/11 2010 MO 16:42 [SE / EM HR 7193] @ 013

Claims (20)

AON / 22 / NOV / 2310 15:44 FAX No,: +49 89 54403080 p. 014 AEP005 • · * * · * * m 11 1. A system for producing aerial photographs with the following components: 5 a client (1) with Access to a geoportal (2) adapted to allow a user to select a region of the earth's surface; a server (3); a communication network (4) for electronically transferring data for identifying the selected area of the earth's surface from the client (1) to the server (3); an aircraft (4) adapted to along a flight route at least part of the selected area to scan the earth's surface with the aid of a camera arrangement, the server (3) being designed to calculate the 2C flight path for the aircraft (4) from the data received by the client (2) for identification of the selected area of the earth's surface and to the aircraft transferred to. 2. System according to claim 1, comprising at least one further aircraft, wherein the server (3) is designed to 25 for each aircraft to calculate a flight route each over a portion of the selected area of the earth's surface. 3. System according to claim X or 2, wherein the aircraft 30 has a memory for storing the aerial images taken with the camera arrangement, A system according to claim 3, wherein each aircraft is equipped with a control unit adapted to process GPS data, store aerial images in the memory and read and process data from the inertial sensors (IMU). 22/1 1 2010 MO 1G: 42 [SE / EM NR 7193] 0014 1ON / 22 / NOV / 20 10 16:44 AEP005 FAX No.: +49 89 54403080 p. 015 M »··» ·· «» · · · · T «| * # * * * # »4 *« · · · # «* f I» 't · • t 4 I 4 ·· »·« «« «··« 12 5 10 15 20 25 30 5. The system according to claim 4, wherein the control unit is configured to use GPS data to associate the aerial images with geographic reference information and store the time-stamped aerial images in the memory. A system according to claim 5, wherein the aerial images are stored in memory in time-synchronized with initial bins and G-S data. 7. System according to one of the preceding claims, wherein each adjacent aerial images partially overlap. A system according to any one of claims 1 to 4, wherein the server (3) is adapted to merge the overlapping aerial images taken by the aircraft (s) by means of image processing into an overall image. A system according to claim 5, wherein the server (3) is adapted to transmit the entire image to the client (1) via the communication network (4). 10. A method of creating aerial images, comprising: selecting a region of the earth's surface using a portal on a client (1); electronically transmitting data for identifying the selected area of the earth's surface from the client (1) to a server (3) via a communications network (4); Calculating a flight path for an aircraft that deviates at least a portion of the selected area of the earth's surface; Transmitting the flight route to the aircraft; 22/11 2010 MO 16:42 [SE / EM NO 7193] ®015 MON / 22 / NOV / 20! Ο 16:45 ΑΕΡ005 I »*« «Μ4» • * * «· · · *« · · * * * * * * FAX No,: +48 89 54403080 v · · p. 016 • * «fl · 13 Scanning of at least a portion of the selected area of the earth's surface with the aid of a camera arrangement. A method according to claim 10, wherein edge marks are set using the geoportal which delimit the area of the earth's surface to be scanned. 10 15 12. A method according to any one of claims 10 or 11, wherein physical markers visible on the aerial images are placed in the selected area of the earth's surface. 13. The method according to any one of claims 10 to 11, wherein each adjacent aerial images partially overlap. 14. The method of claim 13, wherein the partially overlapping aerial images are joined together to form a continuous aerial image 20 according to an image stitching method. 15. The method according to any one of claims 10 to 14, wherein the control unit of an aircraft GPS data processed, stored aerial images and read data from the 25 inertial sensors (IMU) and managed. 16. A method according to any one of claims 10 to 15, wherein the control unit based on GPS data the aerial images unique geographic reference information assigns, 30 each picture with a time stamp and stores in memory. 17. The method of claim 10, wherein the control unit stores the time-stamped 35 aerial images in memory in a time-synchronized manner with the inertial sensors and the GPS data. 22/11 2010 MO 16:42 [SE / EM HR 7193] © 016 MON / 22 / N0V / 201Ο 16:45 • · FAX no.: +49 89 54403080 p. 017 ΑΕΡ005 14 A plug device for use in an aerial imaging system according to claim 3, comprising an electric motor, a battery for power supply, an autopilot unit and a device for supporting a camera assembly comprising at least one camera. 19. An aircraft for use in an aerial imaging system according to claim 18, wherein the aircraft is an unmanned aerial drone. 10 20. An aircraft according to claim 18, wherein the camera assembly is mechanically decoupled from the suspension of the electric motor and preferably comprises a plurality of cameras, the lenses are arranged at different angles 15 to each other. 22/1 1 2010 MO 16:42 [SE / EM NR 7193] @ 017