AU2017101749A4

AU2017101749A4 - Smart Data - Utilizing a combination of Airborne sensor imagery capture from drone and satellite, as well as ground based sensors automatically recognized in airborne imagery, and combined with smart processing to automate detection and presciption areas on farmland. The ground based sensors will include identification of soil moisture, nutrients, light, and any other relevant information. They will also contain on board GNSS, and be solar powered. All information including imagery will then be stored in a central and then algorithms run to give health of particular crops.

Info

Publication number: AU2017101749A4
Application number: AU2017101749A
Authority: AU
Inventors: Benjamin Harris
Original assignee: Nat Drones Pty Ltd
Current assignee: National Drones Pty Ltd
Priority date: 2017-12-14
Filing date: 2017-12-14
Publication date: 2018-02-01
Anticipated expiration: 2025-12-14

Abstract

Abstract The idea behind this device is to allow automation and combination of information and data cap tured from aerial sensors such as a drone, and ground based sensors. By using GNSS information in an Airborne target with automatic feature identification and recognition, as well as soil mea surement sensors underneath the device, and combining this with a multi-spectral sensor infor mation, the combined airborne and ground sensors will give a much more accurate analysis of the overall condition of the land. The devices are comprised of a target, visible from aerial imagery, which has an onboard Li GNSS receiver capable of receving correction information in post processing to allow centimetre level positioning, as wel as an on board companion computer to log the GNSS information. Also con nected to the companion computer on board are an array of soil measurement senors, capable of measuring PH, Nitrogen, Moisture contect, Electrical conductivity and any other relevant infor mation. All devices will have lOT connectivityto send data backto a central server, includingGNSS location and soil measurements at any given time. This data will be combined with drone flyover information to generate an accurate reporting tool for land owners and agronomists. Page 6

Description

Pagel

Page 2

National Drones Patent Information

Currently, drones are being utilized increasingly in farmingapplications and agriculture, to capture imagery of various descriptions for further analysis to be completed from a desktop, and to allow farmers and agronomists a high level overview of conditions on their property. Typical analysis is completed using multispectral cameras, which provide useful data for analysis and can help iden-tify areas which need attention.

One ofthe challenges with only collectingairborne sensor imagery, isthatthearea of intereststill needs to be ground truthed, or soil tested in certain areas. This patent application and invention aims to automate that process.

In the world of aerial surveying, ground targets are often used from airborne imagery to identify the location of surveyed markers on the ground. This allows control points to tie in the photogram-metry processes, and ensure the captured airborne imagery aligns with the real position of these surveyed targets on the ground.

This idea proposes to utilize ground based sensors for soil measurement, connected directly be-neath aerial targets on a property so that not only is the airborne sensor data used for reporting and analyis, but each location on the property (there could be hundreds) with a target soil measurement valúes attached to it, to also give more accurate reporting and analysis.

The aerial target will be solar powered, and have an on-board L1 GNSS receiver, to loga constant GPS position. It will record raw GNSS Information such as pseudorange and carrier phase Information. This can then be compared to a nearby L1/L2 receiver and post processed to give cen-timetre level precisión Information for each target/sensor. Within the target itself will be a com-panion Computer such as a raspberry pi, or intel edison for loggingof GNSS data runningcustom software. The top ofthe target will have a specific pattern such as a QRcode, which means it will automatically be identifable from the aerial imagery. The target/sensor will also incorpórate a light sensor, to measure the amount of sunlight at any given time.

Underneath this aerial target, will be a digital soil sensor/probe. This soil sensor will have the ability to measure moisture, nitrogen, PH, salt levels, electrical conductivity, and temperature. This data will also be logged at regular intervals on board the companion Computer in the aerial target above. All of these targets/sensors will have IOT connectivity and the ability to network between each other.

The smarts occur after the drone flyover, where the aerial target is captured in the imagery and timestamped. Duringthe photogrametric reconstruction ofthe orthophoto, software will iden-tify the aerial targets and match the correspondingsoil valúes to each location. This will give a more accuracte overall picture ofthe land quality, especially when combined with the multispectral imagery from above. Meteorological station Information such as rainfall, wind, sunlight and other Information can also be incorporated into the data for futher analysis.

National Drones Patent Information

By capturing this all of this data, the ability to run predictive analytics around yield Information year on year may be possible. The more data is captured and referenced, the better that machine learning becomes at analyisingthis data, especially when coupled with meteorlogical Information on a yearly basis.

All Information from ground based and aerial sensors (multispectral camera) will automatically be uploaded to a central server for data Processing and analysis. This will be custom software, which can then linkout through API's to farm machinery such as tractors for application of fertil-izers/herbicides or anything else that is deemed necessary by the algorithms.

The patent isto protectthe development of the hardware, and the software used to create the aerial target/soil sensor, as well as the software runningon each device, and the automation in the software processing of the combination of all of this relevant data.

The critical elements and design features of this patent are - 1) Combined solar powered aerial target with onboard computer/GNSS receiver/lnternetof things connectivity for data transmission and telemetry and soil sensors underneath. 2) Pattern on top for automatic software target location and Identification through software 3) Dual Frequency GNSS recelver nearby for correctlon of target location to the centimetre level 4) Automatic data upload to central processing server upon flight completion 4) Software Algorithms and code to allow reportlng on multi-spectral imagery as well as auto-mated soil analysis of the location of each aerial target.

Figure 1 depicts how the ecosystem will work as a whole.

Figure 2 depicts a side view of the combined soil sensor/aerial target schematlc.

Figure 3 depicts a top down view of the potential pattern/QR code for image recognition, howev-er alternative methodology may be used (such as barcodes)

Figure 4 depicts a typical target layout on farmland. The targets are not to scale. Due to the GNSS measurment and the associated soil measurement underneath, along with the pattern recognition the Geo-location of each target is known. As each soil sensor/sampler ¡s connected to a target dlrectly above it, the soil measurement for this location can also be correctly identified. As each target will have telemetry and Internet connectivity, this Information will automatically be sent to a central server for processing, along with the drone multispectral Imagery.

Claims

The claims definining the ¡nvention are as follows. 1) The Targets are proposed to be made of waterproof, weatherproof high density foam, have on board GNSS receivers, as well as solar for power, and a companion Computer for data Processing and logging. They will also utilize a distinctive pattern on top which can be automatically recognised by post processing/image recognition software, and associated via GNSS location with their appropriate soil sensor underneath
2) The targets may have a proximity reading RFID sensor to 'awaken' them when the drone flies over. They will record and upload soil levels consistently, along with their own internal battery levels.
3) The aerial targets will have attached soils probe underneath for different measurements as well as on board battery for data logging and storage. They will also have the ability for múltiple soil probes to be connected if necessary. The targets will have a built in light sensor, to measure the amount of sunlight at any given time. Múltiple probes can be used under the device to take measurements at different soil depths.
4) The targets and probes will all be part of an interconnected network of IOT Devices, constant-ly logging data backto a central server, along with drone recorded multispectral and RGB imagen/from regular flyovers. All ofthis data will be comblnedforaccurate reportingand analysis of crops.
5) The software used for processing all oftheground based and airborne Information will run custom algorithms, capable of producingprescription reports which linkoutto custom API'sfor otherfarm equipmentto complete application of producís.