WO2020047579A1 - Method and system for plant stress determination and irrigation based thereon - Google Patents

Method and system for plant stress determination and irrigation based thereon Download PDF

Info

Publication number
WO2020047579A1
WO2020047579A1 PCT/AU2019/050919 AU2019050919W WO2020047579A1 WO 2020047579 A1 WO2020047579 A1 WO 2020047579A1 AU 2019050919 W AU2019050919 W AU 2019050919W WO 2020047579 A1 WO2020047579 A1 WO 2020047579A1
Authority
WO
WIPO (PCT)
Prior art keywords
thermal
plant stress
plant
video
irrigation
Prior art date
Application number
PCT/AU2019/050919
Other languages
French (fr)
Inventor
Mark Francis KLEMM
Andrew William WESTERN
Dongryeol RYU
David John Aughton
Original Assignee
Rubicon Research Pty Ltd
The University Of Melbourne
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from AU2018903312A external-priority patent/AU2018903312A0/en
Application filed by Rubicon Research Pty Ltd, The University Of Melbourne filed Critical Rubicon Research Pty Ltd
Priority to US17/273,931 priority Critical patent/US20210345567A1/en
Priority to AU2019333920A priority patent/AU2019333920A1/en
Priority to EA202190644A priority patent/EA202190644A1/en
Priority to BR112021004212-1A priority patent/BR112021004212A2/en
Priority to CA3115301A priority patent/CA3115301A1/en
Priority to EP19857549.0A priority patent/EP3846611A4/en
Publication of WO2020047579A1 publication Critical patent/WO2020047579A1/en

Links

Classifications

    • AHUMAN NECESSITIES
    • A01AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01GHORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
    • A01G25/00Watering gardens, fields, sports grounds or the like
    • A01G25/16Control of watering
    • A01G25/167Control by humidity of the soil itself or of devices simulating soil or of the atmosphere; Soil humidity sensors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • AHUMAN NECESSITIES
    • A01AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01GHORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
    • A01G25/00Watering gardens, fields, sports grounds or the like
    • A01G25/16Control of watering
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J5/00Radiation pyrometry, e.g. infrared or optical thermometry
    • G01J5/02Constructional details
    • G01J5/0265Handheld, portable
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J5/00Radiation pyrometry, e.g. infrared or optical thermometry
    • G01J5/02Constructional details
    • G01J5/04Casings
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J5/00Radiation pyrometry, e.g. infrared or optical thermometry
    • G01J5/02Constructional details
    • G01J5/08Optical arrangements
    • G01J5/0859Sighting arrangements, e.g. cameras
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J5/00Radiation pyrometry, e.g. infrared or optical thermometry
    • G01J5/02Constructional details
    • G01J5/08Optical arrangements
    • G01J5/0893Arrangements to attach devices to a pyrometer, i.e. attaching an optical interface; Spatial relative arrangement of optical elements, e.g. folded beam path
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B15/00Systems controlled by a computer
    • G05B15/02Systems controlled by a computer electric
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/174Segmentation; Edge detection involving the use of two or more images
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J5/00Radiation pyrometry, e.g. infrared or optical thermometry
    • G01J2005/0077Imaging
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J5/00Radiation pyrometry, e.g. infrared or optical thermometry
    • G01J5/02Constructional details
    • G01J5/03Arrangements for indicating or recording specially adapted for radiation pyrometers
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • G01N2021/8466Investigation of vegetal material, e.g. leaves, plants, fruits
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10048Infrared image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30181Earth observation
    • G06T2207/30188Vegetation; Agriculture
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/10Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
    • Y02A40/22Improving land use; Improving water use or availability; Controlling erosion

Definitions

  • the present invention relates to methods and systems for plant stress determination and automated irrigation management methods and systems based on plant stress determination.
  • PET Potential Evapotranspiration
  • PET is regarded as the maximum possible water use by crops with given meteorological condition and often overestimates the actual consumption of water by the crops.
  • PET can be converted to the actual evapotranspiration (AET) by adjusting it using crop growth indicator and water stress indicator.
  • AET becomes close to PET as crops are in full growth and in active photosynthetic mode with minimal sign of water stress.
  • the present invention is an extension of the irrigation management systems and methods disclosed in our International Patent Application No. PCT/AU2018/050858, the full contents including description, claims and drawings of which publication are assumed to have been read and incorporated herein by reference to avoid repetition of description.
  • This patent specification spatially derives soil moisture for any location within an irrigation district and the ability to map the soil moisture at any point in time and spatially across the irrigation district. The derivation is achieved using the monitored inputs of: 1. When the irrigation historical data was applied (including how much was applied and therefore having knowledge of the soil moisture status/level following the irrigation)
  • ET may be derived using the known Penman-Monteith equation together with crop information that computes ET from the following input variables:
  • Crop factor (derived from satellite thermal measurements)
  • Soil type at the location and therefore water holding capacity of the soil This is used to determine the irrigation point for the soil and therefore the optimal point at which the plant becomes stressed but does not incur any biological damage.
  • the specification discloses the use of input data from sensors to reflect the highly spatially variable nature of some of these parameters to, in-turn, derive the soil moisture which is also highly spatially variable in nature.
  • System identification techniques are used to produce an algorithm based on the above inputs and the use of soil moisture sensors at various locations in the field to derive the relationship and provide ongoing calibration.
  • a method of plant stress determination using a computer-based camera system having thermal imaging and visual imaging to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof; analyzing both thermal and visual images/video therefrom to form a composite image; determining the thermal activity of the composite
  • plant stress determination system including a computer-based camera system having thermal imaging and visual imaging to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof; analyzing both thermal and visual images/video therefrom to form a composite image; determining the thermal activity of the composite
  • said computer-based camera system is based on a smartphone.
  • said thermal imaging is captured by a thermal camera associated with said smartphone and said thermal imaging may be captured by a thermal camera associated with said smartphone.
  • said thermal and visual images capture at least one or more of information related to time of capture, GPS location, accelerometer data, direction data and inclinometer data.
  • said composite image uses edge detection of said foliage to align/co-register the visual and thermal images/video and the aligned/co-registered thermal images/video may be processed using augmented reality techniques to provide image measurement.
  • the plant stress determination further includes inputs from relevant derived evapotranspiration and soil moisture data from land to be irrigated to provide an irrigation schedule for an operator linked to a networked computer system overseeing said plant stress determination, said relevant derived evapotranspiration and soil moisture data from said land to be irrigated.
  • plant stress determination is calculated by said computer-based camera system and said thermal imaging and said visual imaging may be
  • a further aspect the plant stress determination further includes inputs from a surface representing a non-transpiring leaf whose temperature is measured and a surface representing a transpiring leaf whose temperature is measured.
  • an irrigation management system to irrigate predetermined areas of an irrigation district, said irrigation management system including:
  • a soil moisture determination system to spatially derive the soil moisture at a selected location within an irrigation district to be irrigated, said system including a networked computer system connected to a plurality of weather stations within said irrigation district to measure a selection from: solar radiation spectrum, wind speed, rainfall, temperature, humidity, barometric pressure, and energy measurement from solar panels at each of a plurality of representative locations, said networked computer system having data access to crop factor, soil type, and irrigation historical data at said representative locations, said networked computer system using system identification techniques to produce an algorithm for evapotranspiration based on a predetermined selection from the weather station measurements and the data access to crop factor, soil type, and irrigation historical data at said
  • said networked computer system calibrating said algorithm by direct measurement of the moisture in the soil at each of said representative locations by respective soil moisture sensors; and using measured parameters of rainfall, soil type, irrigation historical data and crop factor with said algorithm to derive or interpolate soil moisture at said selected location within said irrigation district; (b) a plant stress determination system as previously defined associated with said soil moisture determination system to receive said derived soil moisture from said soil moisture system;
  • irrigation management system monitoring said plant stress from plant stress system; said derived or interpolated soil moisture for said predetermined areas;
  • a still further aspect of the invention provides a method of scheduling irrigation of land including steps of determining plant stress using a computer-based camera system having thermal imaging and visual imaging to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof; analyzing both thermal and visual images/video therefrom to form a composite image; determining the thermal activity of the composite image/video and photosynthesis state of said at least one plant; deriving the plant stress from said determination; wherein the plant stress determination further includes inputs from relevant derived evapotranspiration and soil moisture data from land to be irrigated to provide an irrigation schedule to an operator linked to a networked computer system overseeing said plant stress determination, said relevant derived evapotranspiration and soil moisture data from said land to be irrigated.
  • a further aspect of the invention provides method of plant stress determination using a computer-based camera system having visible, near infrared, shortwave infrared, and thermal infrared imaging capability to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof; analyzing selections of composites from said visible and infrared bands in images/video; determining the water stress, leaf water content, leaf pigment condition and photosynthetic activity from the composite image/video of said at least one plant; and deriving the plant stress from said determination.
  • said computer-based camera system is based on a smartphone.
  • the near and shortwave infrared imaging can be captured by an infrared camera associated with said smartphone and said thermal imaging is captured by a thermal camera associated with said smartphone.
  • said thermal, near/shortwave infrared and visual images capture at least one or more of information related to time of capture, GPS location, accelerometer data, direction data and inclinometer data.
  • the composite image uses edge detection of said foliage to align/co-register the visual and thermal images/video and the aligned/co-registered thermal images/video are processed using augmented reality techniques to provide image measurement.
  • the plant stress determination further includes inputs from relevant derived
  • Figure 1 shows a flow diagram illustrating the components and operation of the plant stress determination system of the present invention
  • Figure 2 is a more enhanced drawing of the flow diagram of Figure 1 highlighting the components of the plant stress determination system
  • Figure 3 is a front perspective view of a computer-based camera system to provide thermal and visual images that can be used in the plant determination system;
  • Figure 4 is a rear perspective view of Figure 3;
  • Figure 5 is a front view of the computer-based camera system of Figure 3;
  • Figure 6 is a cross-sectional view along and in the direction of arrows 6-6 in Figure 5;
  • Figure 7 is an exploded view of Figure 3 showing the components forming the computer-based camera system
  • Figure 8 is an exploded view of Figure 4 showing the components forming the computer-based camera system.
  • Figure 9 is a perspective view of the computer-based camera system of Figure 3 showing the operation of the system in the field.
  • Embodiments of the present invention seek to provide improvements of our earlier invention disclosed in International Patent Application No. PCT/AU2018/050858 by determining plant stress or water stress at the location.
  • Plant stress water stress
  • Plant stress is a state where the plant is growing in non-ideal growth conditions that increase the demands made upon it.
  • the effects of stress can lead to deficiencies in growth, crop yields, permanent damage or death if the stress exceeds the plant tolerance limits. Accordingly, accurate irrigation for plants will reduce the stress to maximise plant growth and health.
  • the soil moisture is indirectly an indicator of the plant stress.
  • Plant stress is inversely a measure of the plant photosynthesis (i.e. productive growth). As the water stress builds in the plant the photosynthesis begins to decline. There is an optimal point of plant stress when the crop should be irrigated for water use efficiency and plant production.
  • the plant stress may also vary for different stages of the plant growth cycle. For example, higher stress may encourage root penetration of the soil which in turn can improve plant growth.
  • Figure 1 discloses a flow diagram including the operational aspects of our earlier invention fully described in International Patent Application No. PCT/AU2018/050858, the contents thereof having already been incorporated into this specification and are assumed to have been read.
  • the components and data discussed in our prior application PCT/AU2018/050858 are enclosed within the dashed lines 10.
  • a representative weather station 12 may be associated with a respective flow gate (not shown) incorporated into an irrigation management system (not shown) to provide measured parameters to assist in the derivation of the evapotranspiration as shown in the drawings of our prior application.
  • the weather station 12 inputs are typically temperature 16, solar radiation 18, wind speed 20, atmospheric pressure and humidity 22. Suitable compact weather stations are commercially available.
  • a preferred weather station has been described in International Patent Application No.
  • An algorithm for evapotranspiration 24 can be derived by a networked computer system (not shown) using the weather station 12 inputs with the addition of crop factor 26 based on readily available satellite/air thermal imagery 32, crop type 28 and crop status 30 through data access by the networked computer system.
  • the derived evapotranspiration 24 can then allow soil moisture 34 to be determined by the networked computer system.
  • Soil moisture 34 is derived using the inputs of rainfall 14 from weather station 12, irrigation historical data 36 together with soil type 38 stored in the networked computer system.
  • a representative soil moisture sensor 40 is monitored by the networked computer system to provide feedback to calibrate the algorithm used to derive the soil moisture 34.
  • a typical commercially available soil moisture sensor 40 would be 1 metre long and have measurement probes every 10 cm along its length. These measurement probes will provide a better soil moisture analysis as moisture penetration is important as well as surface moisture.
  • the present invention enhances the operation of the disclosure in International Patent Application No. PCT/AU2018/050858 shown within dashed lines 10.
  • the present invention uses the data from the derived soil moisture 34 as an input into the derivation 44 of plant/crop stress to further finetune irrigation schedules 42 provided by the networked computer system (not shown).
  • Foliage is the main organ of photosynthesis and transpiration in higher plants.
  • the invention provides field measurement (ground point source) 46 of foliage temperature using a thermal imaging camera (not shown). The measurement is taken at a known GPS location at a point in time.
  • the thermal image is taken by an infra-red camera that is coupled to a smartphone (not shown).
  • the infra-red camera is coupled to the smartphone (directly, or via Bluetooth or other wireless communication).
  • the smartphone is linked to the computer network system.
  • a thermal imaging camera is commercially available from FLIR Systems, Inc. for attachment to Apple IOS or Android smartphones.
  • the smartphone can provide the necessary GPS location and the date and time relating to the images taken.
  • the smartphone will run an application for ordering water that has access to the wider data inputs and evapotranspiration 24, soil moisture 34 and plant/crop stress analytics 44.
  • the smartphone will provide a dual image (thermal image 46 from the infra-red camera and visual image 50 from the smartphone camera) of the same view of the crop canopy.
  • the typical use of the smartphone will be to take the images of the crop canopy while the smartphone is handheld with a typical view area size of 1 m x 1 m with the smartphone held at a height of approximately 1 m above the ground.
  • the resulting images will have pixel resolution of approximately a millimetre.
  • the infra-red camera will firstly be used to calibrate satellite data 48 having coarse resolution and subject to infrequent satellite passes, typically - one to two weeks.
  • image processing techniques e.g. the open source imaging program ImageJ
  • the crop canopy image will be processed, and information gained on foliage size/density/colour within the particular view of the image that has been taken.
  • the infra-red image data 46 e.g. the radiated energy spectra
  • additional data 52 e.g. thermal analysis
  • the pairing will align or co-register the images.
  • the measure of the plant photosynthetic state is inversely proportional to the plant stress.
  • This point source analysis at ground level provides a micro analysis of the crop canopy at close proximity. It allows for an instantaneous indirect measure of the status of the plant's photosynthesis.
  • the visual image will be analysed to determine the content of the crop canopy in the view using visualisation software. Of particular interest will be the identification of the leaf matter as opposed to other crop content such as stems, including leaf petioles, nodes and internodes, shoots and flowers.
  • the smartphone will also be able to distinguish other non-plant content such as soil.
  • the visual analysis will also be to determine the size of the components identified in the view and establish, for example, if the leaf matter is old growth or new growth.
  • Plant stress is the object function of soil moisture derivation and estimating the optimal time to irrigate a crop i.e. the irrigation schedule 42.
  • the ground measurement of plant stress will be used within the system identification process to provide ongoing calibration of the irrigation scheduling system. Additional direct inputs to the derivation of plant stress at ground measurement will come from local weather data from weather station 12 such as ambient temperature, humidity, solar radiation, and wind speed. Time of day will also be relevant to the daily plant photosynthesis cycle. Crop type and crop growth stage will also be relevant to the plant stress. These inputs are best accessed via the derived evapotranspiration 24, but not exclusively.
  • the application for ordering water in the smartphone can also suggest parts of an irrigated area to take ground images to give best representation. For example, it would be beneficial to obtain ground measurements at a range of indicative soil moistures (e.g. low, medium and high).
  • the application for water ordering will not only provide an estimated reading on the plant stress for the crop at that location/time, but will also be used in a broader knowledge base/AI system to refine and calibrate the plant stress prediction system for that location (parameters such as soil type, weather) but also for crop type/variety that in turn can be applied across a wider operation.
  • Historic data 54 (e.g. yields) will provide feedback and learning on the optimal plant stress levels at which the irrigation schedule 42 can achieve the most efficient water use and at the same time achieve maximum crop production.
  • the continuous learning and refinement can not only be applied to a specific farm but also used across similar farming enterprises using the same/similar crops and varieties.
  • the platform provides for a wider knowledge base, calibration and learning. This process would employ computer-based data analytics and artificial intelligence algorithms.
  • the systems Identification techniques will be applied to develop a relationship for plant stress based on data inputs described, and also employing fine tuning and calibration via feedback from field measures such as the smartphone, soil moisture probes 40 and historic yield data 54.
  • the system identification processes described in this invention rely directly on sensor data (e.g. rainfall 14, temperature 16) and input data (e.g. crop type 28, soil type 38) to obtain interim derived outputs (e.g.
  • the inputs shown in (sensor and input data) to obtain the interim derived outputs are not limited to the shown structure.
  • temperature 16 may be a direct input to deriving plant stress.
  • the prior art has described the use of thermal images taken at higher altitudes above the crop canopy using towers or drones/aircraft. These are known as air thermal imaging. Towers provide continuous thermal imaging at a point source. Drones/aircraft provide spatial thermal detail but periodically when the drone/aircraft passes over the crop canopy. Typically, the pixel resolution can be 5 cm for a drone.
  • Satellite thermal images have also been obtained at a greater altitude again.
  • the pixel resolution can be 30 m for a Landsat satellite and 250 m for a MODIS satellite.
  • thermal imaging There is also an averaging associated with both types of thermal imaging. If, for example the crop is planted in rows the thermal image will be an average of the crop canopy and the exposed soil.
  • Obtaining images at a ground level overcomes the above resolution problems of both air thermal imaging and satellite thermal images.
  • the analysis that will be undertaken in this process will be able to distinguish between crop canopy and exposed soil.
  • thermal measurement of the leaf is an indication of the rate/level of evapotranspiration and accordingly, crop stress.
  • Thermal measurement of foreign items in the field of view e.g. stems, weeds or soil
  • thermal measurement techniques of a lesser resolution e.g. satellite/air
  • the pixel resolution will be 1 mm. This fine resolution can distinguish between leaf matter and stems, including leaf petioles, nodes and internodes, shoots and flowers.
  • the image processing will also be able to distinguish between old growth and new growth through learning techniques. The images will specifically focus on measuring the leaf matter of the crop canopy without interference from extraneous non-leaf based matter in order to measure the photosynthetic status of the plant.
  • Figure 2 expands the flow diagram shown in Figure 1.
  • a networked computer system 60 typically cloud based, continuously collects data including the derived stress 44 from the irrigation management system 62 and provides operational control based on the irrigation schedules 42 requested from farmers using a remote user interface 64, typically personal computers, personal communication devices
  • the networked computer system includes relevant databases 66 and receives data from weather stations 12, satellite imagery 48, nutrient and crop/yield status detection derived from thermal imaging, soil moisture sensors 40, soil surveys 68, and crop yields 54.
  • the thermal imaging camera 46 and visual imaging camera 50 On the left hand side of Figure 2 the flow chart operation of the thermal imaging camera 46 and visual imaging camera 50 is shown. As previously described, the thermal imaging camera 46 and visual imaging camera may also be substituted by other suitable arrangements and a dedicated system will be later described with reference to Figures 3 to 9.
  • the cameras 46,50 will photograph plants 70 in their field of view and can zoom in to isolate foliage, especially leaf matter. Synchronization of images taken will be controlled by visual imaging camera 50 through the software in the smartphone.
  • the thermal image/video 72 and visual image 74 will be processed at 76 in order to detect the leaf and edges from the of the image/video 72 and visual image 74.
  • the processing continues at 78 to pair the images/videos 72,76 so that a thermal analysis can be realized.
  • the results of the processing at 76 is also passed for image/video measurement using augmented reality techniques at 80.
  • the outputs from processes 78 and 80 are analyzed at 82 by algorithms to provide information on crop/plant recognition, crop type and variety, stage of growth, crop/plant health and recognizable diseases.
  • the outputs from 78 and 82 can be analyzed at 44 to derive plant/crop stress which can be passed to database 66.
  • the processing between the smartphone and networked computer system 60 through database 66 is a two way interaction as indicated by double head arrow 67. This interaction provides, for example:
  • the smartphone (also labelled 86 in Figures 3 to 9) having database 66 access to weather station 12 sensors and spatial interpolation specific to the site where the image is being taken.
  • Uploaded data can provide a growing information base from which machine learning, artificial intelligence and data analysis techniques can be applied in the ongoing improvement and knowledge via feedback mechanisms.
  • FIGS 3 to 9 show a handheld computer-based camera system 84 that can replace the previously described smartphone with attached thermal imaging camera.
  • a smartphone 86 with integrated photographic lens(es) 87 will allow visual images and videos to be taken.
  • the smartphone will include a USB connector 88 and Bluetooth wireless communication functionality.
  • the required software to run the handheld system 84 can be installed as an application to control operation thereof and communicate with the networked computer system 60.
  • Smartphone 86 co-operates with a handheld body 90.
  • Body 90 has a handgrip 92 containing a battery 94 and battery cover 96 to power the electronics contained in body 90.
  • a sensor shield 98 will protect the thermal imaging device 100 and related circuitry 102 from ingress of moisture and contaminants.
  • a phone cradle 104 is shaped to snugly fit smartphone 86 and may include a location switch to confirm smartphone 86 is correctly seated in the phone cradle 104. This switch would also trigger automatic Bluetooth 'pairing' of smartphone 86.
  • a USB connector 106 can be linked to circuitry 102 to provide coupling with USB connector 88 of smartphone 86.
  • a switch 108 is coupled to circuitry 102 to provide the simultaneous operation of both the smartphone camera lens(es) 87 and the thermal imaging device 100 for photographing plants 70.
  • a radar or ultrasonic sensor 110 may assist in obtaining measurement data of the plant foliage 70 in conjunction with the MEMS sensors of smartphone 86 and the augmented reality software 80 ( Figure 2).
  • the radar or ultrasonic sensor 110 will provide a distance measurement between plant 70 in the field of view and the computer-based camera system 84.
  • the computer-based camera system 84 may also include one or more laser sensors 112 to further support measurements of plant foliage 70 and assist with image pairing.
  • the laser sensors 112 can be dot or line lasers.
  • Smartphone 86 will capture data 114 (Figure 2) related to the processing of image/video.
  • Data 114 may include time of capture and microelectromechanical sensors within smartphone 86 including GPS, accelerometer, solid state compass and inclinometer.
  • Figure 9 illustrates the use of camera system 84 photographing plant 70.
  • the software may also suggest where to take the image/video and may also require the correct positioning of the handheld computer-based camera system 84 before an image/video can be recorded and crop stress computed/displayed.
  • the software may have control of the angular positioning of the device to ensure a valid 'Field of View' is being observed.
  • the computer-based suggestions on positioning can be passed to the user on screen 116.
  • the camera system 84 may also include a near infrared and/or a shortwave infrared imaging device. Such a device may allow leaf water content information to be determined.
  • the camera system 84 proposed takes advantages of both crop growth/vigour and water stress information measured simultaneously in the close range from the target crop. By combining visible, near infrared, and thermal infrared images along with on-site ancillary information such as local meteorological data, crop type/growth, and portable calibration target, the device can produce plant-by-plant estimates of crop water stress, crop vigour, and water consumption with a minimal reliance on empirical crop parameters.
  • Embodiments of the invention have been described above by way of non-limiting example only. In practice, a plurality of weather stations 12, flow gates, flow meters and soil moisture sensors 40 are scattered around the irrigation district to provide an extensive irrigation system. Variations and modifications to the embodiments may be made without departing from the scope of the invention.

Abstract

The invention provides various methods of plant stress determination including using a computer-based camera system (84) having thermal imaging (46) and visual imaging (50) to capture foliage at close proximity of at least one plant (70) to provide high resolution images/video (72, 74) thereof; analyzing both thermal and visual images/video (72, 74) therefrom to form a composite image; determining the thermal activity of the composite image/video and photosynthesis state of said at least one plant (70); and deriving the plant stress from said determination. A handheld computer-based camera system (84) is also disclosed having a smartphone (86) and thermal imaging device (100).

Description

METHOD AND SYSTEM FOR PLANT STRESS DETERMINATION AND
IRRIGATION BASED THEREON
The present invention claims priority from Australian Provisional Patent Application No. 2018903312 filed on 5 September 2018, the entire contents of which are incorporated herein by reference.
FIELD
The present invention relates to methods and systems for plant stress determination and automated irrigation management methods and systems based on plant stress determination.
BACKGROUND
Over agricultural fields, the standard meteorological observations - air temperature, relative humidity, solar radiation, wind speed - are used to estimate the Potential Evapotranspiration (PET), which constitutes the primary use of water in agriculture. PET is calculated conventionally by a Penman-Monteith approach based FA056 model. PET can be estimated using only the meteorological input and assumes that there is no limitation of water in soil available for the evapotranspiration.
Consequently, PET is regarded as the maximum possible water use by crops with given meteorological condition and often overestimates the actual consumption of water by the crops. PET can be converted to the actual evapotranspiration (AET) by adjusting it using crop growth indicator and water stress indicator. AET becomes close to PET as crops are in full growth and in active photosynthetic mode with minimal sign of water stress.
The present invention is an extension of the irrigation management systems and methods disclosed in our International Patent Application No. PCT/AU2018/050858, the full contents including description, claims and drawings of which publication are assumed to have been read and incorporated herein by reference to avoid repetition of description. This patent specification spatially derives soil moisture for any location within an irrigation district and the ability to map the soil moisture at any point in time and spatially across the irrigation district. The derivation is achieved using the monitored inputs of: 1. When the irrigation historical data was applied (including how much was applied and therefore having knowledge of the soil moisture status/level following the irrigation)
2. Computing the continuous evapotranspiration (ET) at the location and
therefore the accumulated decline in soil moisture at the location. ET may be derived using the known Penman-Monteith equation together with crop information that computes ET from the following input variables:
a. Solar radiation
b. Temperature
c. Humidity
d. Wind speed
e. Crop factor (derived from satellite thermal measurements)
3. Measuring any rainfall events and deducing the precipitation at the location and the subsequent increase in soil moisture at the location
4. Soil type at the location and therefore water holding capacity of the soil. This is used to determine the irrigation point for the soil and therefore the optimal point at which the plant becomes stressed but does not incur any biological damage.
The specification discloses the use of input data from sensors to reflect the highly spatially variable nature of some of these parameters to, in-turn, derive the soil moisture which is also highly spatially variable in nature.
System identification techniques are used to produce an algorithm based on the above inputs and the use of soil moisture sensors at various locations in the field to derive the relationship and provide ongoing calibration.
It is not admitted that any of the information in this patent specification is common general knowledge, or that the person skilled in the art could be reasonably expected to ascertain or understand it, regard it as relevant, or combine it in any way before the priority date. SUMMARY OF THE INVENTION
In one aspect of the invention there is provided a method of plant stress determination using a computer-based camera system having thermal imaging and visual imaging to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof; analyzing both thermal and visual images/video therefrom to form a composite image; determining the thermal activity of the composite
image/video and photosynthesis state of said at least one plant; and deriving the plant stress from said determination.
In a further aspect there is provided plant stress determination system including a computer-based camera system having thermal imaging and visual imaging to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof; analyzing both thermal and visual images/video therefrom to form a composite image; determining the thermal activity of the composite
image/video and photosynthesis state of said at least one plant; and deriving the plant stress from said determination.
Preferably said computer-based camera system is based on a smartphone. In one aspect said thermal imaging is captured by a thermal camera associated with said smartphone and said thermal imaging may be captured by a thermal camera associated with said smartphone.
In a preferred embodiment said thermal and visual images capture at least one or more of information related to time of capture, GPS location, accelerometer data, direction data and inclinometer data.
In a further aspect said composite image uses edge detection of said foliage to align/co-register the visual and thermal images/video and the aligned/co-registered thermal images/video may be processed using augmented reality techniques to provide image measurement.
In a further embodiment the plant stress determination further includes inputs from relevant derived evapotranspiration and soil moisture data from land to be irrigated to provide an irrigation schedule for an operator linked to a networked computer system overseeing said plant stress determination, said relevant derived evapotranspiration and soil moisture data from said land to be irrigated.
Another aspect the plant stress determination is calculated by said computer-based camera system and said thermal imaging and said visual imaging may be
synchronized to be taken at the same time.
A further aspect the plant stress determination further includes inputs from a surface representing a non-transpiring leaf whose temperature is measured and a surface representing a transpiring leaf whose temperature is measured.
In yet a further aspect of the invention there is provided an irrigation management system to irrigate predetermined areas of an irrigation district, said irrigation management system including:
(a) a soil moisture determination system to spatially derive the soil moisture at a selected location within an irrigation district to be irrigated, said system including a networked computer system connected to a plurality of weather stations within said irrigation district to measure a selection from: solar radiation spectrum, wind speed, rainfall, temperature, humidity, barometric pressure, and energy measurement from solar panels at each of a plurality of representative locations, said networked computer system having data access to crop factor, soil type, and irrigation historical data at said representative locations, said networked computer system using system identification techniques to produce an algorithm for evapotranspiration based on a predetermined selection from the weather station measurements and the data access to crop factor, soil type, and irrigation historical data at said
representative locations; said networked computer system calibrating said algorithm by direct measurement of the moisture in the soil at each of said representative locations by respective soil moisture sensors; and using measured parameters of rainfall, soil type, irrigation historical data and crop factor with said algorithm to derive or interpolate soil moisture at said selected location within said irrigation district; (b) a plant stress determination system as previously defined associated with said soil moisture determination system to receive said derived soil moisture from said soil moisture system;
and said irrigation management system monitoring said plant stress from plant stress system; said derived or interpolated soil moisture for said predetermined areas;
availability of irrigation water to said predetermined areas; and requests for timed irrigation from end users for said predetermined areas; and allowing requested irrigation to occur for said predetermined areas based on said monitoring.
A still further aspect of the invention provides a method of scheduling irrigation of land including steps of determining plant stress using a computer-based camera system having thermal imaging and visual imaging to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof; analyzing both thermal and visual images/video therefrom to form a composite image; determining the thermal activity of the composite image/video and photosynthesis state of said at least one plant; deriving the plant stress from said determination; wherein the plant stress determination further includes inputs from relevant derived evapotranspiration and soil moisture data from land to be irrigated to provide an irrigation schedule to an operator linked to a networked computer system overseeing said plant stress determination, said relevant derived evapotranspiration and soil moisture data from said land to be irrigated.
In yet a further aspect of the inventon there is provided a method of gaining
information on atmospheric conditions related to plant stress by using a thermal target that has:
(a) a surface representing a non-transpiring leaf whose temperature is measured and
(b) a surface representing a transpiring leaf whose temperature is measured.
A further aspect of the invention provides method of plant stress determination using a computer-based camera system having visible, near infrared, shortwave infrared, and thermal infrared imaging capability to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof; analyzing selections of composites from said visible and infrared bands in images/video; determining the water stress, leaf water content, leaf pigment condition and photosynthetic activity from the composite image/video of said at least one plant; and deriving the plant stress from said determination.
Preferably said computer-based camera system is based on a smartphone. The near and shortwave infrared imaging can be captured by an infrared camera associated with said smartphone and said thermal imaging is captured by a thermal camera associated with said smartphone.
In a preferred embodiment said thermal, near/shortwave infrared and visual images capture at least one or more of information related to time of capture, GPS location, accelerometer data, direction data and inclinometer data.
The composite image uses edge detection of said foliage to align/co-register the visual and thermal images/video and the aligned/co-registered thermal images/video are processed using augmented reality techniques to provide image measurement.
The plant stress determination further includes inputs from relevant derived
evapotranspiration and soil moisture data from land to be irrigated to provide an irrigation schedule for an operator linked to a networked computer system overseeing said plant stress determination, said relevant derived evapotranspiration and soil moisture data from said land to be irrigated.
BRIEF DESCRIPTION OF DRAWINGS
An embodiment of the method and apparatus will now be described by way of example only with reference to the accompanying drawing in which:
Figure 1 shows a flow diagram illustrating the components and operation of the plant stress determination system of the present invention;
Figure 2 is a more enhanced drawing of the flow diagram of Figure 1 highlighting the components of the plant stress determination system;
Figure 3 is a front perspective view of a computer-based camera system to provide thermal and visual images that can be used in the plant determination system; Figure 4 is a rear perspective view of Figure 3;
Figure 5 is a front view of the computer-based camera system of Figure 3;
Figure 6 is a cross-sectional view along and in the direction of arrows 6-6 in Figure 5;
Figure 7 is an exploded view of Figure 3 showing the components forming the computer-based camera system;
Figure 8 is an exploded view of Figure 4 showing the components forming the computer-based camera system; and
Figure 9 is a perspective view of the computer-based camera system of Figure 3 showing the operation of the system in the field.
DESCRIPTION OF EMBODIMENTS
Embodiments of the present invention seek to provide improvements of our earlier invention disclosed in International Patent Application No. PCT/AU2018/050858 by determining plant stress or water stress at the location. Plant stress (water stress) is a state where the plant is growing in non-ideal growth conditions that increase the demands made upon it. The effects of stress can lead to deficiencies in growth, crop yields, permanent damage or death if the stress exceeds the plant tolerance limits. Accordingly, accurate irrigation for plants will reduce the stress to maximise plant growth and health. In effect the soil moisture is indirectly an indicator of the plant stress. Plant stress is inversely a measure of the plant photosynthesis (i.e. productive growth). As the water stress builds in the plant the photosynthesis begins to decline. There is an optimal point of plant stress when the crop should be irrigated for water use efficiency and plant production. The plant stress may also vary for different stages of the plant growth cycle. For example, higher stress may encourage root penetration of the soil which in turn can improve plant growth.
Figure 1 discloses a flow diagram including the operational aspects of our earlier invention fully described in International Patent Application No. PCT/AU2018/050858, the contents thereof having already been incorporated into this specification and are assumed to have been read. The components and data discussed in our prior application PCT/AU2018/050858 are enclosed within the dashed lines 10. A representative weather station 12 may be associated with a respective flow gate (not shown) incorporated into an irrigation management system (not shown) to provide measured parameters to assist in the derivation of the evapotranspiration as shown in the drawings of our prior application. The weather station 12 inputs are typically temperature 16, solar radiation 18, wind speed 20, atmospheric pressure and humidity 22. Suitable compact weather stations are commercially available. A preferred weather station has been described in International Patent Application No. PCT/AU2018/051205, the full contents including description, claims and drawings of which publication are assumed to have been read and are incorporated herein by reference to avoid repetition of description. An algorithm for evapotranspiration 24 can be derived by a networked computer system (not shown) using the weather station 12 inputs with the addition of crop factor 26 based on readily available satellite/air thermal imagery 32, crop type 28 and crop status 30 through data access by the networked computer system.
The derived evapotranspiration 24 can then allow soil moisture 34 to be determined by the networked computer system. Soil moisture 34 is derived using the inputs of rainfall 14 from weather station 12, irrigation historical data 36 together with soil type 38 stored in the networked computer system. A representative soil moisture sensor 40 is monitored by the networked computer system to provide feedback to calibrate the algorithm used to derive the soil moisture 34. A typical commercially available soil moisture sensor 40 would be 1 metre long and have measurement probes every 10 cm along its length. These measurement probes will provide a better soil moisture analysis as moisture penetration is important as well as surface moisture.
The present invention enhances the operation of the disclosure in International Patent Application No. PCT/AU2018/050858 shown within dashed lines 10. The present invention uses the data from the derived soil moisture 34 as an input into the derivation 44 of plant/crop stress to further finetune irrigation schedules 42 provided by the networked computer system (not shown). Foliage is the main organ of photosynthesis and transpiration in higher plants. The invention provides field measurement (ground point source) 46 of foliage temperature using a thermal imaging camera (not shown). The measurement is taken at a known GPS location at a point in time. The thermal image is taken by an infra-red camera that is coupled to a smartphone (not shown). The infra-red camera is coupled to the smartphone (directly, or via Bluetooth or other wireless communication). The smartphone is linked to the computer network system. Such a thermal imaging camera is commercially available from FLIR Systems, Inc. for attachment to Apple IOS or Android smartphones. The smartphone can provide the necessary GPS location and the date and time relating to the images taken. The smartphone will run an application for ordering water that has access to the wider data inputs and evapotranspiration 24, soil moisture 34 and plant/crop stress analytics 44. The smartphone will provide a dual image (thermal image 46 from the infra-red camera and visual image 50 from the smartphone camera) of the same view of the crop canopy. The typical use of the smartphone will be to take the images of the crop canopy while the smartphone is handheld with a typical view area size of 1 m x 1 m with the smartphone held at a height of approximately 1 m above the ground. The resulting images will have pixel resolution of approximately a millimetre. The infra-red camera will firstly be used to calibrate satellite data 48 having coarse resolution and subject to infrequent satellite passes, typically - one to two weeks. Using known image processing techniques e.g. the open source imaging program ImageJ, the crop canopy image will be processed, and information gained on foliage size/density/colour within the particular view of the image that has been taken. The infra-red image data 46 (e.g. the radiated energy spectra) will also be 'paired' with the visual image 50 providing additional data 52 (e.g. thermal analysis) for specific foliage within the view. Of particular focus is the 'pairing' of the two images so that the thermal image of the leaf matter is known. The pairing will align or co-register the images. There is a direct correlation between the thermal status (temperature) of the leaf matter and the state of the plant's photosynthesis. The measure of the plant photosynthetic state is inversely proportional to the plant stress. This point source analysis at ground level provides a micro analysis of the crop canopy at close proximity. It allows for an instantaneous indirect measure of the status of the plant's photosynthesis. The visual image will be analysed to determine the content of the crop canopy in the view using visualisation software. Of particular interest will be the identification of the leaf matter as opposed to other crop content such as stems, including leaf petioles, nodes and internodes, shoots and flowers. The smartphone will also be able to distinguish other non-plant content such as soil. The visual analysis will also be to determine the size of the components identified in the view and establish, for example, if the leaf matter is old growth or new growth.
Plant stress is the object function of soil moisture derivation and estimating the optimal time to irrigate a crop i.e. the irrigation schedule 42. The ground measurement of plant stress will be used within the system identification process to provide ongoing calibration of the irrigation scheduling system. Additional direct inputs to the derivation of plant stress at ground measurement will come from local weather data from weather station 12 such as ambient temperature, humidity, solar radiation, and wind speed. Time of day will also be relevant to the daily plant photosynthesis cycle. Crop type and crop growth stage will also be relevant to the plant stress. These inputs are best accessed via the derived evapotranspiration 24, but not exclusively.
The application for ordering water in the smartphone can also suggest parts of an irrigated area to take ground images to give best representation. For example, it would be beneficial to obtain ground measurements at a range of indicative soil moistures (e.g. low, medium and high). The application for water ordering will not only provide an estimated reading on the plant stress for the crop at that location/time, but will also be used in a broader knowledge base/AI system to refine and calibrate the plant stress prediction system for that location (parameters such as soil type, weather) but also for crop type/variety that in turn can be applied across a wider operation.
Historic data 54 (e.g. yields) will provide feedback and learning on the optimal plant stress levels at which the irrigation schedule 42 can achieve the most efficient water use and at the same time achieve maximum crop production.
The continuous learning and refinement can not only be applied to a specific farm but also used across similar farming enterprises using the same/similar crops and varieties. The platform provides for a wider knowledge base, calibration and learning. This process would employ computer-based data analytics and artificial intelligence algorithms.
The systems Identification techniques will be applied to develop a relationship for plant stress based on data inputs described, and also employing fine tuning and calibration via feedback from field measures such as the smartphone, soil moisture probes 40 and historic yield data 54. The system identification processes described in this invention rely directly on sensor data (e.g. rainfall 14, temperature 16) and input data (e.g. crop type 28, soil type 38) to obtain interim derived outputs (e.g.
evapotranspiration 24, soil moisture 34) in the overall process of producing an optimal irrigation schedule 42. The inputs shown in (sensor and input data) to obtain the interim derived outputs are not limited to the shown structure. For example, temperature 16 may be a direct input to deriving plant stress.
An additional embodiment of this patent will be that the smartphone device will provide instant output of the plant stress for the crop captured in the view.
Alternatively, the prior art has described the use of thermal images taken at higher altitudes above the crop canopy using towers or drones/aircraft. These are known as air thermal imaging. Towers provide continuous thermal imaging at a point source. Drones/aircraft provide spatial thermal detail but periodically when the drone/aircraft passes over the crop canopy. Typically, the pixel resolution can be 5 cm for a drone.
Satellite thermal images have also been obtained at a greater altitude again. The higher the altitude of imaging the less resolution there is with detail in the images. Typically, the pixel resolution can be 30 m for a Landsat satellite and 250 m for a MODIS satellite. There is also an averaging associated with both types of thermal imaging. If, for example the crop is planted in rows the thermal image will be an average of the crop canopy and the exposed soil.
Obtaining images at a ground level, in accordance with the invention, overcomes the above resolution problems of both air thermal imaging and satellite thermal images. The analysis that will be undertaken in this process will be able to distinguish between crop canopy and exposed soil. As evapotranspiration occurs in the leaf, and thermal measurement of the leaf is an indication of the rate/level of evapotranspiration and accordingly, crop stress. Thermal measurement of foreign items in the field of view (e.g. stems, weeds or soil) would produce errors in this measurement as they are at a different temperature to that of the leaf. Thus, thermal measurement techniques of a lesser resolution (e.g. satellite/air) cannot distinguish leaf temperature accurately as they are averaging temperatures over a much larger area that includes the foreign items. Typically, the pixel resolution will be 1 mm. This fine resolution can distinguish between leaf matter and stems, including leaf petioles, nodes and internodes, shoots and flowers. The image processing will also be able to distinguish between old growth and new growth through learning techniques. The images will specifically focus on measuring the leaf matter of the crop canopy without interference from extraneous non-leaf based matter in order to measure the photosynthetic status of the plant.
These image processing techniques are of a known science and in the public domain.
Figure 2 expands the flow diagram shown in Figure 1. In order to avoid duplication of description, identical integers in Figure 1 will have the same reference numerals and function in Figure 1. The right hand side of Figure 2 shows a networked computer system 60, typically cloud based, continuously collects data including the derived stress 44 from the irrigation management system 62 and provides operational control based on the irrigation schedules 42 requested from farmers using a remote user interface 64, typically personal computers, personal communication devices
(smartphones, tablet computers or similar devices). The networked computer system includes relevant databases 66 and receives data from weather stations 12, satellite imagery 48, nutrient and crop/yield status detection derived from thermal imaging, soil moisture sensors 40, soil surveys 68, and crop yields 54.
On the left hand side of Figure 2 the flow chart operation of the thermal imaging camera 46 and visual imaging camera 50 is shown. As previously described, the thermal imaging camera 46 and visual imaging camera may also be substituted by other suitable arrangements and a dedicated system will be later described with reference to Figures 3 to 9. The cameras 46,50 will photograph plants 70 in their field of view and can zoom in to isolate foliage, especially leaf matter. Synchronization of images taken will be controlled by visual imaging camera 50 through the software in the smartphone. The thermal image/video 72 and visual image 74 will be processed at 76 in order to detect the leaf and edges from the of the image/video 72 and visual image 74. The processing continues at 78 to pair the images/videos 72,76 so that a thermal analysis can be realized. The results of the processing at 76 is also passed for image/video measurement using augmented reality techniques at 80. The outputs from processes 78 and 80 are analyzed at 82 by algorithms to provide information on crop/plant recognition, crop type and variety, stage of growth, crop/plant health and recognizable diseases. The outputs from 78 and 82 can be analyzed at 44 to derive plant/crop stress which can be passed to database 66. The processing between the smartphone and networked computer system 60 through database 66 is a two way interaction as indicated by double head arrow 67. This interaction provides, for example:
• The smartphone (also labelled 86 in Figures 3 to 9) having database 66 access to weather station 12 sensors and spatial interpolation specific to the site where the image is being taken.
• The potential to upload images and other data obtained from the smartphone when suitable internet access is available.
• Uploaded data can provide a growing information base from which machine learning, artificial intelligence and data analysis techniques can be applied in the ongoing improvement and knowledge via feedback mechanisms.
Figures 3 to 9 show a handheld computer-based camera system 84 that can replace the previously described smartphone with attached thermal imaging camera. A smartphone 86 with integrated photographic lens(es) 87 will allow visual images and videos to be taken. The smartphone will include a USB connector 88 and Bluetooth wireless communication functionality. The required software to run the handheld system 84 can be installed as an application to control operation thereof and communicate with the networked computer system 60. Smartphone 86 co-operates with a handheld body 90. Body 90 has a handgrip 92 containing a battery 94 and battery cover 96 to power the electronics contained in body 90. A sensor shield 98 will protect the thermal imaging device 100 and related circuitry 102 from ingress of moisture and contaminants. A phone cradle 104 is shaped to snugly fit smartphone 86 and may include a location switch to confirm smartphone 86 is correctly seated in the phone cradle 104. This switch would also trigger automatic Bluetooth 'pairing' of smartphone 86. A USB connector 106 can be linked to circuitry 102 to provide coupling with USB connector 88 of smartphone 86.
A switch 108 is coupled to circuitry 102 to provide the simultaneous operation of both the smartphone camera lens(es) 87 and the thermal imaging device 100 for photographing plants 70. A radar or ultrasonic sensor 110 may assist in obtaining measurement data of the plant foliage 70 in conjunction with the MEMS sensors of smartphone 86 and the augmented reality software 80 (Figure 2). The radar or ultrasonic sensor 110 will provide a distance measurement between plant 70 in the field of view and the computer-based camera system 84. The computer-based camera system 84 may also include one or more laser sensors 112 to further support measurements of plant foliage 70 and assist with image pairing. The laser sensors 112 can be dot or line lasers.
Smartphone 86 will capture data 114 (Figure 2) related to the processing of image/video. Data 114 may include time of capture and microelectromechanical sensors within smartphone 86 including GPS, accelerometer, solid state compass and inclinometer. Figure 9 illustrates the use of camera system 84 photographing plant 70. The software may also suggest where to take the image/video and may also require the correct positioning of the handheld computer-based camera system 84 before an image/video can be recorded and crop stress computed/displayed. The software may have control of the angular positioning of the device to ensure a valid 'Field of View' is being observed. The computer-based suggestions on positioning can be passed to the user on screen 116.
In an additional embodiment, the camera system 84 may also include a near infrared and/or a shortwave infrared imaging device. Such a device may allow leaf water content information to be determined. The camera system 84 proposed takes advantages of both crop growth/vigour and water stress information measured simultaneously in the close range from the target crop. By combining visible, near infrared, and thermal infrared images along with on-site ancillary information such as local meteorological data, crop type/growth, and portable calibration target, the device can produce plant-by-plant estimates of crop water stress, crop vigour, and water consumption with a minimal reliance on empirical crop parameters.
Embodiments of the invention have been described above by way of non-limiting example only. In practice, a plurality of weather stations 12, flow gates, flow meters and soil moisture sensors 40 are scattered around the irrigation district to provide an extensive irrigation system. Variations and modifications to the embodiments may be made without departing from the scope of the invention.

Claims

THE CLAIMS DEFINING THE INVENTION ARE AS FOLLOWS:
1. A method of plant stress determination using a computer-based camera system having thermal imaging and visual imaging to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof;
analyzing both thermal and visual images/video therefrom to form a composite image; determining the thermal activity of the composite image/video and photosynthesis state of said at least one plant; and deriving the plant stress from said determination.
2. The method as claimed in claim 1 , wherein said computer-based camera system is based on a smartphone.
3. The method as claimed in claim 2, wherein said thermal imaging is captured by a thermal camera associated with said smartphone.
4. The method as claimed in any one of the preceding claims, wherein said thermal and visual images capture at least one or more of information related to time of capture, GPS location, accelerometer data, direction data and inclinometer data.
5. The method as claimed in any one of the preceding claims, wherein said composite image uses edge detection of said foliage to align/co-register the visual and thermal images/video.
6. The method as claimed in claim 5, wherein the aligned/co-registered thermal images/video are processed using augmented reality techniques to provide image measurement.
7. The method as claimed in any one of the preceding claims, wherein the plant stress determination further includes inputs from relevant derived evapotranspiration and soil moisture data from land to be irrigated to provide an irrigation schedule for an operator linked to a networked computer system overseeing said plant stress determination, said relevant derived evapotranspiration and soil moisture data from said land to be irrigated.
8. The method as claimed in any one of the preceding claims, wherein the plant stress determination is calculated by said computer-based camera system.
9. The method as claimed in any one of the preceding claims, wherein said thermal imaging and said visual imaging are synchronized to be taken at the same time.
10. A plant stress determination system including a computer-based camera system having thermal imaging and visual imaging to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof;
analyzing both thermal and visual images/video therefrom to form a composite image; determining the thermal activity of the composite image/video and photosynthesis state of said at least one plant; and deriving the plant stress from said determination.
11. The plant stress determination system as claimed in claim 10, wherein said computer-based camera system is based on a smartphone.
12. The plant stress determination system as claimed in claim 11 , wherein said thermal imaging is captured by a thermal camera associated with said smartphone.
13. The plant stress determination system as claimed in any one of claims 10 to
12, wherein said thermal and visual images capture at least one or more of
information related to time of capture, GPS location, accelerometer data, direction data and inclinometer data.
14. The plant stress determination system as claimed in any one of claims 10 to
13, wherein said composite image uses edge detection of said foliage to align/co- register the visual and thermal images/video.
15. The plant stress determination system as claimed in claim 14, the aligned/co- registered thermal images/video are processed using augmented reality techniques to provide image measurement.
16. The plant stress determination system as claimed in any one of claims 10 to
15, wherein the plant stress determination further includes inputs from relevant derived evapotranspiration and soil moisture data from land to be irrigated to provide an irrigation schedule to an operator linked to a networked computer system overseeing said plant stress determination, said relevant derived evapotranspiration and soil moisture data from said land to be irrigated.
17. The plant stress determination system as claimed in any one of claims 10 to
16, wherein the plant stress determination is calculated by said computer-based camera system.
18. The plant stress determination system as claimed in any one of claims 10 to
17, wherein said thermal imaging and said visual imaging are synchronized to be taken at the same time.
19. An irrigation management system to irrigate predetermined areas of an irrigation district, said irrigation management system including:
(a) a soil moisture determination system to spatially derive the soil moisture at a selected location within an irrigation district to be irrigated, said system including a networked computer system connected to a plurality of weather stations within said irrigation district to measure a selection from: solar radiation spectrum, wind speed, rainfall, temperature, humidity, barometric pressure, and energy measurement from solar panels at each of a plurality of representative locations, said networked computer system having data access to crop factor, soil type, and irrigation historical data at said representative locations, said networked computer system using system identification techniques to produce an algorithm for evapotranspiration based on a predetermined selection from the weather station measurements and the data access to crop factor, soil type, and irrigation historical data at said representative locations; said networked computer system calibrating said algorithm by direct measurement of the moisture in the soil at each of said representative locations by respective soil moisture sensors; and using measured parameters of rainfall, soil type, irrigation historical data and crop factor with said algorithm to derive or interpolate soil moisture at said selected location within said irrigation district; (b) a plant stress determination system as claimed in any one of claims 10 to 18 associated with said soil moisture determination system to receive said derived soil moisture from said soil moisture system;
and said irrigation management system monitoring said plant stress from plant stress system; said derived or interpolated soil moisture for said predetermined areas; availability of irrigation water to said predetermined areas; and requests for timed irrigation from end users for said predetermined areas; and allowing requested irrigation to occur for said predetermined areas based on said monitoring.
20. An irrigation scheduling system including one or more features as herein described.
21. A method of scheduling irrigation of land including steps of determining plant stress using a computer-based camera system having thermal imaging and visual imaging to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof; analyzing both thermal and visual images/video therefrom to form a composite image; determining the thermal activity of the composite image/video and photosynthesis state of said at least one plant; deriving the plant stress from said determination; wherein the plant stress determination further includes inputs from relevant derived evapotranspiration and soil moisture data from land to be irrigated to provide an irrigation schedule to an operator linked to a networked computer system overseeing said plant stress determination, said relevant derived evapotranspiration and soil moisture data from said land to be irrigated.
22. A plant stress determination system in combination with soil moisture determination system including one or more features as herein described.
23. A plant stress determination method in combination with a method of spatially deriving soil moisture at a selected location including one or more features as herein described.
24. A method of gaining information on atmospheric conditions related to plant stress by using a thermal target that has: (a) a surface representing a non-transpiring leaf whose temperature is measured and
(b) a surface representing a transpiring leaf whose temperature is measured.
25. The method of any one of claims 1 to 9, wherein the plant stress determination further includes inputs from a surface representing a non-transpiring leaf whose temperature is measured and a surface representing a transpiring leaf whose temperature is measured.
26. A method of plant stress determination using a computer-based camera system having visible, near infrared, shortwave infrared, and thermal infrared imaging capability to capture foliage at close proximity of at least one plant to provide high resolution images/video thereof; analyzing selections of composites from said visible and infrared bands in images/video; determining the water stress, leaf water content, leaf pigment condition and photosynthetic activity from the composite image/video of said at least one plant; and deriving the plant stress from said determination.
27. The method as claimed in claim 26, wherein said computer-based camera system is based on a smartphone.
28. The method as claimed in claim 27, wherein said near and shortwave infrared imaging is captured by an infrared camera associated with said smartphone.
29. The method as claimed in any one of claims 26 to 28, wherein said thermal imaging is captured by a thermal camera associated with said smartphone.
30. The method as claimed in any one of the preceding claims, wherein said thermal, near/shortwave infrared and visual images capture at least one or more of information related to time of capture, GPS location, accelerometer data, direction data and inclinometer data.
31. The method as claimed in any one of claims 26 to 30, wherein said composite image uses edge detection of said foliage to align/co-register the visual and thermal images/video.
32. The method as claimed in claim 31 , wherein the aligned/co-registered thermal images/video are processed using augmented reality techniques to provide image measurement.
33. The method as claimed in any one of claims 26 to 32, wherein the plant stress determination further includes inputs from relevant derived evapotranspiration and soil moisture data from land to be irrigated to provide an irrigation schedule for an operator linked to a networked computer system overseeing said plant stress determination, said relevant derived evapotranspiration and soil moisture data from said land to be irrigated.
PCT/AU2019/050919 2018-09-05 2019-08-29 Method and system for plant stress determination and irrigation based thereon WO2020047579A1 (en)

Priority Applications (6)

Application Number Priority Date Filing Date Title
US17/273,931 US20210345567A1 (en) 2018-09-05 2019-08-29 Method and system for plant stress determination and irrigation based thereon
AU2019333920A AU2019333920A1 (en) 2018-09-05 2019-08-29 Method and system for plant stress determination and irrigation based thereon
EA202190644A EA202190644A1 (en) 2018-09-05 2019-08-29 METHOD AND SYSTEM FOR DETERMINING PLANT STRESS AND IRRIGATION BASED ON THIS BASIS
BR112021004212-1A BR112021004212A2 (en) 2018-09-05 2019-08-29 method for determining plant stress, system for determining plant stress, system for managing irrigation, system for scheduling irrigation, method for scheduling land irrigation, and method for obtaining information about atmospheric conditions related to plant stress
CA3115301A CA3115301A1 (en) 2018-09-05 2019-08-29 Method and system for plant stress determination and irrigation based thereon
EP19857549.0A EP3846611A4 (en) 2018-09-05 2019-08-29 Method and system for plant stress determination and irrigation based thereon

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
AU2018903312A AU2018903312A0 (en) 2018-09-05 Method and system for plant-oriented irrigation
AU2018903312 2018-09-05

Publications (1)

Publication Number Publication Date
WO2020047579A1 true WO2020047579A1 (en) 2020-03-12

Family

ID=69721466

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU2019/050919 WO2020047579A1 (en) 2018-09-05 2019-08-29 Method and system for plant stress determination and irrigation based thereon

Country Status (7)

Country Link
US (1) US20210345567A1 (en)
EP (1) EP3846611A4 (en)
AU (1) AU2019333920A1 (en)
BR (1) BR112021004212A2 (en)
CA (1) CA3115301A1 (en)
EA (1) EA202190644A1 (en)
WO (1) WO2020047579A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022040756A1 (en) * 2020-08-28 2022-03-03 Rubicon Research Pty Ltd Refill point or target deficit for crop irrigation
KR20220101935A (en) * 2021-01-12 2022-07-19 주식회사 에이치티솔루션 Smartphone cradle thermometers and haccp managing system including the smartphone cradle thermometers
CN116559383A (en) * 2023-07-07 2023-08-08 中国农业科学院农业环境与可持续发展研究所 Photosynthetic rate detection method based on interaction of biochar returning root system and soil moisture and application thereof

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11694434B2 (en) * 2020-10-16 2023-07-04 Verdant Robotics, Inc. Precision agricultural treatment based on growth stage in real time
US11425852B2 (en) 2020-10-16 2022-08-30 Verdant Robotics, Inc. Autonomous detection and control of vegetation
CN115104505A (en) * 2021-03-19 2022-09-27 上海兰桂骐技术发展股份有限公司 Decision-making method for field irrigation
US11399531B1 (en) 2021-10-20 2022-08-02 Verdant Robotics, Inc. Precision detection and control of vegetation with real time pose estimation
WO2023108213A1 (en) * 2021-12-14 2023-06-22 Rubicon Research Pty Ltd Methods and systems for classifying and benchmarking irrigation performance
CN114485955A (en) * 2021-12-19 2022-05-13 国网甘肃省电力公司天水供电公司 Distribution network infrared temperature measurement intelligent infrared imager based on equipment dynamic sensing technology
CN114663762A (en) * 2022-03-28 2022-06-24 杭州睿胜软件有限公司 Method and computer system for assisting user in plant maintenance
CN114698540B (en) * 2022-04-11 2023-12-05 广州大学 Irrigation method for inducing root system to grow downwards based on wet point duration deviation

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010187577A (en) * 2009-02-17 2010-09-02 Seiwa:Kk Method and apparatus for monitoring stomatal aperture
US20170372137A1 (en) * 2015-01-27 2017-12-28 The Trustees Of The University Of Pennsylvania Systems, devices, and methods for robotic remote sensing for precision agriculture
US20180129175A1 (en) * 2016-11-07 2018-05-10 FarmX Inc. Systems and Methods for Soil Modeling and Automatic Irrigation Control
CZ31855U1 (en) * 2017-12-20 2018-06-19 Výzkumný ústav rostinné výroby v.v.i. A device for determination of water stress by LWIR aerial photography using a thermal camera placed on a UAV device with automatic correction of thermal radiation of the atmosphere

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8028470B2 (en) * 2009-04-21 2011-10-04 Deere & Company Robotic watering unit
EP3032946B1 (en) * 2013-07-11 2022-10-05 Blue River Technology Inc. Method for automatic phenotype measurement and selection
JP5979567B1 (en) * 2015-04-30 2016-08-24 パナソニックIpマネジメント株式会社 Plant stress detection apparatus and plant stress detection method
WO2017106962A1 (en) * 2015-11-17 2017-06-29 Universite Laval System and method for real-time water management

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010187577A (en) * 2009-02-17 2010-09-02 Seiwa:Kk Method and apparatus for monitoring stomatal aperture
US20170372137A1 (en) * 2015-01-27 2017-12-28 The Trustees Of The University Of Pennsylvania Systems, devices, and methods for robotic remote sensing for precision agriculture
US20180129175A1 (en) * 2016-11-07 2018-05-10 FarmX Inc. Systems and Methods for Soil Modeling and Automatic Irrigation Control
CZ31855U1 (en) * 2017-12-20 2018-06-19 Výzkumný ústav rostinné výroby v.v.i. A device for determination of water stress by LWIR aerial photography using a thermal camera placed on a UAV device with automatic correction of thermal radiation of the atmosphere

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
A. MATESE ET AL.: "Estimation of Water Stress in Grapevines Using Proximal and Remote Sensing Methods", REMOTE SENSING, vol. 10, 2018, pages 11 4, XP055689868 *
I. GARCIA-TEJERO ET AL.: "Assessing the Crop-Water Status in Almond ( Prunus dulcis Mill. ) Trees via Thermal Imaging Camera Connected to Smartphone", SENSORS, vol. 18, 2018, pages 1050, XP055689870 *
I. LEINONEN ET AL.: "Combining thermal and visible imagery for estimating canopy temperature and identifying plant stress", JOURNAL OF EXPERIMENTAL BOTANY, vol. 55, no. 401, 2004, pages 1423 - 1431, XP055689872 *
M. GOVENDER ET AL.: "Review of commonly used remote sensing and ground-based technologies to measure plant water stress", WATER SA, vol. 35, 2009, pages 741 - 752, XP055689876 *
O. ADEYEMI ET AL.: "Advanced Monitoring and Management Systems for Improving Sustainability in Precision Irrigation", SUSTAINABILITY, vol. 9, 2017, pages 3 53, XP055654946 *
See also references of EP3846611A4 *
Y. OSROOSH ET AL.: "Comparison of irrigation automation algorithms for drip-irrigated apple trees", COMPUTERS AND ELECTRONICS IN AGRICULTURE, vol. 128, 2016, pages 87 - 99, XP029751505, DOI: 10.1016/j.compag.2016.08.013 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022040756A1 (en) * 2020-08-28 2022-03-03 Rubicon Research Pty Ltd Refill point or target deficit for crop irrigation
KR20220101935A (en) * 2021-01-12 2022-07-19 주식회사 에이치티솔루션 Smartphone cradle thermometers and haccp managing system including the smartphone cradle thermometers
KR102561598B1 (en) 2021-01-12 2023-08-01 주식회사 에이치티솔루션 Smartphone cradle thermometers and haccp managing system including the smartphone cradle thermometers
CN116559383A (en) * 2023-07-07 2023-08-08 中国农业科学院农业环境与可持续发展研究所 Photosynthetic rate detection method based on interaction of biochar returning root system and soil moisture and application thereof
CN116559383B (en) * 2023-07-07 2023-10-24 中国农业科学院农业环境与可持续发展研究所 Photosynthetic rate detection method based on interaction of biochar returning root system and soil moisture and application thereof

Also Published As

Publication number Publication date
EP3846611A1 (en) 2021-07-14
EA202190644A1 (en) 2021-07-08
US20210345567A1 (en) 2021-11-11
EP3846611A4 (en) 2022-05-25
CA3115301A1 (en) 2020-03-12
BR112021004212A2 (en) 2021-05-25
AU2019333920A1 (en) 2021-05-06

Similar Documents

Publication Publication Date Title
US20210345567A1 (en) Method and system for plant stress determination and irrigation based thereon
US11647701B2 (en) Plant treatment based on morphological and physiological measurements
Friedli et al. Terrestrial 3D laser scanning to track the increase in canopy height of both monocot and dicot crop species under field conditions
US11291172B2 (en) Methods and systems for irrigation guidance
Katsigiannis et al. An autonomous multi-sensor UAV system for reduced-input precision agriculture applications
Osroosh et al. Economical thermal-RGB imaging system for monitoring agricultural crops
EP2633460B1 (en) System and method for calibrating agricultural measurements
US10585210B2 (en) Apparatus for radiometric correction and orthorectification of aerial imagery
JP7229864B2 (en) REMOTE SENSING IMAGE ACQUISITION TIME DETERMINATION SYSTEM AND CROPT GROWTH ANALYSIS METHOD
US10426106B2 (en) Methods and systems for assessing a field of plants for irrigation
US11953872B2 (en) Methods and systems for the management of precise agricultural irrigation
WO2020230126A1 (en) Systems and methods for phenotyping
Hama et al. Examination of appropriate observation time and correction of vegetation index for drone-based crop monitoring
Katsigiannis et al. Fusion of spatio-temporal UAV and proximal sensing data for an agricultural decision support system
Simpson et al. UAS-based high resolution mapping of evapotranspiration in a Mediterranean tree-grass ecosystem
Ahmad et al. Spatio-temporal analysis of chickpea crop in arid environment by comparing high-resolution UAV image and LANDSAT imagery
JP2021171057A (en) Crop related value derivation device and crop related value derivation method
Sadiq et al. A review on the imaging approaches in agriculture with crop and soil sensing methodologies
US20220124960A1 (en) Commercial Farm Optimization Utilizing Simulation, Remote Sensing, and Farmer Inputs
Sairam Application of Remote Sensing in Agriculture
Bağış et al. Image based automated phenological stage detection of cereal plants
KR102561018B1 (en) Weather environment and crop growth analysis system for field crop cultivation
Mendes et al. A low-cost system to estimate leaf area index combining stereo images and normalized difference vegetation index
Alsadik et al. UAV Remote Sensing for Smart Agriculture
WO2021081451A1 (en) Commercial farm optimization utilizing simulation, remote sensing, and farmer inputs

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19857549

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
NENP Non-entry into the national phase

Ref country code: DE

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112021004212

Country of ref document: BR

ENP Entry into the national phase

Ref document number: 3115301

Country of ref document: CA

ENP Entry into the national phase

Ref document number: 2019857549

Country of ref document: EP

Effective date: 20210406

ENP Entry into the national phase

Ref document number: 2019333920

Country of ref document: AU

Date of ref document: 20190829

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 112021004212

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20210305