AU2021100538A4 - Crop Health Monitoring System Using IoT and Machine Learning - Google Patents
Crop Health Monitoring System Using IoT and Machine Learning Download PDFInfo
- Publication number
- AU2021100538A4 AU2021100538A4 AU2021100538A AU2021100538A AU2021100538A4 AU 2021100538 A4 AU2021100538 A4 AU 2021100538A4 AU 2021100538 A AU2021100538 A AU 2021100538A AU 2021100538 A AU2021100538 A AU 2021100538A AU 2021100538 A4 AU2021100538 A4 AU 2021100538A4
- Authority
- AU
- Australia
- Prior art keywords
- crop
- data
- iot
- node
- health
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
- 230000036541 health Effects 0.000 title claims abstract description 59
- 238000012544 monitoring process Methods 0.000 title claims abstract description 29
- 238000010801 machine learning Methods 0.000 title claims abstract description 15
- 238000005516 engineering process Methods 0.000 claims abstract description 22
- 238000013459 approach Methods 0.000 claims abstract description 8
- 238000007781 pre-processing Methods 0.000 claims abstract description 5
- 238000012800 visualization Methods 0.000 claims abstract description 5
- 238000012517 data analytics Methods 0.000 claims abstract description 3
- 230000002708 enhancing effect Effects 0.000 claims description 4
- 238000002360 preparation method Methods 0.000 claims 1
- 230000003595 spectral effect Effects 0.000 claims 1
- 230000010354 integration Effects 0.000 abstract description 17
- 230000002123 temporal effect Effects 0.000 abstract description 14
- 238000010348 incorporation Methods 0.000 abstract description 3
- 230000001419 dependent effect Effects 0.000 abstract description 2
- 239000002689 soil Substances 0.000 description 40
- 241000209140 Triticum Species 0.000 description 14
- 238000000034 method Methods 0.000 description 14
- 230000012010 growth Effects 0.000 description 11
- 230000003862 health status Effects 0.000 description 10
- 235000021307 Triticum Nutrition 0.000 description 8
- 238000013480 data collection Methods 0.000 description 8
- 238000009313 farming Methods 0.000 description 8
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 8
- 229930002875 chlorophyll Natural products 0.000 description 7
- 235000019804 chlorophyll Nutrition 0.000 description 7
- ATNHDLDRLWWWCB-AENOIHSZSA-M chlorophyll a Chemical compound C1([C@@H](C(=O)OC)C(=O)C2=C3C)=C2N2C3=CC(C(CC)=C3C)=[N+]4C3=CC3=C(C=C)C(C)=C5N3[Mg-2]42[N+]2=C1[C@@H](CCC(=O)OC\C=C(/C)CCC[C@H](C)CCC[C@H](C)CCCC(C)C)[C@H](C)C2=C5 ATNHDLDRLWWWCB-AENOIHSZSA-M 0.000 description 7
- 238000011161 development Methods 0.000 description 7
- 230000018109 developmental process Effects 0.000 description 7
- 239000003337 fertilizer Substances 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 6
- 235000015097 nutrients Nutrition 0.000 description 6
- 238000012706 support-vector machine Methods 0.000 description 6
- QVFWZNCVPCJQOP-UHFFFAOYSA-N chloralodol Chemical compound CC(O)(C)CC(C)OC(O)C(Cl)(Cl)Cl QVFWZNCVPCJQOP-UHFFFAOYSA-N 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 230000007613 environmental effect Effects 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000011160 research Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 238000009826 distribution Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000012272 crop production Methods 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000005070 ripening Effects 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000010200 validation analysis Methods 0.000 description 3
- 241000132092 Aster Species 0.000 description 2
- 229920000742 Cotton Polymers 0.000 description 2
- 241000196324 Embryophyta Species 0.000 description 2
- 241000219146 Gossypium Species 0.000 description 2
- 241000238631 Hexapoda Species 0.000 description 2
- 240000007594 Oryza sativa Species 0.000 description 2
- 235000007164 Oryza sativa Nutrition 0.000 description 2
- 240000000111 Saccharum officinarum Species 0.000 description 2
- 235000007201 Saccharum officinarum Nutrition 0.000 description 2
- 240000008042 Zea mays Species 0.000 description 2
- 235000016383 Zea mays subsp huehuetenangensis Nutrition 0.000 description 2
- 235000002017 Zea mays subsp mays Nutrition 0.000 description 2
- 230000002411 adverse Effects 0.000 description 2
- 230000003698 anagen phase Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013145 classification model Methods 0.000 description 2
- 230000002860 competitive effect Effects 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 235000013305 food Nutrition 0.000 description 2
- 238000003973 irrigation Methods 0.000 description 2
- 230000002262 irrigation Effects 0.000 description 2
- 238000009318 large scale farming Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 235000009973 maize Nutrition 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000010899 nucleation Methods 0.000 description 2
- 238000012907 on board imaging Methods 0.000 description 2
- 239000011368 organic material Substances 0.000 description 2
- 239000000575 pesticide Substances 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 235000009566 rice Nutrition 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 241000760358 Enodes Species 0.000 description 1
- 241001274961 Rubus repens Species 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 235000013339 cereals Nutrition 0.000 description 1
- 238000000701 chemical imaging Methods 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000446 fuel Substances 0.000 description 1
- JEIPFZHSYJVQDO-UHFFFAOYSA-N iron(III) oxide Inorganic materials O=[Fe]O[Fe]=O JEIPFZHSYJVQDO-UHFFFAOYSA-N 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000000087 stabilizing effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
- 230000003442 weekly effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/188—Vegetation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/02—Agriculture; Fishing; Forestry; Mining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
- G16Y10/00—Economic sectors
- G16Y10/05—Agriculture
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2101/00—UAVs specially adapted for particular uses or applications
- B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01W—METEOROLOGY
- G01W1/00—Meteorology
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01W—METEOROLOGY
- G01W1/00—Meteorology
- G01W2001/006—Main server receiving weather information from several sub-stations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10036—Multispectral image; Hyperspectral image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30128—Food products
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30188—Vegetation; Agriculture
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Business, Economics & Management (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Economics (AREA)
- General Business, Economics & Management (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Agronomy & Crop Science (AREA)
- Data Mining & Analysis (AREA)
- Strategic Management (AREA)
- Medical Informatics (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Development Economics (AREA)
- Tourism & Hospitality (AREA)
- Mining & Mineral Resources (AREA)
- Marine Sciences & Fisheries (AREA)
- Animal Husbandry (AREA)
- Human Resources & Organizations (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Accounting & Taxation (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
Crop Health Monitoring System Using loT and Machine Learning
Due to the inherent agrarian aspect of the economy, the agricultural sector holds
paramount importance in many countries. Some countries have their GDP dependent on
agriculture, but they rely on manual crop monitoring, which is a system that is labor intensive
and ineffective. In comparison to this, in developing countries, many cutting-edge technology
based technologies are being used to increase crop yield with optimum resource utilization. To
this end, this invention suggested an integrated approach using IoT, machine learning and drone
technology for monitoring crop health. The incorporation of these sensing modalities produces
heterogeneous data which is not only differing in absorbed parameter also in the temporal
fidelity. The spatial resolution of these approaches is also different, so the proposed scheme
suggests the optimum integration of these sensing modalities and their implementation in
practice. The proposed work is essentially an indigenous, technology-based agricultural solution
capable of providing important insights into crop health by extracting additional features from
the multi-modal data set and minimizing the effort to survey crops, particularly useful when the
agricultural land is large.
1
Data Analytics
Drone &
Local Data . Visualization
(NIR map, NDVI map, Crop Server Preprocessing
health map)
End User
Crop Area
Slave Slave
Node Node
SSlave
Node
-MMaterr
SlavedeN
-Ne- ----. Slave
eNode
Master -
Node
Slave Slave
Node Node
Fig. 1
1
Description
Data Analytics Drone
& Local Data . Visualization
(NIR map, NDVI map, Crop Server Preprocessing health map)
End User
Crop Area
Slave Slave Node Node
SSlave Node
SlavedeN -Ne--MMaterr ----. Slave eNode Master -
Node Slave Slave Node Node
Fig. 1
TITLE OF THE INVENTION Crop Health Monitoring System Using IoT and Machine Learning
[001]. The present disclosure is generally related to a Crop Health Monitoring System Using IoT and Machine Learning that is helpful for monitoring the crop thereby enhancing the crop yield.
[002]. Due to the inherent agrarian aspect of the economy, the agricultural sector holds paramount importance in many countries. Some countries have their GDP dependent on agriculture, but they rely on manual crop monitoring, which is a system that is labor intensive and ineffective. In comparison to this, in developing countries, many cutting edge technology-based technologies are being used to increase crop yield with optimum resource utilization. The main objective of this invention is to propose and suggest an integrated approach using IoT, machine learning and drone technology for monitoring crop health thereby helping in enhancing the crop yield.
[003]. Due to its natural resources, many agricultural countries, including fertile arable land, good climate conditions and the world's largest irrigation system. The GDP of several countries has a huge influence on the country's economy. Notwithstanding all the necessary conditions for the cultivation of crops, few countries are still unable to generate surplus yields to meet national and foreign market needs. Each year, due to several factors such as extreme climatic variations, lack of technology adoption, improper use of major resources such as water, fertilizer, and pesticides, few countries face a huge loss in the agricultural sector.
[004]. The improper use of these tools results in a loss of organic material and nutrients in the crop and a substantial reduction in the yield of the crop. Technology-based methods can be used to resolve these issues in order to solve manual farming activities that are fundamentally time-consuming and laborious. In some countries the wheat, rice, maize, sugarcane and cotton are planted as main crops. Among these, wheat is the widely sown crop. There are multiple growth phases in the wheat plant, such as seeding, tillering, booting, heading and ripening.
[005]. The wheat plant has unique water, temperature, solar radiation and nutrient/fertilizer requirements for optimal growth at each point. These requirements are specifically associated with climate change, such as the frequency of rainfall and changes in temperature. The primary resources should be used in a controlled manner in a site specific way for the optimal production of the wheat plant, as the deficiency of any resource can adversely affect crop growth, whereas the excessive use of these resources can harm the crop. Precision Agriculture (PA) is commonly practiced worldwide to accurately predict resource requirements and basically improves food production with the optimum use of resources.
[006]. As of today, IoT-based smart farming systems are rapidly gaining popularity as they provide real-time status of crop-related environmental variables using low-cost sensors. Not only do these systems advance PA practices, but they also play a key role in making the crop monitoring system more competitive and successful. On the other hand, because of their sensitivity to the high cost of maintenance & deployment and power constraints, IoT-based systems are typically ideal for small to medium-scale farming. In comparison to IoT, remote sensing, which is based on a reflective study of satellite images, is commonly used for large scale farming.
[007]. Satellite images have been used conventionally as a primary source of knowledge for the study of crop status in precision agriculture. But it is very costly to acquire the most recent aerial/satellite imagery, and data processing is often expensive and complicated. In addition, low-resolution images collected from satellites are only suitable for large-scale studies. In studies focused on precision agriculture, this limits their applicability.
[008]. On the other hand, satellites (such as Fast Bird, ASTER) that provide higher resolution images have long revisit times, making them unsuitable for applications that need regular images (such as insect monitoring, nutrient stress monitoring, etc.). Low altitude systems, such as drones with on-board imaging sensors, are used to address these restrictions, offering high-resolution images and versatile data collection. The multispectral data obtained using the drone is usually used to measure the crop's VIs to assess its health status.
[009]. Generally, as a good predictor of crop health, NDVI is used. However, if the crop health conditions are only calculated from the NDVI values, due to the varying degree of chlorophyll content at different stages of the crop, the inferred crop health status will be misleading. Each stage of crop development has a predefined set of NDVI values, which can help to determine the health of the crop. The low NDVI value does not always apply to the stressed or unhealthy crop. In order to assess the crop health status, we need to incorporate the temporal details of the crop production stage.
[0010]. In view of the above, we have proposed a multi modal data-driven approach based on IoT, drone-based remote sensing and machine learning for agricultural monitoring.
[0011]. The proposed work is focused on testing the hypothesis that multi-modal data integration will boost crop health information representation compared to the crop health status reflected only by NDVI. The data collected from IoT nodes and drones were analyzed at different growing stages of the crop for this reason and their health maps were created to localize the area under stress. The invention's main contributions are highlighted below:
[0012]. IoT and drone multispectral data integration for monitoring of crop health. Heterogeneous data are produced by both these sensing modalities that not only vary in nature (i.e. observed parameter) but also have different temporal fidelity. The spatial resolution of these methods is also different, so this invention discusses the optimum integration of these sensing modalities and their implementation in practice.
[0013]. Creation of crop health maps for better visualization by field survey of the stressed areas and their subsequent validation. Creation of data maps for IoT sensors that provide insights to identify the variables affecting crop health. This multi-modal data integration for crop health mapping distinguishes the work proposed from the current work. Most of the current agricultural technology-related solutions are either based on IoT data or data from remote sensing, but the proposed system takes advantage of the advantages of both technologies to provide a better solution.
[0014]. A framework focused on the integration of the latest technologies such as drone based remote sensing, IoT and machine learning has been proposed for crop health monitoring. The incorporation of these sensing modalities produces heterogeneous data with different temporal fidelity, not only differing in nature (i.e. observed parameters). The spatial resolution of these approaches is also different, so the proposed scheme discusses the optimum integration of these sensing modalities and their implementation in practice. Multi-modal data was obtained from various sources, including IoT sensors and a drone mounted on it with a multispectral camera.
[0015]. At variable intervals and of variable length, this multi source data was developed. This knowledge was then mapped for integration to a standard temporal resolution and labeled to conduct supervised classification. Together with several deep learning models, machine learning techniques such as SVM and NB were applied to identify each pixel as safe, unhealthy or stressed. Among these chosen models, NN's M4 model was found to be the most appropriate model for our multi-modal data set and given 98.4% classification accuracy.
[0016]. Conventionally, NDVI is used for crop health monitoring, but it only provides health information based on the value of chlorophyll, while data from various sources such as soil moisture, soil temperature, humidity and air temperature are needed for a comprehensive representation of crop health. Data from two different modalities were combined for this purpose and crop health maps were created to provide a clear picture of crop health relative to the information given by NDVI maps.
[0017]. Because of its natural resources, many agricultural countries, including fertile arable land, good climate conditions and the world's largest irrigation system. The GDP of several countries has a huge influence on the country's economy. Notwithstanding all the necessary conditions for the cultivation of crops, few countries are still unable to generate surplus yields to meet national and foreign market needs. Each year, due to several factors such as extreme climatic variations, lack of technology adoption, improper use of major resources such as water, fertilizer, and pesticides, few countries face a huge loss in the agricultural sector.
[0018]. The improper use of these tools results in a loss of organic material and nutrients in the crop and a substantial reduction in the yield of the crop. Technology-based methods can be used to resolve these issues in order to solve manual farming activities that are fundamentally time-consuming and laborious. In some countries wheat, rice, maize, sugarcane and cotton are planted as main crops. Among these, wheat is the primary crop, widely sown. Tere are multiple growth phases in the wheat plant, such as seeding, tillering, booting, heading and ripening.
[0019]. The wheat plant has unique water, temperature, solar radiation and nutrient/fertilizer requirements for optimal growth at each point. These requirements are specifically associated with climate change, such as the frequency of rainfall and changes in temperature. The primary resources should be used in a controlled manner in a site specific way for the optimal production of the wheat plant, as the deficiency of any resource can adversely affect crop growth, whereas the excessive use of these resources can harm the crop. Precision Agriculture (PA) is commonly practiced worldwide to accurately predict resource requirements and basically improves food production with the optimum use of resources.
[0020]. As of today, IoT-based smart farming systems are rapidly gaining popularity as they provide real-time status of crop-related environmental variables using low-cost sensors. Not only do these systems advance PA practices, but they also play a key role in making the crop monitoring system more competitive and successful.
[0021]. On the other hand, because of their sensitivity to the high cost of maintenance &
deployment and power constraints, IoT-based systems are typically ideal for small to medium-scale farming. In comparison to IoT, remote sensing, which is based on a reflective study of satellite images, is commonly used for large scale farming. Satellite images have been used conventionally as a primary source of knowledge for the study of crop status in precision agriculture. But it is very costly to acquire the most recent aerial/satellite imagery, and data processing is often expensive and complicated. In addition, low-resolution images collected from satellites are only suitable for large-scale studies.
[0022]. In studies focused on precision agriculture, this limits their applicability. On the other hand, satellites (such as Fast Bird, ASTER) that provide higher resolution images have long revisit times, making them unsuitable for applications that need regular images (such as insect monitoring, nutrient stress monitoring, etc.).
[0023]. Low-altitude systems, such as drones with on-board imaging sensors, are used to address these restrictions, offering high-resolution images and versatile data collection. The multispectral data obtained using the drone is usually used to measure the crop's VIs to assess its health status. Generally, as a good predictor of crop health, NDVI is used. However, if the crop health conditions are only calculated from the NDVI values, due to the varying degree of chlorophyll content at different stages of the crop, the inferred crop health status will be misleading.
[0024]. Each stage of crop development has a predefined set of NDVI values, which can help to determine the health of the crop. The low NDVI value does not always apply to the stressed or unhealthy crop. In order to assess the crop health status, we need to incorporate the temporal details of the crop production stage. In view of the above, we have proposed a multi modal data-driven approach based on IoT, drone-based remote sensing and machine learning for agricultural monitoring.
[0025]. The proposed work is focused on testing the hypothesis that multi-modal data integration will boost crop health information representation compared to the crop health status reflected only by NDVI. The data collected from IoT nodes and drones were analyzed at different growing stages of the crop for this reason and their health maps were created to localize the area under stress. The invention's main contributions are highlighted below:
[0026]. IoT and drone multispectral data integration for monitoring of crop health. Heterogeneous data are produced by both these sensing modalities that not only vary in nature (i.e. observed parameter) but also have different temporal fidelity. The spatial resolution of these methods is also different, so this invention discusses the optimum integration of these sensing modalities and their implementation in practice.
[0027]. Creation of crop health maps for better visualization by field survey of the stressed areas and their subsequent validation. Creation of data maps for IoT sensors that provide insights to identify the variables affecting crop health. This multi-modal data integration for crop health mapping distinguishes the work proposed from the current work. Most of the current agricultural technology-related solutions are either based on IoT data or data from remote sensing, but the proposed system takes advantage of the advantages of both technologies to provide a better solution.
[0028]. Proposed work: In this invention we proposed an integrated health monitoring system for wheat crops. The key building blocks of the framework are IoT agri-nodes, data transmission communication networks, multispectral camera drones, local data archiving servers, and data visualization web portals. In Figure 1, the high level system architecture is shown. The specifics of the building blocks listed above are described below.
[0029]. Development of IoT Agri Nodes: A typical IoT node is mainly built by carefully selecting sensors, a communication module and a power source. The descriptions of these components are listed below for the purpose of the present research work.
[0030]. IoT Sensors: Three instruments, including the air temperature and humidity sensor, the soil temperature sensor and the soil moisture sensor, were used for the proposed device. The 'DHT11' digital sensor was used to record air temperature and humidity, the 'DS18B20' digital sensor was selected to detect soil temperature, and the 'Capacitive Soil Moisture' sensor was used to track the level of soil moisture.
[0031]. There are two main soil humidity sensor types, namely volumetric and tensiometric. The volumetric sensors measure the amount of water in the soil, while the tensiometric sensors measure the potential for water in the soil. Grove - Capacitive Soil Moisture Sensor (Corrosion Resistant), which is volumetric in nature, are the selected capacitive soil moisture sensors used for this research work. They are not factory calibrated, so it is a challenge to optimally calibrate these sensors.
[0032]. This problem is solved by manually configuring the sensors using multi-point calibration techniques in the laboratory and in the field. On the other hand, factory calibrated volumetric soil moisture sensors are more reliable but also costly at the same time, and the implementation of such sensors in numbers goes beyond the pilot studies' budgetary provisions.
[0033]. An analog voltage level outputs the capacitive soil moisture sensor, which is inversely proportional to the moisture content. This sensor is sensitive to variations in soil temperature, salinity and pH content. The measurements of the calibrated IoT sensors were reasonably reliable and this was confirmed by comparing the readings with the NARC supplied commercial sensors.
[0034]. Communication Module: The purpose of the communication module is to transfer data efficiently from one end to the other, and it plays a key role in characterizing the system's performance. Due to its long range of transmission, the 'LoRA' (Long Range) communication module was considered more fitting for our research work. In addition, there is a clear line of sight in an agriculture land that makes LoRa technology more successful for such a set up. Two forms of frequencies, i.e. 433MHz and 868MHz, work with the LoRa modules.
[0035]. The 433MHz frequency module was used for our application because this frequency band is license-free, but in certain countries, the 868mhz ISM band is not free. The LoRa modules use their patented modulation techniques in which data is sent using Chirps and can be interfaced via the serial-parallel interface (SPI) protocol with the microcontroller as well. The IoT agri nodes were designed for data transmission using the star topology, which has 8 x slave nodes and 1 x master node.
[0036]. The slave nodes used LoRa technology to send data to the master node, while GSM technology was used to send data from the master node to the web portal. The benefit of star topology is its cost-effectiveness, but at the same time, it also has a single failure point. This is because it is the duty of a single master node to send data which is obtained from all nodes to the local server. To a certain degree, in order to address this constraint, the SD card was placed on the master node to store the data for 24 hours in case of any contact failure at the end of the master node.
[0037]. Power Module: In IoT-based systems, the primary concern for keeping the device operational for real-time monitoring is the continuous supply of electricity. Solar energy is the richest source of energy for powering the proposed farming method. For this reason, a 1OW solar panel and a 4Ah battery power the slave nodes to provide long battery backups in case of rainfall and gloomy weather. To give it adequate fuel, the master node is fitted with a 40W solar panel with a 5Ah battery.
[0038]. System Deployment: The IoT agri nodes are deployed in a star topology across the field of wheat. The field is divided into a matrix of 3 x 3 consisting of 9 cells, where 0.15 acres is the area occupied by each cell. In order to collect IoT sensor data, we deployed one IoT node in each cell. Along with other components, each IoT node consists of sensors. Due to the budget limitations of this research work, these IoT nodes can not be deployed in bulk. So, we selected a small experimental plot and accordingly established the topology of deployment.
[0039]. We have reported multiple readings in each cell by placing IoT nodes at different locations. On average, the temperature and soil moisture readings did not differ significantly. Although the soil moisture information for the specified crop field can be increased by deploying more IoT nodes in each cell, we have deployed 9 nodes to cover the subject area for the preliminary sample. We will be able to incorporate more sensors in the future and embrace robot data collection technology. The farming robot is going to be like a rotating trolley that will travel around the field and collect data from sensors.
[0040]. Methodology: The multi-modal data obtained from IoT nodes and drone imagery was combined to obtain a comprehensive picture of the health of the crop in order to improve our proposed method. Data from two different modalities was created at a variable temporal resolution, which was further mapped for data processing and analysis to a fixed size representation. DJI Phantom 4 advance drone with Sentera multispectral imager mounted on it compiled the optical and multispectral images.
[0041]. Different machine learning and deep learning algorithms were subsequently applied to the fused data for classification of crop health and generation of health maps. Later, for validation purposes, the health maps were compared with the NDVI maps and data maps of the IoT sensors. The block diagram showing the different modules of the proposed system and the data flow sequence between them is shown in Figure 2, while the specifics of each processing stage are listed below.
[0042]. Data Pre-Processing: Data was collected from two heterogeneous sources, including IoT agri-nodes and drones, where IoT node data was sent to the local server over a 5-minute period and drone imagery was collected on a weekly basis. The IoT nodes were powered by solar panels, but the batteries were not adequately charged in the event of cloudy or rainy weather conditions. As a result, the data was not transmitted to the web server, resulting in missing values, which were then interpolated by bi-linear interpolation.
[0043]. The data transmission was restored when the batteries reinstated their charge. The sensors were reconfigured and spent some time stabilizing within the first few minutes of the resumption of the power supply. During this interval, since the values did not correspond to the true values of the environmental parameters, the transmitted data was regarded as an outlier and the outliers were thus excluded from the data collection. The box plot was used to pictorially represent the statistical overview of all IoT agri-node data in terms of monthly average, variance, min-max values and outliers.
[0044]. While the line in the box shows the mean of the specific variable, the two whiskers we use to display the minimum and maximum values of a particular variable, and the black circles show the outliers that were later omitted from the data collection. For humidity, which was mainly due to the weather variability in the year 2019-20, a distinct difference over the span of three months was observed. Similarly, it has been observed in the soil moisture box plot that its values have steadily increased from 10% to 90%. This observed soil moisture behavior is due to the fact that the soil moisture was lower during the initial crop growing stage, but the soil moisture increased incrementally as the crop grew in size.
[0045]. From the air temperature box map, it was observed that the temperature profile behavior is negatively associated with the humidity data. The rise in air temperature has also been due to changes in weather conditions. During the months of December and January, in the early morning, the crop was usually covered by frost and dew drops. Because of these variables, the temperature of the soil fell quickly, ranging from 5oC 15oC, which gradually increased soil moisture.
[0046]. The multispectral data was also pre-processed, in addition to the IoT node data. The drone was flown at the recommended height of 120 feet with speeds of 6 miles per hour to capture the multispectral images. The multispectral imaging provider, i.e. Sentera, issued those recommendations. The ground sampling distance (GSD) with these configurations was 1.2 inches per pixel. GSD can be computed as GSD = (a X h)/ f Where 'h' is the platform altitude;' f indicates the image sensor's focal length and aa refers to the imager's size of the charged-coupled device (CCD) cell. In the reported imagery, there was no cloud cover as the drone normally flies below the height of the cloud. On average, in a single flight mission, 25 to 30 images were captured with a 70 percent overlap in the image material. In order to achieve the full scene representation, this overlap was then used to stitch the images seamlessly.
[0047]. NDVI Maps: We have 9 x IoT agri nodes deployed in the field described earlier; therefore, the stitched image acquired during the pre-processing phase was mapped on these 9 cells. To find the chlorophyll content of the crop, which is an indicator of crop health status, the NDVI values of the stitched images were computed. Using Eq 2, the NDVI of the stitched images was computed as NDVI = NIR - R / NIR + R where the Close Infra Red band is 'NIR,' and the Red band is 'R,' For the entire wheat growth cycle, the NDVI profile usually represents a parabola. The NDVI value is low compared to the matured stage of stem elongation process during the early stages of crop growth, and a decrease in NDVI values is again observed during the grain ripening phase when the crop turns golden and loses the quality of chlorophyll.
[0048]. For the three-month period beginning from November to January, spatial NDVI maps were created. Due to an unstable drone flight, the NDVI map for the month of November did not cover the entire region. NDVI charts, however, include the full region of the study area for the other months.
[0049]. The crop is in the stage of stem elongation, which begins after the process of 5 leaves and lasts until the end of February. The crop height is not uniform across the field at this point, i.e. low in some regions and high in other regions. In the NDVI picture, the green color represents the region where the crop is more dense and taller. The variance in crop height and thickness over our study area is primarily due to several factors, such as the sunlight reaching that plant, the distribution of fertilizer in that area, and the slope of the terrain to keep the water in that field.
[0050]. NDVI maps have the benefit of offering an overall image of crop health and helping to classify the crop under stress. They have, however, limited information about the stressed environment. The causes of stress, such as rust disease, lack of soil moisture, severe weather conditions, evapotranspiration, inadequate fertilizer, etc, are not established in these maps. A ground survey to verify the results of NDVI maps was then carried out and a visual inspection showed that the stressed areas indicated by NDVI maps were present in the field of crops.
[0051]. NDVI is not the only criterion for assessing crop health since the presence of chlorophyll content in the plants is only suggested. However, additional information and knowledge regarding the crop growth is required to determine its health status. This includes information about meteorological parameters such as air temperature, humidity, soil parameters such as soil temperature and soil moisture information, and knowledge about the crop development stage. For this purpose, the maps of the data obtained from
IoT sensors were also plotted to relate them to the NDVI maps and to draw some useful inferences as discussed below.
[0052]. IoT Sensors Data Maps: The IoT sensor data was used to highlight the environmental events affecting crop health in order to analyze the variations in crop health illustrated by crop NDVI maps. The atmospheric and soil parameter maps were generated for this purpose. Cellular variation in atmospheric parameters (air temperature and humidity) and variation in soil parameters is shown in these charts (soil moisture and soil temperature). The NDVI map variation corresponded directly to the re-elected changes in the IoT results. For example, the area covered by the cell-8 was due to high humidity and soil moisture in the area under stress.
[0053]. As a result, wheat growth was carried out and, due to excessive moisture and low temperature, the leaves of the crops turned yellow. At this stage, the ideal temperature requirement is 22°C (as stated by NARC agriculture experts), but the recorded average temperature was 16C, thus suppressing crop growth across the study area. Several other factors, such as terrain slope, fertilizer distribution across the field, soil type, and seed quality, affected crop growth.
[0054]. The study area's terrain surface was not consistent & smooth and varied through each cell. In the soil moisture map, this is visible (cell wise). The soil was rough and the dip in the ground allowed the runoff water to collect for a longer period of time. At this point, this increased the soil moisture more than the crop needed, which caused stress on the crop. From the above discussion, it is clear that the data from IoT sensors provided added information to better understand the NDVI maps and helped to assimilate the reasons for the stressed crop. Subsequently, this multi-source data (IoT sensors, NDVI, and stage of crop development) was integrated in order to generate more detailed crop health maps.
[0055]. Multi-Modal Data Integration: The data is generally generated from heterogeneous sources at variable intervals and of variable duration in multi-modal data acquisition. For integration and further processing, this multi source data is mapped to a standard temporal resolution. In our focused study, because of the unusual environmental changes, the IoT data was logged at an interval of 5 minutes. Likewise, the drone imagery was documented after each week in order to discern the gradual growth of the crop.
[0056]. The IoT node data was averaged over 7 days and mapped to the temporal resolution of multispectral imager data in order to integrate the data from these various modalities. The next move was to map the two separate data sets to the same spatial resolution, after equalizing the temporal resolution. The NDVI values obtained from each cell of the 3 x 3 matrix layout at a given crop creation stage were flattened into a vector, where the NDVI value of each pixel was represented by the vector index.
[0057]. Furthermore, this was mapped to the corresponding IoT agri-node data, where each IoT data record consisted of air temperature, humidity, soil temperature, soil moisture and crop development stage temporal information. For each cell of the field's 3 x 3 matrix structure, the same process was repeated. A matrix [X]ij comprised the final data collection, where I represented the number of records and j denoted the number of features, including data from IoT sensors, NDVI values, and information related to the stage of crop production. It was labeled record wise to conduct crop health classification once the information was incorporated and prepared for the entire crop area.
[0058]. Crop Health Classification: the data was classified into three groups for crop health classification, i.e. 'Unhealthy', 'Stressed' and 'Healthy'. Since we were dealing with a multi-class problem, the labeled vector was translated into one-hot coding. A separate label matrix [Y]i,k has therefore been developed, where I = no records and k = no classes. I = 570408 and j = 6 were included in the final classification data set [X]ij, where I was the total number of records and j indicated the total characteristics, including NDVI values, crop development stage and IoT sensor data.
[0059]. The data set was subsequently divided, where 2/3 of the data set was used for training and 1/3 of the data set for testing purposes. As a result, 382173 records were included in the training data set and 188235 records were contained in the evaluation data set. In the training data collection, the distribution of the records for three grades, i.e. 'Unhealthy',' Stressed' and 'Healthy' was respectively 36,904, 137,887, 207,282. Likewise, the distribution of 'Unhealthy',' Stressed' and 'Healthy' groups was 18379, 68336 and 101520 records, respectively, for the test data collection.
[0060]. Several classification models were evaluated for crop health classification, including Naive Bayes (NB), Support Vector Machine (SVM) and Neural Network (NN), which were found to be sufficient for the nature of the data collected. The NB is a supervised classification algorithm that uses the Bayes theorem to classify different objects based on probabilities. SVM, on the other hand, is a classifier that uses the kernel feature to create highly discriminating classes with large margins. SVM was trained using 'Radial Basis' as a kernel function for this study work.
[0061]. The NN was trained using various shallow and deep learning models with different configurations of hyper-parameters, including I hidden layer range, (ii) hidden layer nodes, (iii) activation function, and (iv) loss function. Cross entropy is used as a loss function in NN for this research work, since it minimizes the difference in classification problems between expected and actual values. This cross entropy is computed as Cross Entropy = - yi (log (y;) where N is the total number of records, yi is the actual mark of ith record (ground truth), and Y; is the expected value of a classifier computed ith record. On our multi modal data set, the classification models listed above were applied and the results obtained are discussed below.
[0062]. A framework focused on the integration of the latest technologies such as drone based remote sensing, IoT and machine learning has been proposed for crop health monitoring. The incorporation of these sensing modalities produces heterogeneous data with different temporal fidelity, not only differing in nature (i.e. observed parameters). The spatial resolution of these approaches is also different, so the proposed scheme discusses the optimum integration of these sensing modalities and their implementation in practice. Multi-modal data was obtained from various sources, including IoT sensors and a drone mounted on it with a multispectral camera.
[0063]. At variable intervals and of variable length, this multi source data was developed. This knowledge was then mapped for integration to a standard temporal resolution and labeled to conduct supervised classification. Together with several deep learning models, machine learning techniques such as SVM and NB were applied to identify each pixel as safe, unhealthy or stressed. Among these chosen models, NN's M4 model was found to be the most appropriate model for our multi-modal data set and given 98.4% classification accuracy.
[0064]. Conventionally, NDVI is used for crop health monitoring, but it only provides health information based on the value of chlorophyll, while data from various sources such as soil moisture, soil temperature, humidity and air temperature are needed for a comprehensive representation of crop health. Data from two different modalities were combined for this purpose and crop health maps were created to provide a clear picture of crop health relative to the information given by NDVI maps.
[0065].
Claims (5)
- CLAIMS: We Claim: 1. We claim that the present disclosure is generally related to a Crop Health Monitoring System Using IoT and Machine Learning that is helpful for monitoring the crop thereby enhancing the crop yield.
- 2. As we claimed in 1, this invention suggested an integrated approach using IoT, machine learning and drone technology for monitoring crop health.
- 3. We claim that this invention develops the IoT sensors data maps which provided insights for identifying the factors affecting the crop health.
- 4. We claim that this invention uses both the IoT data and remote sensing data together for monitoring the crops which offers better solution.
- 5. We claim that this invention will help in enhancing the crop yield.Data Analytics Drone & Visualization Local Data (NIR map, NDVI map, Crop Preprocessing Server health map)End UserCrop Area 2021100538Slave Slave Node NodeSlave Slave Node Node Master Node Slave Node Master Node Slave Slave Node NodeFig. 1IoT Agri Nodes Drone Multi- Data spectral DataDPP DPP (NR, Missing (RC) value ) 2021100538Crop health Data Set classification Crop health Preparation (MMDI) (NR, Missing maps value )Fig. 2
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU2021100538A AU2021100538A4 (en) | 2021-01-28 | 2021-01-28 | Crop Health Monitoring System Using IoT and Machine Learning |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU2021100538A AU2021100538A4 (en) | 2021-01-28 | 2021-01-28 | Crop Health Monitoring System Using IoT and Machine Learning |
Publications (1)
Publication Number | Publication Date |
---|---|
AU2021100538A4 true AU2021100538A4 (en) | 2021-04-15 |
Family
ID=75397226
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
AU2021100538A Ceased AU2021100538A4 (en) | 2021-01-28 | 2021-01-28 | Crop Health Monitoring System Using IoT and Machine Learning |
Country Status (1)
Country | Link |
---|---|
AU (1) | AU2021100538A4 (en) |
-
2021
- 2021-01-28 AU AU2021100538A patent/AU2021100538A4/en not_active Ceased
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Shafi et al. | A multi-modal approach for crop health mapping using low altitude remote sensing, internet of things (IoT) and machine learning | |
CA3129617C (en) | Yield forecasting using crop specific features and growth stages | |
Stanton et al. | Unmanned aircraft system-derived crop height and normalized difference vegetation index metrics for sorghum yield and aphid stress assessment | |
US8731836B2 (en) | Wide-area agricultural monitoring and prediction | |
CN108982369B (en) | Plot scale crop growth monitoring method integrating GF-1WFV and MODIS data | |
WO2018107245A1 (en) | Detection of environmental conditions | |
EP3803478A1 (en) | Training a machine learning algorithm and predicting a value for a weather data variable, especially at a field or sub-field level | |
Yun et al. | Framework of SAGI agriculture remote sensing and its perspectives in supporting national food security | |
JP2020140347A (en) | Crop management system and crop management method | |
Hama et al. | Examination of appropriate observation time and correction of vegetation index for drone-based crop monitoring | |
Zhu et al. | UAV flight height impacts on wheat biomass estimation via machine and deep learning | |
AU2021100538A4 (en) | Crop Health Monitoring System Using IoT and Machine Learning | |
AU2021107439A4 (en) | A Novel method- Crop Health Monitoring System Using IoT and Machine Learning | |
Doraiswamy et al. | Evaluation of MODIS data for assessment of regional spring wheat yield in Kazakhstan | |
Zheng et al. | Using high spatial and temporal resolution data blended from SPOT-5 and MODIS to map biomass of summer maize | |
Park et al. | Paddy rice inventory studies using drone imagery on a small town area in South Korea | |
Kubiak et al. | Remote sensing semi-automatic measurements approach for monitoring bioenergetic crops of Miscanthus spp. | |
Gumma | Pilot studies on GP level Crop yield estimation using Technology (Rabi 2020-21) using Sentinel 1&2 Satellite data for Rice and Wheat | |
Maitra et al. | Application of Remote Sensing & Geographic Information Systems for Agriculture-A Review | |
Matsumura et al. | Comparison between UAV and Satellite Data and Applying Deep learning to Classify Satellite Images for Agriculture Practices in The Eastern Hokkaido | |
CN109475080A (en) | The method for determining the attributes of vegetation of useful plant | |
Claverie et al. | Spatialization of crop leaf area index and biomass by combining a simple crop model SAFY and high spatial and temporal resolutions remote sensing data | |
Gumma | Crop yield assessment at Gram Panchayat level using technology Rabi–2022-23 | |
EP4250250A1 (en) | Carbon soil backend | |
Hoque | Variability of wheat phenology from Sentinel-1 and-2 time series: a case study for Brandenburg, Germany |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FGI | Letters patent sealed or granted (innovation patent) | ||
MK22 | Patent ceased section 143a(d), or expired - non payment of renewal fee or expiry |