Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
  • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
  • G06Q50/02—Agriculture; Fishing; Forestry; Mining
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
  • G06Q10/00—Administration; Management
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
  • G06Q10/00—Administration; Management
  • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/20—Image preprocessing
  • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/40—Extraction of image or video features
  • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
  • G06V10/507—Summing image-intensity values; Histogram projection analysis
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/10—Terrestrial scenes
  • G06V20/188—Vegetation
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/10—Terrestrial scenes
  • G06V20/194—Terrestrial scenes using hyperspectral data, i.e. more or other wavelengths than RGB
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/60—Type of objects
  • G06V20/68—Food, e.g. fruit or vegetables
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
  • G06V2201/10—Recognition assisted with metadata

Definitions

• the invention relates to an agricultural decision support system that classifies agricultural products and monitors the phenological development of the products by analysing multispectral satellite images with the help of machine learning and remote sensing techniques.
• Figure 1 shows the graph of the daily calorie consumption over the years. In order to adapt to this process, it becomes a necessity to produce effective and dynamic agricultural policies (support, incentives, etc.) and to make ground truth controls. For this, detailed, accurate and uninterrupted data production is required.
• UHUZAM operating under ITU
• UHUZAM is an application and research centre.
• This centre within the scope of the Agricultural Monitoring and Information System (TARBiL) project, periodic monitoring of the cultivated areas with high spatial resolution satellite images during the phenological period, determination of product types and spatial distribution by object-based image classification, production of plant indexes for tracking product development, transferring these images, classification and plant index data produced and statistical result reports to the relevant units under the Ministry of Food, Agriculture and Livestock are executed..
• the methods applied in the centre have not been converted into service, and the works carried out within the scope of the TARBIL project since 2008 have been brought to TRL 3 level, but beyond this level they have not been converted into ready-to-use models.
• Precision agricultural practices are developed by analysing agricultural areas with images taken from satellite and air platforms within Tiibitak UZAY.
• the products grown in the GAP region are analysed by using data from air platforms with multi band satellite images, including RASAT and Gokturk-2 images.
• RASAT multi band satellite images
• Gokturk-2 images As a result of the analyses, it is aimed to determine the crops sown in agricultural land, estimate the crop, the fertilizer and irrigation need of the soil, and the plant disease status.
• product tracking is done with the NDVI technique and the reflection values are recorded through this technique.
• the present invention relates to an agricultural decision support system that meets the above-mentioned requirements, eliminates all disadvantages, and brings some additional advantages.
• the primary purpose of the invention is to contribute to the production of quality and efficient food by using the resources correctly and on time; to provide solutions that will secure economic and sustainable agricultural production.
• the invention can be expressed as an agricultural decision support system that follows the phenological development of agricultural products by analysing multispectral satellite images with the help of machine learning and remote sensing techniques.
• generalization of reflection models tracking of the reflection values of the plant using all wavelengths offered by satellites and product classification with spectral product signatures are the most basic steps.
• using all bands (wavelengths) in satellite images will be a critical threshold for perceiving the soil in which the plant grows and thus determining the development level of the plant, as well as allowing a better understanding of the plant.
• agricultural knowledge and machine learning areas of expertise are combined with an interdisciplinary study, so that large data analysis can be performed from field scale to national scale based on soil characteristics and real-time satellite images.
• the obtained information can be segmented, and product identification can be made in parcel and sub-parcel detail, products can be separated according to their development level and efficiency, yield estimation can be made, and anomalies and damages in product development can be detected.
• Analyses mentioned in the invention can be carried out using computer vision techniques with a resolution below the plot scale.
• the accuracy of the cultivated area, yield estimation is increased.
• the same technical ability also improves the performance in terms of the location of the damaged area estimated by the damage detection module and the clarity of the area.
• the invention it is shown as the superiority of the accuracy and sensitivity of the plant indexes obtained based on the multispectral product signature over object- oriented indexes.
• a multispectral product signature is created, and in all subsequent analyses, remote sensing techniques are enriched with machine learning techniques.
• the invention is more precise and detailed than the systems in the prior art in terms of both product identification and analysis methods.
• the feature calculation technique used in classification is a new method. This method examines the effects of reflections at different wavelengths that make up the satellite images in relation to each other throughout the phenological phase. In this way, the acquired features become more distinctive and generalizable than the indexes currently in use. In addition, the segmentation and normalization existing in our solution provides a spatial resolution that can be exceeded by parcel resolution.
• the invention is based on the ideas of making decisions on current data using every new satellite image and automating agricultural processes. For this reason, the output formats are designed to be compatible with agricultural information systems in accordance with the general GIS formats and can be opened and examined in other commercial platforms (.shp, .dbf, .tif etc.). Since our solution does not require costly exploration methods such as infrastructure installation in production areas or using drones, it can be scaled quickly and used automatedly in large areas. In this way, it will be an infrastructure that can help decision makers to see the whole with the analysis of the big data produced.
• the method used in the system subject to invention makes decisions based on up-to-date data by using each new satellite image and automates agricultural processes.
• the stage of creating digital signatures of the product developments mentioned in the method includes creating a histogram from the pixel reflection values taken from each band of satellite images that coincide with the relevant parcels for the specified products, creating the probability density function (pdf) that gives the distribution probabilities of the pixel values for the relevant band (imaging wavelength) of the relevant satellite image of the product by normalizing the histogram with the total number of pixels, recording these probability density functions as temporal series by repeating in all bands (wavelengths) of each new satellite image throughout the phenological phase for different samples, creating multidimensional features on which plant growth is modelled, producing probability maps containing the expected reflection values of the specific product in a particular region throughout the year, as a result of this process repeated with each reference parcel obtained from the training data set, creating digital signatures of product developments by repeating this process for different products and different regions.
• PDF probability density function
• the product classification mentioned in the method is made by examining the effects of reflections of different wavelengths that form satellite images based on the matched features throughout the phenological phase in relation to each other.
• the development monitoring mentioned in the method is made by using multispectral reflection values obtained from satellite images by using all bands of the satellite, converting the development values of the products into metrics, and evaluating whether the product follows the expected development process, and using supervised learning techniques.
• the harvest estimation mentioned in the method is made by finding the damage presence on the planted areas on a parcel basis, determining the amount of damage on an area basis and to report the damage to the user by estimating in tonnage and using product development information and soil characteristics.
• the invention makes decisions based on updated data using each new satellite image and automates agricultural processes.
• the data mentioned in the method is stored for each agricultural parcel on a one-year or multi-year basis.
• Another preferred embodiment of the invention is a desktop, mobile (Android/ iOS), or web application platform that allows the information platform mentioned in the method to provide inputs to the analysis modules in line with the user authorization and the opportunity to examine their outputs, accessing the outputs of all analysis modules, reporting and visual outputs to be given according to analysis types.
• Spectral signature product catalogue which is one of the main outcomes of the invention and has not yet been produced in Turkey until today, however can be used in many areas of public and private organizations, is a strategic product for the needs.
• each administrative unit is represented with a sub-administrative level unit (country with provinces, province with districts, district with villages, village with parcels) in the sample platform view.
• Figure-8 demonstrates the village analysis result produced and visualized by combining sample parcel-based analyses (mosaic).
• Figure-9 demonstrates the district analysis result produced and visualized by combining the sample parcel-based analysis (mosaic).
• Figure- 10 demonstrates the presentation of the sample parcel-based analysis results as a series of images.
• Figure- 11 demonstrates the sample parcel -based analysis and visual and textual information.
• Figure- 12 demonstrates a sample analysis and classification result.
• Figure- 13 demonstrates the sample reference parcels and management screen.
• the data set is a factor that directly affects learning success. For this reason, there is a need for a high descriptive and large data set as possible. Selecting highly descriptive reference plots for products while creating this data set requires a significant amount of agricultural knowledge and experience. In order to meet this agricultural knowledge and experience, the data set has been shaped by field studies in line with the experiences in studies of our consultant professor and one of the inventors, who has been working on remote product detection for many years, Prof. Dr. Yusuf Kurucu, At the end of the study, a digital signature (reference catalogue) of the product developments at the level of sub-regions and phenological stage of selected field crops was created. Thus, in other words, a model of the usual development phase of the product is created by learning the expected multi-band reflection values during the phenological stages of the specified products.
• a histogram is created from the pixel reflection values taken from each band of satellite images that coincide with the relevant parcels. This histogram is normalized with the total number of pixels and the probability density function (pdf) is created for the relevant band (imaging wavelength) of the relevant satellite image of the product. These probability density functions give the distribution probabilities of pixel values for the relevant band of that product. These probability density functions are repeated in all bands (wavelengths) of each new satellite image throughout the phenological phase for different samples and are recorded as temporal series. Thus, multidimensional features on which plant growth is modelled are formed.
• Figure 5 shows a sample product development output. Analysis of Parcels Given as User Input:
• Our solution which is the basis for this application, is an analysis and information platform. Users define the agricultural parcels they want to analyse and the analysis types they want to apply through this interface. Access to analysis results is also made through this interface. The data obtained from analysis modules are presented to the user in visual and textual reports through the same interface. The interface, which enables different types of analysis to be performed through different menus for different user groups, can also present the reports as visual and textual reports customized for the relevant user types.
• the basis of our analysis is to examine the time-dependent changes of all bands of satellite images belonging to the analysed parcel and their correlations with each other.
• This module has undertaken the task of including every new satellite image received in the analysis without delay, thus ensuring the automatic continuous tracking.
• soil groups classified based on yield potential have been created by making similarity analysis in terms of basic soil properties taking into account the land slope characteristics, sea height, soil depth, salinity, alkalinity, drainage insufficiency, structure, soil lime content, characteristics, and by making query models in the Geographical Information System. These maps are used during yield and loss tonnage estimation.
• this analysis module development deficiencies can be detected during the production process and reported to the relevant decision makers on a location basis.
• the analysis outputs of this module which can be accessed by the notification platform, will be reported on the information platform in accordance with the relevant user profile.
• Statement control analysis module The product in the output of the classification module in the relevant parcels is compared with the farmer's declaration obtained by various methods (farmer registration systems, third party database containing the product declaration for insurance/credit/premium, etc.). If the result of agreement with the declaration is negative, a warning is given to the operator via the information platform.
• This platform which offers the user the opportunity to provide input to analysis modules in line with their authorizations and to examine their outputs, is designed to access the outputs of all analysis modules.
• Visual outputs (figures and graphics) to be given according to analysis types with reporting can also be accessed via the user platform.
• the information platform developed and prototyped as a desktop application, is also designed, and developed in accordance with mobile (Android / iOS) and web platforms.
• the system is based on the ideas of making decisions based on up-to-date data and automating agricultural processes using each new satellite image. For this reason, the output formats are designed to be compatible with agricultural information systems in accordance with the common GIS formats and can be opened and examined in other commercial platforms (.shp, .dbf, .tiff etc.). Since our solution does not require costly exploration methods such as installing infrastructure in production areas or using drones, it is possible to work in large areas automatically by scaling quickly. In this way, an infrastructure that can provide big data and help decision makers to see the whole will be created.
• the invention is a method that classifies agricultural products by analysing multispectral satellite images with the help of machine learning and remote sensing techniques in order to provide agricultural decision support to users systematically and follows the phenological development of the products and is characterised in that containing these operating steps:

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Physics & Mathematics (AREA)
• Business, Economics & Management (AREA)
• Multimedia (AREA)
• Human Resources & Organizations (AREA)
• Strategic Management (AREA)
• Economics (AREA)
• Tourism & Hospitality (AREA)
• General Business, Economics & Management (AREA)
• Marketing (AREA)
• Health & Medical Sciences (AREA)
• General Health & Medical Sciences (AREA)
• Entrepreneurship & Innovation (AREA)
• Animal Husbandry (AREA)
• Mining & Mineral Resources (AREA)
• Marine Sciences & Fisheries (AREA)
• Life Sciences & Earth Sciences (AREA)
• Primary Health Care (AREA)
• Operations Research (AREA)
• Quality & Reliability (AREA)
• Agronomy & Crop Science (AREA)
• Development Economics (AREA)
• Educational Administration (AREA)
• Game Theory and Decision Science (AREA)
• Management, Administration, Business Operations System, And Electronic Commerce (AREA)
• Image Analysis (AREA)

Applications Claiming Priority (2)

Publications (2)

Family

ID=76574660

Family Applications (1)

Country Status (2)

Families Citing this family (5)

Family Cites Families (6)

• 2020
  • 2020-12-02 EP EP20906140.7A patent/EP4081959A4/de active Pending
  • 2020-12-02 WO PCT/TR2020/051210 patent/WO2021133310A1/en unknown

Also Published As

Similar Documents

Legal Events