BR102022013738A2 - METHOD IMPLEMENTED BY COMPUTER AND AGRICULTURAL SYSTEM - Google Patents

Abstract

A geographical position of an agricultural machine is captured. Agricultural data that corresponds to a geographic position is received. Georeferenced visual clues are displayed that are indicative of the agricultural data received.

Description

METHOD IMPLEMENTED BY COMPUTER AND AGRICULTURAL SYSTEM CROSS REFERENCE TO RELATED ORDERS

[001] The present application is based on and claims the benefit of US provisional patent application Serial No. 63/231,870, filed on August 11, 2021, the contents of which are incorporated herein by reference in their entirety.

DESCRIPTION FIELD

[002] The present description refers to agriculture. More specifically, the present description refers to an infrastructure for accessing data and generating an augmented display.

FUNDAMENTALS

[003] To be successful in the field of agriculture, a farmer typically needs to make a relatively large number of decisions each year. These decisions can determine the farmer's success that year.

[004] Decisions can be quite comprehensive. For example, the farmer can make decisions about what types of crops to plant in a given year, when to plant them, how much fertilizer to apply and when to apply it, what types of pest or weed control agents to apply (and when ), among a wide variety of other decisions. These are just examples of decisions that are made by a farmer.

[005] Currently, mobile computing devices are widely used. These computing devices include tablets, wearable computers, smart phones, and personal digital assistants, to name a few. These mobile devices are often equipped with functionality that can be used to download and run mobile applications. They often include communication components that can be used to perform a relatively wide variety of different types of communication, such as communicating using the cell phone network, using a wireless local area network (such as Wi-Fi), as well as even using a wired connection (such as Ethernet or others). These mobile devices generally include functionality that allows them to exchange data or connect to a wide area network (such as the Internet) wirelessly using radio waves or other electromagnetic or other communication techniques. Additionally, these mobile devices often include image capture components such as cameras and position and orientation systems that allow mobile devices to calculate their own position and orientation.

[006] The above discussion is merely provided for general background information, and is not intended to be used as an aid in determining the scope of the claimed subject matter.

SUMMARY

[007] A geographic position of an agricultural machine is captured. Agricultural data that corresponds to a geographic position is received. Georeferenced visual cues are displayed to generate an enhanced display on a display device. The georeferenced visual clues are indicative of the agricultural data received.

[008] Example 1 is a computer-implemented method for controlling an agricultural system, comprising: receiving predictive visual data indicative of an expected view of a geographic location at an agricultural work site; receive a true visual representation of the geographic location; compare the expected visualization of the geographic location with the actual visual representation of the geographic location to identify an anomaly; generating a geographically referenced anomaly indicator indicative of the anomaly; and generating an enhanced display, the enhanced display showing the enhanced geographic location with the geographically referenced anomaly indicator.

[009] Example 2 is the computer-implemented method of any or all of the previous examples and further comprises: detecting a user interaction with the enhanced display; process the detected user interaction to identify additional anomaly information corresponding to the identified anomaly; and modify the enhanced view with the additional anomaly information.

[0010] Example 3 is the computer-implemented method of any or all of the previous examples wherein processing the detected user interaction comprises: performing diagnostic processing to identify, as additional information about the anomaly, a diagnostic factor that contributes to a possible cause of the anomaly; and generating a diagnostic factor display element, wherein modifying the enhanced display comprises adding the diagnostic factor display element to the enhanced display.

[0011] Example 4 is the computer-implemented method of any or all of the preceding examples wherein processing the detected user interaction comprises: performing prognostic processing to identify corrective information indicative of a possible remedy for the anomaly; and generating a possible remedy display element, wherein modifying the enhanced display comprises adding the possible remedy display element to the enhanced display.

[0012] Example 5 is the computer-implemented method of any or all of the previous examples in which the agricultural system includes an agricultural machine and further comprises: automatically controlling the agricultural machine to implement the possible solution.

[0013] Example 6 is the computer-implemented method of any or all of the previous examples and further comprises: identifying a severity level of the anomaly; and generating the geographically referenced anomaly indicator to be indicative of the severity level.

[0014] Example 7 is the computer-implemented method of any or all of the previous examples in which identifying the severity level of the identified anomaly comprises: generating an anomaly metric value indicative of a measure of difference between the expected view and the actual visual representation of the geographic location; comparing the metric value of the anomaly with a metric threshold value to obtain a metric difference value; and identify the severity level of the identified anomaly based on the metric difference value.

[0015] Example 8 is the computer-implemented method of any or all of the previous examples wherein receiving predictive visual data comprises: identifying a neighboring geographic location adjacent to the geographic location; and generate the predictive visual data based on a visualization of the neighboring geographic location.

[0016] Example 9 is the computer-implemented method of any or all of the previous examples wherein generating a geographically referenced anomaly indicator comprises: generating a geometrically shaped display element, a geometrically shaped display element of geometric shape being indicative of the geographically referenced anomaly indicator.

[0017] Example 10 is the computer-implemented method of any or all of the previous examples wherein generating a geographically referenced anomaly indicator comprises: generating a colored display element, a color of the colored display element being indicative of the geographically referenced anomaly indicator.

[0018] Example 11 is the computer-implemented method of any or all of the preceding examples wherein receiving predictive visual data indicative of an expected view of a geographic location at an agricultural work site comprises: obtaining a prior state of the geographic location and previous conditions affecting the previous state; generate a predictive state of the geographic location based on the previous state and previous conditions; and generate the predictive visual data based on the predictive state of the geographic location.

[0019] Example 12 is an agricultural system, comprising: an expected state identification system identifying predictive visual data indicative of an expected view of a geographic location at an agricultural work site; a comparison system receiving an actual visual representation of the geographic location and comparing the expected visualization of the geographic location with the actual visual representation of the geographic location to generate a comparison result; an anomaly identifier identifying an anomaly based on the comparison result; an anomaly output system generating a geographically referenced anomaly indicator indicative of the anomaly; and a display generation system generating an enhanced display, the enhanced display showing the enhanced geographic location with the geographically referenced anomaly indicator.

[0020] Example 13 is the farming system of any or all of the preceding examples and further comprising: a user interaction detector configured to detect a user interaction with the enhanced display; a workflow processing system configured to process the detected user interaction to identify additional anomaly information corresponding to the identified anomaly; and a display output system configured to modify the enhanced display with the additional anomaly information.

[0021] Example 14 is the agricultural system of any or all of the preceding examples wherein the workflow processing system comprises: a diagnostic system configured to perform diagnostic processing to identify, such as additional anomaly information, a diagnostic factor that contributes to a possible cause of the anomaly and generating a diagnostic factor display element, the display generation system modifying the enhanced display by adding the diagnostic factor display element to the enhanced display.

[0022] Example 15 is the agricultural system of any or all of the preceding examples wherein the workflow processing system comprises: a prognostic system configured to perform prognostic processing to identify corrective information indicative of a possible remedy for the anomaly and generating a possible remedy display element, the view generation system modifying the enhanced display by adding the possible remedy display element to the enhanced display.

[0023] Example 16 is the agricultural system of any or all of the previous examples in which the comparison system is configured to generate an anomaly metric value indicative of a measure of difference between the expected display and the actual visual representation of the geographic location and to compare the anomaly metric value to a metric threshold value to obtain a metric difference value, and wherein the anomaly identifier is configured to identify a severity level of the identified anomaly based on the metric difference value.

[0024] Example 17 is the agricultural system of any or all of the previous examples in which the display generation system is configured to generate, as the geographically referenced anomaly indicator, a geometrically shaped display element, being a geometric shape of the geometric shape display element indicative of the geographically referenced anomaly indicator.

[0025] Example 18 is the agricultural system of any or all of the previous examples in which the display generation system is configured to generate, as a geographically referenced anomaly indicator, a colored display element, a color of the colored display being indicative of the geographically referenced anomaly indicator.

[0026] Example 19 is the agricultural system of any or all of the preceding examples and further comprising: a prior state identification system configured to identify a prior state of the geographic location; a prior condition identification system configured to identify prior conditions that affect the prior state; and an expected state identification system configured to generate a predictive state of the geographic location based on the previous state and previous conditions.

[0027] Example 20 is a computing system, comprising: at least one processor; and a data store that stores computer-executable instructions that, when executed by the at least one processor, cause the at least one processor to perform steps, comprising: receiving predictive visual data indicative of an expected view of a geographic location at a location of agricultural work; receive a true visual representation of the geographic location; compare the expected visualization of the geographic location with the actual visual representation of the geographic location to identify an anomaly; generating a geographically referenced anomaly indicator indicative of the anomaly; and generating an enhanced display, the enhanced display showing the enhanced geographic location with the geographically referenced anomaly indicator.

[0028] This Summary is provided only to introduce some concepts in a simplified way. The concepts are described below in the Detailed Description. This Summary is not intended to identify key features or essential characteristics of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that resolve any or all of the disadvantages observed in the background.

BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figures 1A shows a block diagram of an illustrative agricultural system.

[0030] Figure 1B shows an example of a block diagram of a mobile device, which may be on an agricultural machine or carried by an operator or in another system.

[0031] Figure 1C is a block diagram of an example of a display generator.

[0032] Figure 1D is a block diagram of an example of a diagnostic/prognostic system.

[0033] Figure 2 is a flowchart illustrating an example of the operation of the system shown in Figure 1A when allowing mobile devices and other display devices to subscribe to a hosted data service.

[0034] Figure 3 is a flowchart illustrating an example of the operation of the system shown in Figure 1 in providing data to a requesting device.

[0035] Figure 4A is a block diagram illustrating an example of some data sets that can be served by the hosted data service shown in Figure 1.

[0036] Figure 4B shows an example of images.

[0037] Figures 5A-5B (collectively referred to as Figure 5) show a flowchart of an example of a display device in obtaining data and rendering a combined view.

[0038] Figure 5C is a block diagram of an illustrative combined view.

[0039] Figure 5D is an illustrative set of images.

[0040] Figure 6 is a flowchart that illustrates an example of calculating a location of the ground or agricultural surface that is being visualized.

[0041] Figure 7 is a flowchart illustrating an example of the operation of capturing an image and corresponding data and sending the image and/or data to a cloud-based service to obtain analysis results.

[0042] Figure 8 is a flowchart illustrating an example of receiving data captured in the cloud-based service.

[0043] Figure 9A is a block diagram of an example of a processed/derived data generation system.

[0044] Figure 9B is a block diagram of an example of an anomaly identification system.

[0045] Figure 9C is a block diagram of an example of an interleaved culture data processing system.

[0046] Figure 10 is a block diagram of an example of a display device.

[0047] Figure 11 is a block diagram illustrating an example of additional data sets that can be generated and/or saved by the hosted data service shown in Figure 1.

[0048] Figure 12 is a flowchart that illustrates an example of the operation of the system shown in Figure 1.

[0049] Figure 13 is a flowchart showing an example of receiving and displaying data.

[0050] Figure 14A and 14B show a flowchart showing an example of different types of data that can be displayed.

[0051] Figures 15-24 show example displays.

[0052] Figures 25-27 show several examples of mobile devices.

[0053] Figure 25-27 shows several examples of mobile devices.

DETAILED DESCRIPTION

[0054] While the fundamentals section mentioned some decisions that are made by a farmer, a number of other decisions must also be made throughout the growing season, and even after the growing season. For example, during the growing season, the farmer may want to evaluate the progress of various crops to determine whether additional fertilizers or pest or weed control agents should be applied. The farmer may also want to identify pests and weeds and other plants or animals in order to determine what types of control agents should be applied. After the growing season, the farmer may want to analyze yield data to determine if changes should be made during the next growing season.

[0055] All of these types of decisions are also influenced by a wide variety of other data. For example, all of these decisions can be influenced by the weather that was experienced during that year. Decisions can also be influenced by, among other things, the different types of soil that reside in a given field, the types and locations of application of fertilizers or other chemicals, and the locations of seeds. Decisions can be influenced by the type of equipment the farmer is using, the types of hybrids or plants the farmer is planting, and a whole host of other data.

[0056] Currently, some farmers try to track and consider all this information, when making their decisions, in several ways. Some farmers simply rely on their memory to make these decisions. Other farmers attempt to keep track of multiple items of information using a plurality of different spreadsheets on one or more different computing devices (such as a laptop or desktop computer). These farmers often attempt to access data through a wide area network (such as the Internet) to augment the data used to make their decisions. However, this type of information collection is complicated, time-consuming, and can be error-prone. It also consumes a large amount of computer processing and memory overhead. It is also relatively inaccurate. Additionally, trying to obtain and correlate data from multiple sources consumes a large amount of processing and memory resources in computing, rendering, and navigating through many user interface views.

[0057] The present description describes a system that can pre-process and store a wide variety of information. The information can be georeferenced to an agricultural area, such as a field. The information may be displayed in an augmented reality display where objects or items on the agricultural surface are enhanced in display by computer-generated perceptual information indicative of the georeferenced information. Perceptual information can be visual, auditory, tactile, or other information.

[0058] Figure 1A is a block diagram of an illustrative agricultural system 100. System 100 includes cloud 102. Figure 1A also shows that agricultural machine 103, mobile device 104 and other systems 105 can communicate with items in the cloud 102 over network 106. Network 106 may be a wide area network, a local area network, a near field communication network, a Wi-Fi network, a cellular network, or any of various other networks or combinations of networks. It will be appreciated that the mobile device 104 may be carried by an operator 110 of the agricultural machine 103 or mounted in the operator's compartment of the agricultural machine 103. Furthermore, the agricultural machine 103 may have its own display device 107 (in addition to or instead of the mobile device 104), sensor(s) 111, and other features of the machine 109. The agricultural machine 103 may be a harvester, a planting machine, a tillage machine, a sprayer, a tow vehicle (such as a tractor), a implement or other agricultural machine 103. The agricultural machine 103 may also be a towing vehicle that is towing an implement or another vehicle coupled to another implement or accessory. In the example discussed in this document, it is assumed that operator 110 uses mobile device 104 to access an agricultural data service hosted 112 in cloud 102. Service 112 may access data stores 114-116, also in cloud 102, or data stores 118, 120 and 122 that are located on other systems or devices, outside of cloud 102. Figure 1A also shows that hosted data service 112 can access other services 124 in cloud 102, or even additional services 126, which are provided outside of cloud 102.

[0059] Furthermore, Figure 1A shows that cloud 102 includes private cloud stores 128 that can store private data of multiple users or operator 110. In the example described herein, operator 110 is illustratively a farmer who has a cloud storage private cloud storage 128 that stores agricultural information for the farmer 110. The private cloud storage 128 is generally not accessible by other unauthorized persons.

[0060] In an example, hosted agricultural data service 112 illustratively includes data retrieval component 130, runtime search component 132, one or more processors (such as computer processors or servers) 134, application server 136, signature component 137, geotagged, pre-indexed data storage 138, processed/derived data generation system 139, and diagnostic/prognostic system 141. Each of these items will be described in more detail below.

[0061] The mobile device 104 and the display device 107 can take a wide variety of different forms, some of which are described below. In the present example, it is assumed that mobile device 104 is used to generate an interactive data display. In the example shown in Figure 1B, mobile device 104 illustratively includes distance calculation component 140, touch components 142, speech components 144, display component 146, positioning system 148, guidance system 150, display device 152, processor 154 , data storage 156 , user interface (UI) component 158 (which may include display generator 159 and other UI functionality 161 ), one or more mobile agricultural applications 160 , image capture/view component 162 , user input mechanisms 164, communication component 166, and one or more motion sensors 168 and other functionality of the mobile device 163. The positioning system 148 may include a position detector 165, position correction system 167, and other items 169. Position correction system 167 may include RTK correction system 171, low-resolution matching system 173, phytoreferencing system 175, UI interaction correction system 177, and other items 179.

[0062] Figure 1C is a block diagram showing an example of display generator 159 (of the mobile device 104 shown in Figure 1B) in more detail. The display generator 159 may be a display generation system that includes display output system 181, user interaction detector 183, touch gesture processor 185, menu interaction processor 187, speech interaction processor 189, interaction mode switching detector 191 and other items 193. Figure 1D is a block diagram showing an example diagnostic/prognostic system 141 (of the hosted agricultural data service 112 in Figure 1A) in more detail. Figure 1D shows that system 141 includes workflow trigger detector 195, workflow processing system 197, and other items 199. Workflow processing system 197 may include template 201, neural network 203, system rule-based 205, lookup tables 207, formula-based system 209, artificial intelligence-based system 211 and/or other items 213.

[0063] Before describing the general operation of system 100 in more detail, a brief description of some of the items and components shown in Figures 1A, 1B, 1C and 1D will first be provided. The hosted agricultural data service 112 is shown in a cloud computing architecture. In one example, cloud computing provides computing, software, data access, and storage services that do not require end-user knowledge of the physical location or configuration of the system that provides that service. In several examples, cloud computing delivers services over a wide area network, such as the Internet, using appropriate protocols. For example, cloud computing providers deliver applications over a wide area network and the applications can be accessed through a web browser or any other computing component. Software or system components 100, as well as corresponding data, may be stored on servers in a remote location. Computing resources in a cloud computing environment may be consolidated in a remote data center location or may be dispersed. Cloud computing infrastructures can provide services through shared data centers, even if they appear to be a single access point to the user. Thus, the components and functions described here can be provided from a service provider in a remote location using a cloud computing architecture. Alternatively, the components and functions may be provided from a conventional server or installed directly on client devices, or the components and functions may be provided in other ways.

[0064] The description is intended to include public cloud computing and private cloud computing. Cloud computing (public and private) provides a substantially seamless pooling of resources as well as a reduced need on the part of the entity using cloud computing to manage and configure the underlying hardware and software infrastructure.

[0065] A public cloud is managed by a vendor and typically supports multiple consumers using the same infrastructure. Additionally, a public cloud, unlike a private cloud, can free end users from managing hardware and software. A private cloud can be managed by the organization itself and the infrastructure is typically not shared with other organizations. The organization still maintains the hardware to some extent, such as installations and repairs, etc. In the example shown in Figures 1A-1D, all elements and data stores for the hosted agricultural data service 112 are shown as being arranged in the cloud 102. However, in other examples, parts of the hosted agricultural data service 112, or the data sources it maintains (such as data storage 138), may be located outside of cloud 102 and accessed by service 112. For example, data elements and stores may be hosted in a remote location by a service, or provided as a service through a cloud, or accessed through a connection service that resides in the cloud. Likewise, items in mobile device 104 and service 112 and other items in system 100 may be distributed in other ways and reside in different items in system 100 or elsewhere. All these architectures are covered in this document.

[0066] In service 112, data crawler component 130 illustratively crawls various websites or data stores to obtain relevant data. The data crawler component 130 intermittently crawls websites and data stores for up-to-date information and indexes that data and loads it into the pre-indexed geotagged data store 138. A wide variety of different data can be loaded into the data store. data 138. Some examples of this are discussed in more detail below in relation to Figures 4 and 11 and other Figures

[0067] Furthermore, additional data may be generated or derived from other data. For example, current runtime data may be obtained from agricultural machine 103 and other data sources and compared or aggregated by the processed/derived data generation system 139 and/or diagnostic/prognostic system 141. The results or aggregations comparison data can also be geotagged by system 139 and/or system 141 and stored in the pre-indexed geotagged data store 138. Some more examples of data generation by system 139 and system 141 are described below.

[0068] The runtime search component 132 illustratively performs runtime searches for data that is not pre-indexed and stored in data store 138. Such data may also be processed by the data generation system. data 139 and system 141. The mobile application server 136 serves to illustrate the various mobile applications that are downloaded by the various mobile devices 104.

[0069] The diagnostic/prognostic system 141 can execute workflows that perform diagnosis, operations and that perform prognostic operations. The workflow trigger detector 195 detects one or more triggers to initiate diagnostic, prognostic, and/or other workflows. Workflows may be executed by one or more components in workflow processing system 197. For example, the workflow may be executed using one or more models 201, neural network 203, rule-based systems 205, tables 207, formula-based system 209, artificial intelligence (AI)-based system 211, or other items 213. Triggers can be detected based on different variables detected, anomalies detected, or a wide variety of other items. Workflow processing system 197 executes workflows that can identify/diagnose problems and generate corrective output. The corrective output may be presented to the operator 110 or may take the form of automated corrective action control signals to automatically control the agricultural machine 103 or other items based on the diagnosis and/or prognosis.

[0070] In mobile device 104, display component 146 is illustratively any type of display component that operator 110 can use to obtain a view of their surroundings. For example, when the operator 110 is a farmer standing in a field or sitting on the agricultural machine 103, the viewing component 146 may be a camera lens or other viewer, which allows the farmer to view an agricultural surface, such as a piece of ground in the field. The distance calculation component 140 illustratively calculates a distance between the mobile device 104 and the piece of ground being viewed. The positioning system 148 calculates a position of the mobile device 104. The system 148 may include a position detector 165 that detects a geographic position (e.g., location and/or pose) of the mobile device 104 and the position correction system 167 can correct the detected position to make it more accurate. The real-time kinematic (RTK) correction system 171 can perform RTK-based correction. The low resolution combination system 173 can combine position signals from multiple lower resolution detectors 165 to obtain a higher resolution (more accurate) position. The phytoreference system 175 may detect phytoreference information to correct the position detected by the detector 165. The user interface interaction correction system 177 may use information from operator interactions with a monitor to correct the position signal of the detector 165. For example, a monitor may show a field boundary superimposed on the field being viewed on the monitor. If the overlaid boundary does not match the actual boundary, operator 110 can interact with the display to move the overlay boundary to the correct location over the boundary being viewed. This correction can then be used to correct the position signal of the detector 165. Other logic 179 can correct the position signal of the detector 165 in other ways, such as based on the position of other known items or markers, etc.

[0071] User interface component 158 includes display generator 159 and other user interface functionality 161 that can be used to control various user interface components on mobile device 104. An example of display generator 159 is described in further details with respect to Figure 1C and, illustratively, generates user interface displays on the display device 152 with user interaction mechanisms with which the operator 110 can interact to control and manipulate the mobile agricultural application 160 and other parts of the mobile device 104.

[0072] Figure 1C shows an example of display generator 159 that may be a display generation system that includes display output system 181, user interaction detector 183, touch gesture processor 185, touch gesture processor menu 187, mode switching interaction detector 191, speech interaction processor 189, and other items 193. The display output system 181 generates an output to the operator or user interface component 158. The user interaction detector 183 detects user interactions with the output. For example, user interactions may be touch gestures that are detected by detector 183 and processed by touch gesture processor 185. When user inputs are interactions with menus, these interactions are detected by detector 183 and processed by the touch processor. menu interaction 187. When speech recognition/synthesis functionality is provided, speech interactions are detected by detector 183 and processed by speech interaction processor 189. User interactions may begin in one mode (such as using tap) and switch to a different mode (such as using speech interactions). Switching to a different mode may be detected by interaction mode switching detector 191. For example, operator 110 may initiate interactions using touch gestures and then speak a wake-up word (e.g., “Johnny”) to trigger the interaction mode switching detector 191 to allow the speech interaction processor 189 to begin processing speech interactions. These are just examples.

[0073] Applications 160 are illustratively mobile applications that are downloaded from service 112 and that are served by mobile application server 136. In an example where displays are generated on display device 107, which may be a mobile device or a different device, then the application does not need to be a mobile application, but can be another application downloaded and run on the agricultural machine 103. The interaction mechanisms illustratively allow the operator 110 to interact with and manipulate the service 112. The image capture component 162 may be a camera or other component that illustratively allows the operator 110 to capture the image viewed through the display component 146. The touch components 142 may be hardware components that illustratively allow user interface displays displayed to the operator 110 be displayed on a display device that enables a touch-sensitive display device. The touch component 142 may be capacitive or other sensors. Touch components 142 allow the user to provide user input through user interface displays using touch gestures. The speech components 144 may include such things as a microphone, speaker, etc., and illustratively allow the user to provide input using speech. The guidance system 150 illustratively calculates the orientation of the mobile device 104. For example, the guidance system 150 may include a compass, gyroscopes, inertial measurement unit(s), accelerometer(s), or other components that may be used to calculate the orientation or pose of the mobile device 104. Other user input mechanisms 164 may be hardware or software user input mechanisms. For example, the mechanisms 164 may be mini keyboards or hardware keyboards, a thumb pad, joysticks, buttons, switches, etc. Furthermore, the mechanisms 164 may be mini keyboards or virtual keyboards that are actuated using touch gestures. The mechanisms 164 may also be user-actionable display elements, such as icons, links, buttons, text boxes, check boxes, drop-down menus, etc. The motion sensors 168 may be used to detect when the mobile device 104 is being moved. For example, accelerometers may detect whether a mobile device 104 is being panned, tilted, or otherwise moved. Likewise, the motion sensors 168 may include image analysis components that perform pixel-level image analysis on the image displayed by the display component 146 to determine whether the mobile device 104 is being panned to the right, left or right. , up or down or being moved in other ways.

[0074] Before providing a detailed description of the operations of system 100, a brief overview of the operation will be provided. It is first assumed that the operator 110 is illustratively a farmer or other person holding the mobile device 104 or having the mobile device 104 mounted in the operator's compartment of the agricultural machine 103 and is in a field or on another agricultural surface or on a piece of agricultural land. The operator 110 first triggers mobile agricultural applications 160 which, in themselves, allow the operator 110 to view an agricultural item (such as some portion of the field or agricultural surface, a plant, a weed, a pest, etc.) through the viewing component 146 (such as through a camera lens). When operator 110 is viewing a desired agricultural item (such as a patch of ground, a plant, a pest, etc.), operator 110 illustratively actuates a user interaction mechanism generated by mobile agricultural application 160 to indicate that the operator desires obtain agricultural information from the hosted service 112 about the viewed item (e.g., about the agricultural surface or piece of ground being viewed, the plant, pest, etc.). In response, and when desired, various components of the mobile device 104 calculate the position of the viewed item globally and relative to the mobile device 104, to identify the specific portion of the agricultural surface or ground being viewed via the visualization component 146.

[0075] Application 160 then generates a request for desired data to the hosted agricultural data service 112. Mobile application server 136 first determines whether the requested data is in the pre-indexed data store 138. If so, the server 136 provides this data back to the mobile device 104. Otherwise, the server 136 uses the runtime search component 132 to search and obtain the desired data. The search component 132 then provides this data back to the mobile application server 136.

[0076] In one example, mobile agricultural application 160 uses various components of mobile device 104 to display received data. For example, display generator 159 may generate a combined image of data received from service 112 and the ground (or other agricultural surface or item) being viewed via display component 146. For example, data received from service 112 is indicative of a particular feature of the ground or plant or pest being viewed and are used by display generator 159 to generate an overlay image that is rendered as an overlay on the image being viewed via display component 146.

[0077] As a specific example, where the operator (e.g., farmer) 110 is viewing a piece of land in a field, the operator 110 may request the location of drainage pipes in that field. In this case, the mobile application server 136 returns data indicative of the location of the georeferenced drainage pipe, relative to the patch of ground being viewed via the visualization component 146. The mobile agricultural application 160 then controls the display generator 159 to generate an enhanced image display, using the drainage duct location information, to overlay a visual indication of where the drainage ducts reside, over the image of the piece of ground being viewed via the visualization component 146. I.e. , an image of the drain duct is overlaid on the floor image to show the geographic correspondence between the location of the drain tile and the piece of ground being viewed. Operator 110 can then interact with the enhanced display in a number of different ways. User interactions may be detected by user interaction detector 183 and, depending on the type of interaction, may be processed by one or more of processors 185, 186, 189, and 191 or other processors. This is just one example, and other examples are discussed below.

[0078] It will be recognized that the signature need not be used. However, Figure 2 is a flow chart illustrating an example of the operation of system 100 in allowing operator 110 to subscribe to hosted agricultural data service 112. In one example, subscription component 137 first receives a request from operator 110 ( such as via a mobile device, a desktop computer, a laptop computer, etc.) that the operator wishes to subscribe to service 112. This is indicated by block 200 in Figure 2.

[0079] The subscription component 137 then performs a variety of different subscription operations for the operator 110 to subscribe to the service, as indicated by block 202. Such operations may include, for example, receiving authentication information 204 from the operator 110. These Authentication information may include account information, a password, a username, etc. The subscription operations may also include receiving device identification information that uniquely identifies the device 104 and associates it with the operator's subscription. This is indicated by block 206. The device identification information may also indicate whether the device 104 is a smart phone, a tablet, or another device that has a different format for its display screen.

[0080] The subscription component 137 may also offer the operator 110 a variety of different subscription levels. For example, it may be that operator 110 may subscribe to obtain data from certain databases for one price and subscribe to obtain other data and additional services for another price. Identification of the desired data and services that operator 110 wishes to subscribe to is indicated by block 208 in Figure 2.

[0081] Subscription operations 202 may also include identifying a payment mechanism that operator 110 wishes to use to pay for the subscription (if sold). This is indicated by block 210 in Figure 2. Of course, it will be noted that signature operations may also include other operations, and this is indicated by block 212.

[0082] Once the operator 110 has successfully subscribed to the service 112, the mobile application server 136 illustratively downloads one or more mobile agricultural applications 160 that run on the mobile device 104 for the operator 110 to access the service 112. Offloading of mobile applications is indicated by block 214 in Figure 2.

[0083] In one example, the operator 110 may choose to use the hosted agricultural data service 112 to obtain data and services, but keep its data locally (such as on a mobile device 104, in a data store on the agricultural vehicle 103, or on your desktop or laptop, etc.). Alternatively, however, operator 110 may illustratively choose to have its data stored in a private cloud data store 128, which is accessible only by operator 110 or other authorized users. If the operator chooses to use the private cloud data store 128, then the mobile application server 136 illustratively allocates private cloud resources in the data store 128 to the operator 110. Causing the operator 110 to choose to use the private cloud data storage 128 and allocating resources are indicated by blocks 216 and 218 in Figure 2, respectively.

[0084] After subscription, operator 110 can use service 112 to obtain data and services. Figure 3 is a flowchart illustrating an example of the operation of the hosted agricultural data service 112 in obtaining and/or generating pre-indexed data for storage 138 and responding to data requests from a mobile device 104. In one example, service 112 illustratively includes data retrieval component 130. Component 130 may be a crawler that continuously, or intermittently, crawls and pre-indexes a variety of data sources to identify new data that may be of interest to various operators 110 or other users. This data can be accessed in an illustrative manner by the operator 110, given its particular subscription. The data sources tracked by the tracker component may be data sources stored and served in the cloud 102, such as data sources 114 or 116, or may be data stores hosted on other websites or otherwise accessible by the component 130, e.g. through cloud 102. These data stores are indicated by data stores 118, 120 and 122.

[0085] In another example, component 130 is a loading component that receives data uploaded by the farmer (or other user), crowdsourced entities, or other servers or data stores. Component 130 may also obtain data in other ways.

[0086] It will also be noted that component 130 can obtain data through a variety of services. For example, several different services can be accessed to identify weed varieties, pest identities, seed information, or other information. These services may include image processing services, speech or pattern recognition services, database services, etc. In addition, several services provide meteorological information, information on soil type and a whole range of other information. The services and the processed/derived data generation system 139 may also be used to perform analysis of data that is collected and sent to these services. The diagnostic/prognostic system 141 may perform analysis or data and generate additional data, control outputs, or other outputs. Different types of data are described in more detail below in relation to Figures 4 and 11.

[0087] The hosted agricultural data service 112 may obtain information from services that are hosted in the cloud 102 (such as services 124) or those that are hosted elsewhere and are accessible by the service 112 (such as other services 126). Interaction with data sources or data services to identify new data of interest is indicated by block 220 in Figure 3.

[0088] Component 130 also illustratively includes an indexer and a geotag. The indexer indexes the newly obtained data identified or obtained by component 130 and stores in the geotag tags the newly obtained data with geographic information when there is none and when geographic information is available or can be generated for the newly obtained data . The component 130 then stores the newly obtained data in the pre-indexed geotagged data store 138, as indicated by block 222. By way of example, where the weather data is pre-indexed, the weather data will be illustratively specific to several different geographic locations. The weather data is therefore tagged with the geographic information so that the weather data can be retrieved based on the geographic locations of interest. The same can be true for weed data, pest data, soil type data, and a wide variety of other types of data. Data can be indexed in many different ways.

[0089] At some point, the mobile application server 136 illustratively receives a request for information from the mobile device 104. Receiving a request for data is indicated by block 224 in Figure 3. As described below, the data request will illustratively include a variety of information, such as the geographic location of the image of the agricultural surface or item being viewed on the mobile device 104, perhaps the image itself, and other information.

[0090] The mobile device application server 136 illustratively uses the information in the request to determine whether the requested data is available in the pre-indexed, geotagged data store 138. This is indicated by block 226. If so, the server 136 retrieves the requested data from data store 138, as indicated by block 228. Otherwise, server 136 illustratively prepares a search request for the run-time search component 132 or obtains data in other ways, as indicated by block 230. Component 132 may be a browser, search engine, or other component that takes a query as input and searches various websites, data stores, services, etc. to get relevant search results. Performing a run-time search for the requested data and receiving the search results is indicated by blocks 229. It will be recognized that server 136 may also search one or more caches for the requested data.

[0091] Additionally, server 136 may also obtain the requested data in other ways. For example, when the data received in the request is an image of a plant or pest and the data requested is to identify some attribute of the image (such as identifying the plant or pest or determining whether the plant is as mature as it should be) the image may be sent to another entity for a response. For example, an image of a pest can be sent to an agronomist for identification. An image of a plant can be sent to a seed company to identify whether the plant is healthy and maturing properly. An image can be sent to an automated image processing system for recognition. Request data may be sent to other services as indicated by block 231. These are examples only.

[0092] Furthermore, in one example, the data generation system 139 and/or diagnostic/prognostic system 141 receives information (such as the operator's request or data from different services or external websites) and generates additional data as indicated by the block 233. For example, the data generation system 139 may obtain the operator's location as well as information from a fertilizer company and generate a suggested fertilizer application map that can be downloaded for the user's use or that can be downloaded automatically to the user's fertilizer equipment and used by that equipment. In another example, the diagnostic/prognostic system 141 may receive an image or data from the request and identify a problem and a solution based on the received image or data. This can be used to send data for display to the operator in an enhanced display or to generate control signals. These are just examples. Receipt of results to respond to the request is indicated by block 232.

[0093] Regardless of how the requested data is obtained (whether from data storage 138 or by performing a search or generating it using data generation system 139 or system 141 or otherwise), the data can be formatted for rendering on the device specific requester 104. For example, if the mobile device 104 is a smart phone, where the display space that can be used to generate a display is relatively small, the information may be formatted in a manner. However, if the mobile device 104 is a tablet computer, with a larger display area, the data may be formatted differently. Data formatting for rendering on the requesting device is indicated by block 234 in Figure 3.

[0094] It should also be noted that formatting the data for proper rendering can be performed in a wide variety of different locations and at a wide variety of different times. For example, when data is stored in data storage 138, the data may be stored in different formats so that it is easily accessible for return (in the appropriate format) to different types of display devices. That is, the same data may be stored in store 138 in a format suitable for rendering on a smart phone, in another format suitable for rendering on an alert monitor, in another format suitable for rendering on a tablet computer, and in yet another format that is suitable for rendering on a laptop or desktop computer, etc. Alternatively, the data may simply be provided back to the specific requesting device 104, and that device 104 may be responsible for formatting the data for proper rendering on its own display screen. Formatting can be performed in other locations and at different times as well, and those discussed here are discussed for example purposes only. In either case, the requested data is eventually sent back to the requesting display device, as indicated by block 236 in Figure 3. The requesting display device may be a mobile device 104, an in-cab display device 107, or another display device. display 237.

[0095] Before describing the operation of mobile device 104 and other items in system 100 in more detail, some example data sets will be described. Figure 4A shows an example of several different data sets that can be obtained by the data acquisition component 130 and generated by the data generation system 139 and diagnostic/prognostic system 141 and stored in the pre-indexed and geotagged data store. 138, or which may be retrieved at runtime by the runtime search component 132 or by the server 136 or other components in other ways (such as crowdsourcing, independent loading, etc.). Data sets are generally indicated by the number 238 in Figure 4A.

[0096] Some of the data sets may include 240 soil types. By way of example, soil samples may be obtained and soil types may be analyzed and stored for a variety of different geographic locations. Currently, there are some databases that already contain geographically indexed information that indicates the different types of soil located in different locations. Soil type data 240 may also be generated in other ways. Soil types may include, for example, clay, clay, sand, silt, peat, saline, etc. ), or in other ways as well.

[0097] Some data stores may also include soil sample data 242. By way of example, a farmer may have already collected soil samples from various locations in the field of interest (the field being viewed). These soil samples can be analyzed to determine a large amount of information (represented by sample data) about the individual soil sample that was collected. The soil sample data is illustratively correlated with the geographic location from which the samples were taken and the sample data may be stored for access by the hosted agricultural data service 112.

[0098] Data sets 238 may also include fertilizer application data 244, such as fertilizer application rates that were previously used. Data 244 may also include fertilizer content and application rates that are suggested under current field conditions being viewed by operator 110.

[0099] Data sets 238 may also include the location of drainage pipe 246, historical yield data 248 and previous equipment configurations, or data about previous equipment that was used to cultivate the field of interest, on previous occasions, as indicated. by block 250. The above equipment configurations and equipment data 250 may include a wide variety of different information. For example, data 250 may include last year's combination settings, the downward force used by the seeder to plant the seed, or a wide variety of other equipment settings and data.

[00100] Data sets 238 may also illustratively include seasonal weather data 252. The weather data may include rainfall data 254, weather maps 256, pest maps 258, hot days 260 and other weather information 262. By way of example , pest maps 258 may include data indicative of common pests in the geographic region, pests that have been detected there, or other information. The data may also include images of pests that can be used to identify pests that the farmer has encountered and other information.

[00101] The data sets 238 may include planting data 264. The planting data may include, for example, the date 266 of previous plantings, the specific hybrid 268 that was planted, the depth 270, the population 272, the data of fertilizer application 274 and a wide range of other planting data 276.

[00102] Additionally, data sets 238 may include data from sellers 278. Sellers may include a wide variety of different sellers, such as seed companies, fertilizer companies, equipment manufacturers, and other sellers. When the seller is the seed company, for example, the data may include information that can be accessed to determine whether the crop is developing as expected under the circumstances. By way of example, the seed company may take into consideration the location of the agricultural surface or item being viewed by the farmer, the soil type, climate data, and fertilizer data for that geographic location and estimate the expected plant height 280 or other plant maturity attributes 281, under all relevant conditions. The data may also include the expected appearance of the plant 282 and a wide variety of other data 284. As an example, maturity attributes 281 may include images of a given hybrid at various stages of growth. Figure 4B, for example, shows images of a given plant at three different growth stages. The steps may correspond to different times after planting, considering different climatic conditions, different types of soil and all other relevant information corresponding to the location of the plant. A farmer can request data that shows what this particular plant should look like given all the other ground data the farmer is viewing. The system can access a mapping between growing conditions (e.g. climate, soil type, fertilizer history, etc.) and images of the plant at different levels of maturity, and an image of the specific growth stage can be returned to the farmer based on all the information. For example, if the plant is hybrid A and is located on a certain soil type where the climate is generally cool and a first level of fertilizer has been applied, the image returned may be the stage 1 image. plant is hybrid A, but is in a location with a different soil type and where a second level of fertilizer was applied, the image returned may be the stage 3 image. The same can be done for pest infestation. For example, the farmer may submit a request that has an image of a plant with certain characteristics indicative of a pest. The image can be compared with stored images of plants that have been affected by various pests to identify an image similar to the one submitted by the farmer. This can be used to indicate that the plant is being affected by a certain pest and that certain pesticides should be applied. These are just examples and others are described below. It should be noted that although seller data 278 has been provided in relation to whether the seller is a seed company or a fertilizer company, the data will vary based on the specific seller. It can also come from universities, agronomists or others.

[00103] Figures 5A and 5B (collectively Figure 5) show a flowchart illustrating an example of the general operation of the architecture 100 (and more specifically the mobile device 104) in generating an enhanced view of an agricultural surface by overlaying visual cues onto it. view, based on data received from agriculture hosted data service 112. It should be noted that the generated enhanced view may also be a plant that the farmer is looking at, and the overlaid information may be an image of a plant as described above. Furthermore, it will be noted that the enhanced view may be generated as an alert monitor in a machine window 103 or the enhanced view may be generated as a projected view or in other ways, but is being described as an enhanced view on the mobile device. 104 for example purposes.

[00104] Mobile device 104 first receives input from operator 110 by launching one or more of mobile agricultural applications 160. This is indicated by block 300 in Figure 5.

[00105] Operator 110 then provides inputs to invoke visualization component 146 to obtain a view of an identifiable agricultural item (such as an agricultural surface or patch of ground 209, a plant 301, weed 303, pest 305, or other item 309 ) near which operator 110 is standing. This is indicated by block 302. This may be done, for example, by the operator triggering the image capture functionality (e.g., the camera functionality) on the mobile device 104. In this case, the view through the lens is illustratively displayed on a display device 152 (such as a display screen) on the mobile device 104, so that the operator 110 can view the agricultural surface (e.g., piece of ground) or plant through the lens of the image capture component 162.

[00106] When the piece of ground (or plant or other image) is being viewed, the application 160 illustratively controls the distance calculation component 140 and the positioning system 148 to calculate a location of the piece of ground being viewed, as indicated by block 304. An example of how this is done is described in more detail below in relation to Figure 6.

[00107] Operator 110 then provides user input to mobile device 104 indicating that operator 110 wishes to obtain data for the piece of ground (or plant or other image) being viewed. This is indicated by block 306. By way of example, application 160 may provide a set of user interaction mechanisms that are displayed to operator 110 on display device 152. The user interaction mechanisms may allow the user navigate menus or other user input mechanisms to select the data the user wants to see. Application 160 may, for example, display buttons or icons that correspond to drainage pipe data, weather data, planting data, plant image data, pest image data, or any of the various data sets described above in with respect to Figure 4A and those described below with respect to Figure 11. When the user triggers one of these user interaction mechanisms (such as with a tap gesture), the application 160 generates a request for this data, along with the location of the chunk. of ground (or plant or pest, etc.) being viewed and sends the request to the hosted agricultural data service 112. Sending the request is indicated by block 308 in Figure 5.

[00108] The request itself can include a variety of different things. For example, it may illustratively include authentication information that identifies the operator 110 to the hosted agricultural data service 112, as indicated by block 310. The location information may also include the location of the agricultural surface or ground being viewed, as indicated by the block 312. Location may be identified using GPS coordinates, other location information (such as a location derived from an estimated navigation system, a cellular triangulation system, or a LORAN system), and the location information may also illustratively include the orientation of the mobile device. The request will also illustratively include an indication of the data being requested, as indicated by block 314. The location information may also include other information 316.

[00109] The hosted agricultural data service 112 receives the request and obtains the requested data and sends it back to the mobile device 104. The service 112 does this, for example, as described above in relation to Figure 3.

[00110] Mobile device 104 then receives the requested data from hosted service 112, as indicated by block 318 in Figure 5. Application 160 then uses user interface component 158 to generate an enhanced display that renders an image indicative of the data received (received from hosted service 112) overlaid on the view of the surface or agricultural item (e.g., piece of ground or plant or other image) being viewed via visualization component 146. This is indicated by block 320 in Figure 5. It will be noted that the view does not need to be overlaid, but simply displayed anyway. The view may also be displayed on a separate display device 107 (such as one mounted in the cab of a piece of agricultural equipment, a warning monitor in glasses, a warning monitor in a cab window, a projected display that is projected onto the field, or otherwise) as indicated by block 323.

[00111] Generating the enhanced display can be done in several different ways. For example, the user interface component 158 may output the enhanced image as a composite image or as two separate images, one being superimposed on the other. The user interface component 158 may also generate the enhanced images in other ways. The actual rendering of the enhanced image may use inputs from the various sensors and components in the mobile device 104 to accurately render a representation of the enhanced image indicative of the received data being overlaid or overlaid on the image being viewed.

[00112] By way of example, the orientation detection system 150 can detect whether the mobile device 104 is being tilted downward or upward, or to the right or left. The motion sensors 160 may determine whether the device is being moved (such as panning left or right or otherwise). The distance calculation component 140 may identify the focal point of the image capture component (such as the focal point of the camera lens) to determine how far the piece of ground is from the mobile device 104 (and therefore the operator 110 ). This can be used to provide a perspective for rendering the overlaid data.

[00113] Taking the example of the drainage duct, the application 160 uses the various components of the mobile device 104 to accurately overlay the location of the drainage blocks over the floor view being displayed to the operator 110. The overlay can be done using the proper perspective so that visual cues indicating where the drainage pipe resides in relation to the agricultural surface are accurately located in relation to the image being displayed. Taking the example of a floor plan image, the application 160 illustratively displays the floor plan image received from the service 112 over, side by side, or otherwise juxtaposed with the floor plan image being viewed.

[00114] Figure 5C is a block diagram of an illustrative user interface display 322. In the example shown in Figure 5C, the user interface display 322 is shown on a display device 324. The display device 324 may illustratively be an LCD monitor (or other display device) that displays the view from a camera lens or other image capture component 162 on the mobile device 104. The illustratively enhanced image includes a view of the agricultural surface (e.g., ground) 326 being viewed through the display device 324, together with a visual indication showing an image indicative of the data received from the service 112 overlaid or otherwise displayed on the floor view 326. The overlaid data is indicated by 328 in FIG. 5C.

[00115] Figure 5D shows another example. In Figure 5D an image 327 of a plant being viewed by the user is shown in juxtaposed relationship to an image 329 received from the service 112. Image 329, for example, shows an image of a plant, as the seed seller or agronomist believes. which should remain, having all the information for the location of the plant. If the two are not the same, the user can also ask for suggestions such as different rates of fertilizers, pesticides, etc. Likewise, as discussed below, the processed/derived data generation system 139 includes an anomaly identification system that compares expected data to actual data to identify anomalies that may arise for the operator 110 or other users or systems and/or or they can be used for control. Likewise, the diagnostic/prognostic system 141 may generate information indicative of a likely cause for the anomaly or other information.

[00116] Overlaid (or juxtaposed) data can be displayed in several different ways. For example, the data may be used to shade view 326 in varying colors 330 based on the particular data being displayed. By way of example, if the data displayed on the floor view is of the floor type, different colors can be used to indicate different floor types that are located on the floor being/viewed. Of course, colors can also be used to indicate other things, such as the estimated soil moisture, the fertilizer application locations and rates that were used (or suggested) in the various locations, the types of pests or weeds that were identified or expected on that piece of ground, etc.

[00117] The overlaid data can also be generated in the form of another image 332. For example, the location of the drainage duct (or plant growth stage) can be represented by a dashed line or another image that resembles a duct of actual drainage (or plant). The overlaid data 328 may be displayed as a computer-generated graph 334 or as alphanumeric information 336. For example, the alphanumeric information 336 may be a numeric indicator that identifies a planting depth as it varies across the ground being viewed. The alphanumeric information 336 can also be used to indicate many other things. Of course, the overlaid data 328 can be a graph (bar chart, pie chart, histogram, etc.), an icon, tiles, different shapes, or other indicia. The clues may be centered on plants or rows or furrows and may be indicative of a nozzle spray width, estimated root zone, or comprised of other visual indicators 338 as well.

[00118] At any time while the operator 110 is viewing the floor view along with the overlay data, the operator 110 may provide input requesting that the information representing the view and the overlay data be sent to another person or sent to storage . This is indicated by block 340 in the flowchart of Figure 5. For example, operator 110 may direct that the combined image be emailed to an agronomist, the seed company, or anyone else. When this is done, application 160 illustratively interacts with communication component 166 to send the image as desired, as indicated by block 342.

[00119] Operator 110 may also illustratively move mobile device 104 to change the view of the agricultural surface, such as the ground (or plant) in the generated image, as indicated by block 344 in Figure 5. By way of example, operator 110 may move the mobile device 104 by moving it to the right or left or tilting it up or down to view a different piece of floor, as indicated by block 346.

[00120] Additionally, operator 110 can change the distance of the mobile device 104 from the piece of ground being viewed, as indicated by block 348. To do this, operator 110 can illustratively change the focal point of the capture component of image 162 that is being used to generate the image. For example, when the image includes a piece of ground extending from approximately three feet away from mobile device 104 to approximately forty feet away from mobile device 104, operator 110 may illustratively touch the touch-sensitive display screen at a portion of the image to shift the camera's focal point to the tapped location. This may have the effect of moving the location of the ground being viewed from a location closer to the operator 110 to a point further away from the operator or vice versa.

[00121] In another example, application 160 displays a distance cursor on the display screen that can be manipulated by operator 110 using touch (or other) gestures to indicate a distance from the mobile device 104 to which data is being wanted. For example, it may be that data is being searched for operator 110 close to the ground. Operator 110 may then wish to retrieve data for a more distant piece of ground. Operator 110 may trigger the cursor to indicate that he or she wishes to retrieve data for a patch of ground 1/8 mile away from the operator, etc. These measurements are provided for example purposes only. Distance change component 350 is shown in the user interface display of Figure 5C to represent the cursor or other touch-sensitive distance change mechanisms that can be used to change distance. Of course, distance change mechanisms can be manipulated using a stick, a thumb switch, or another mechanism other than touch gestures.

[00122] Referring again to the flowchart of Figure 5, operator 110 may also move or manipulate the visualization component in other ways. For example, the operator can simply walk across the field so that the image of the ground or plants being viewed changes. Moving the view component in other ways is indicated by block 352 in Figure 5.

[00123] While operator 110 is viewing a piece of ground or plants, operator 110 may also provide input requesting different data for the same piece of ground or plants, as indicated by block 354 in Figure 5. For example, the operator 110 may first request that the drainage duct data be overlaid on the floor image. Operator 110 may then request that fertilizer application rate data be overlaid on the viewed ground. The operator 110 may then request that the floor type data be overlaid onto the floor image. Operator 110 may then request that an image of expected plant maturity be displayed. All of these images can be accumulated to generate an enhanced view of the floor with multiple overlapping data sets, or each overlapping data set can be erased when a new one is requested. In any case, when new information is requested in block 354 in Figure 5, processing reverts to block 308 where a request for the new data is sent from the mobile device 104 to the hosted agricultural data service 112.

[00124] It should also be noted that, at any point, the operator 110 can provide user input to close the mobile agricultural application 160, as indicated by block 356 in Figure 5. When this happens, the application can be closed.

[00125] Another feature should also be mentioned. At any time when the operator is viewing information, the operator can provide user input to save this information to the operator's stored agricultural data. If the user is using private cloud storage 128, when storage input is received from the operator, the application 160 will provide the information to the mobile application server 136 in the hosted data service 112 for storage in the operator's private cloud storage 128 If the operator is storing the information locally or in a different storage, then the application 160 sends the information to storage in that location.

[00126] The information is stored in an illustrative form along with a date/time stamp so that it can be retrieved later at any time. This allows the 110 operator or other service to perform historical analysis against the captured image and overlay data.

[00127] Figure 6 shows an example of the operation of the distance calculation component 140 that can be used by the application 160 to obtain a measurement of the position of the piece of ground or item being viewed and to generate an enhanced image that shows it in perspective , in relation to the position and orientation of the mobile device 104. It should be noted that this can be done in a wide variety of different ways and that described in relation to Figure 6 is just an example.

[00128] Application 160 illustratively identifies the location coordinates of mobile device 104 using positioning system 148, as indicated by block 360 in Figure 6. The location coordinates illustratively include a longitude 362, a latitude 364, an elevation 366 and may also include other information as indicated by block 368. Location coordinates may be obtained using a variety of different position detectors 165. For example, location coordinates may be obtained using a global positioning system (GPS) , using a long-range navigation aid system (LORAN), using any of a variety of estimated navigation systems, using a cell tower triangulation system, or using other systems. The detected location can also be corrected by the position correction system, as indicated by block 367.

[00129] Once the location of the mobile device 104 is identified, the application 160 uses the distance calculation component 140 to identify an offset of the focal point (or the ground being seen as identified by another feature) of the mobile device 104, as indicated by block 370 in Figure 6. In one example, the distance is distinguished in terms of its direction from the mobile device 104, as indicated by the block 372. The distance is also distinguished by the distance 374 from the mobile device 104, any change in elevation (such as if the user is pointing the camera on the mobile device 104 slightly up or down to accommodate terrain, etc.) as indicated by block 376, and the offset can also be calculated using other information 378.

[00130] The application 160 also illustratively identifies the physical size of the floor to be viewed. This can be determined illustratively based on the zoom in/out configuration of the view finder (or camera) being used to generate the ground view. As an example, if the zoom in/out setting is set to 1x, the size of the floor being viewed can be calculated as a first size. If the zoom in/out setting is set to 5x, the floor size may be calculated as a different size. Identifying the physical size is indicated by block 380. Identifying the field of view size based on the zoom in/out configuration is indicated by block 382. Calculating the physical size of the floor being viewed can also be performed in other ways. ways, as indicated by block 384.

[00131] Once all this information is received, the distance calculation component 140 illustratively calculates the location and boundaries of the agricultural surface or ground being visualized in the displayed image, as indicated by block 386 in Figure 6. In one example, The location signal that identifies the location and boundaries of the ground being viewed can be generated by a GPS receiver in the mobile device 104. In one example, the position correction system 167 can obtain high-precision location data that can be used to correct the GPS signal or other position signal detected by the position detector 165. For example, during planting, high accuracy (e.g., RTK corrected data) can be generated for the locations where individual plants are located . If plants have emerged, the high-precision location of the plants in the view can be used to correct the lower-precision GPS signal from the GPS receiver in the mobile device 104. Correction of the GPS signal is indicated by block 387. Other corrections 389 as well can be performed, as discussed elsewhere. All of this information, or a subset thereof, may be output as the location of the floor being viewed, as indicated by block 388. Again, it will be noted that the particular location, boundaries, and orientation of the floor being viewed may also be calculated in other ways, and the description in relation to Figure 6 is just an example.

[00132] Operator 110 may also use mobile device 104 to capture images or other data and send it to service 112 for storage or analysis. Figure 7 is a flowchart illustrating an example of the operation of the mobile device 104 in capturing data (such as an image or other data) and sending it to the hosted agricultural data service 112 for analysis or storage. In the example shown in Figure 7, the operator 110 first uses the image capture component 162 to capture an image on the mobile device 104, as indicated by block 390 in Figure 7. The image may be of an agricultural surface, such as the ground ( as discussed above) or it may be an image of a plant 392, a weed 394, a pest 396, animal tracks 397 or any of a wide variety of other information 398.

[00133] Application 160 illustratively tags the image with the desired data, as indicated by block 400 in Figure 7. By way of example, the image can be tagged with time, date and location information as indicated by block 402. Or the image may be marked with other information as indicated by block 404.

[00134] The image and added data (if any) are then sent to the agricultural data service hosted 112 in cloud 102. Sending the image and data to the service 112 is indicated by block 406 in Figure 7.

[00135] As discussed above, operator 110 may illustratively request that information be sent to others. If so, the information may be sent directly by the mobile device 104 using the communication component 166 or the information may be sent by the hosted agricultural data service 112. Sending the image and/or data to others is indicated by block 408 in the Figure 7. Sending the information (image and/or data) to other services 124 or 126 is indicated by block 410. The information can also be sent to an agronomist 412, to the seed company 414 (or other sellers), or for other people 416 too.

[00136] The hosted agricultural data service 112, such as the processed/derived data generation system 139 and/or the diagnostic/prognostic system 141, can then perform processing on the image and data it has received. Furthermore, any other recipient of the information may perform the processing and return the results of that processing, as indicated by block 418. This processing may take a wide variety of different forms. For example, in one example, operator 110 captures an image of a plant and attaches the location of that plant. The image and location are then sent to a seed company or an agronomist who compares the stored image with an image showing the expected maturity of the plant (as shown above in Figure 4B), at that location, given the hybrid, climate data , soil type information, fertilizer information, and other information that was attached or corresponding to the image. The seed company can then illustratively send back a comparison image showing the expected maturity of the plant (e.g., what the plant should look like under these circumstances, as shown in Figure 5D) and superimposed on the image of the actual plant that was sent by operator 110. The seed company can also send recommendations (such as increasing or decreasing fertilizer application, etc.).

[00137] The image and/or data can also be automatically mapped to other stored images. For example, if the image is a plant, the service returning the data might obtain the location of the plant, the weather and soil data for that location, the fertilizer and cropping history, the herbicide or insecticide or other chemical application, the hybrid, and map to an image of the plant at expected maturity, with all the information. Mapping of the image and/or data to stored images is indicated by block 419. The image and/or image may also be automatically compared to libraries of other images and/or data or examined by consultants for identification purposes. For example, where operator 110 captures an image of a weed, it may be viewed by a consultant or compared with other images in a weed identification service, to identify the weed and send the weed identification back to the operator 110. The same can be done for various pests and other information. Comparing the image with stored images and returning the results is indicated by block 420 in Figure 7. System 139 may compare the actual image and/or data with an expected image and/or data to identify anomalies, as indicated by block 421.

[00138] The recipient receiving the information may also return recommendations on how to proceed, as indicated by block 422. For example, the diagnostic/prognostic system 141 may perform an image- and/or data-based workflow, based in a comparison with expected data or based on other criteria. The workflow can generate output by automatically suggesting or performing control operations. For example, if the image is a pest, recommendations could include pesticides to apply. If the image is of a diseased plant, recommendations may include treatment options to treat the disease. These are just examples. The diagnostic/prognostic system 141 or other systems may return analysis results of various types, and this is indicated by block 424. Obviously, the systems may also return other information, and this is indicated by block 426.

[00139] The mobile device 104 and/or agricultural machine 103 receives the results and can perform a variety of different actions. Receipt of results is indicated by block 428 in Figure 7. Information may be displayed on device 104, on an alert monitor on agricultural machine 103, or in other ways as indicated by block 429. Mobile agricultural application 160 may simply store the results, as indicated by operator 110, as indicated by block 430. Again, the results may be stored locally, in several different locations, or in private cloud storage 128. The mobile device 104 and/or the display device 107 on the machine agricultural machine 103 may generate the superimposed or juxtaposed image (augmented reality) and superimpose or juxtapose the information received about the image being viewed as discussed above, as indicated by block 432. The mobile device 104 and/or control system on the agricultural machine 103 may perform automated control of machine 103 or other items, as indicated by block 433. Obviously, the mobile device 104 and/or agricultural machine 103 and/or other systems may also perform other operations, as indicated by block 434.

[00140] Figure 8 is a flowchart illustrating an example of the operation of the hosted agricultural data service 112 when receiving information from the mobile device 104. The service 112 first receives the captured data (the image and/or any other data) from the mobile device 104, as indicated by block 436 in Figure 8.

[00141] Service 112 then stores the data for later use or communication with other authorized users or systems, as indicated by block 438 in Figure 8. By way of example, the information may be stored for later access by operator 110. The image and/or data may also be stored for later access or provided for immediate analysis by an agronomist identified by operator 110, as indicated by block 440 in Figure 8. The image and/or data may be stored for access by other systems, such as system processed/derived data generation system 139, diagnostic/prognostic system 141, etc., for further processing and analysis to generate additional data, as indicated by block 443. The image and/or data may be stored or sent to a vendor, such as the seed company, the fertilizer company, etc., as indicated by block 442. The image and/or data may also be stored or sent to other services or data stores or entities, as indicated by block 444.

[00142] Figure 9A is a block diagram showing an example of a processed/derived data generation system 139 in more detail. In the example shown in Figure 9A, the data generation system 139 includes return on investment (ROI) generator 450, soil erosion data generator 452, animal presence/tracking data generator 454, pest/disease 456, path planning generator 458, section/field boundary data generator 460, anomaly identification system 500, seed/moisture contact data generator 501, trench structure data generator 502, composite agricultural score generator 503, residue data generator 504, intercrop data processing system 505 and other items 462. The ROI generator 450 may include seed count generator 464, chemical application metric generator 466, yield estimator 468, profit estimator 470, and other items 472. Path planning generator 458 may include compression data generator 474 (which itself may include wheel/track identifier 476 and mass identifier of vehicle 478), path generator 480, and other items 482. The section/field boundary data generator 460 may include boundary locator 484, overlap locator 486, and other items 488. The ROI generator 450 may generate a metric of ROI indicative of the financial productivity of a portion of a field. The portion may be one square foot, several square feet, a line, a section, or other geographic areas of a field. The ROI generator 450 illustratively identifies the location for which the ROI metric should be generated and then aggregates data corresponding to that location.

[00143] For example, location identifier 463 receives a location identifier (which identifies a geographic location under analysis) in a request from a mobile device or other display device requesting that ROI metric information corresponding to the location under analysis be sent for display. The seed count generator 464 accesses the locations of particular individual seeds within the area for which the ROI is being considered. In one example, the ROI of an entire field can be considered, and then the ROI metric for individual parts of that field can be broken down so that they can be displayed. The chemical application metric generator 466 obtains the discrete location of spray or other chemical application locations to identify the amount of chemical that was applied to the location for which the ROI generator is generating the ROI metric. Yield estimator 468 generates a yield estimate for that location. Yield estimator 468 may generate a yield estimate based on historical yield estimates or based on current plant development information or based on other criteria. Profit estimator 470 obtains cost information corresponding to seeds, corresponding chemicals applied at the location, corresponding fuel and labor costs, and any other information that is used in generating an ROI metric. The metric generator 471 takes the information generated or obtained from the seed count generator 464, chemical application metric generator 466, yield estimator 468, and profit estimator 470 and generates an ROI metric indicative of the ROI for the geographic location under analysis.

[00144] The soil erosion data generator 452 may receive an image of the geographic location under analysis and compare it to a previous image of the location under analysis to identify soil erosion. The soil erosion data generator 452 can generate a soil erosion metric indicative of the soil erosion that has occurred. The soil erosion metric can be a quantifier indicative of the amount of soil that has eroded, a severity level indicative of the severity of soil erosion that has occurred, or any of a wide variety of other metrics.

[00145] The animal presence/tracking data generator 454 can illustratively receive an image of the geographic location under analysis. The image may include one or more animal tracks that can be compared with a library of animal track images to identify a particular animal that made the tracks. Likewise, the pest/disease data generator 456 may receive an image of a pest or a diseased plant. The image can be compared by an image processor with a library of pests or diseased plants to identify the pest or diseased plant. The pest/disease plant data generator 456 may then generate output identifying the pest or disease and also identifying a recommended treatment (e.g., chemical application, etc.) to treat the pest or disease. Likewise, the identity of the pest or disease can be provided to the diagnostic/prognostic system 141 which can identify and produce a treatment for automatic or manual implementation.

[00146] The path planning generator 458 may generate a path that can be automatically downloaded to the agricultural machine 103 so that the agricultural machine 103 can automatically follow the path or the generated path can be displayed to the operator 110 so that the operator 110 can control the agricultural vehicle 103 to follow the path, or the path can be used in other ways. The path can be generated based on a wide variety of different criteria. In one example, the path may be generated to avoid undue compaction in certain areas or to avoid driving through crops or in muddy areas or to direct driving in other ways. Therefore, the wheel/track identifier 476 illustratively obtains the dimensions of the agricultural machine 103 to identify where the wheels and tracks are in relation to the position sensor on the mobile device 104 or other GPS or location sensor on the agricultural machine. 103. The wheel/track path identifier 476 then identifies the particular path that the wheels or tracks or other ground engagement elements on the agricultural machine 103 are taking as the machine 103 moves through the field. The vehicle mass identifier 478 identifies the mass of the agricultural machine 103 at different geographic locations in the field. The mass of the vehicle may be constant except for the change in the fuel level in the machine. Therefore, the vehicle mass identifier 478 can detect the fuel level in the machine and adjust the mass of the vehicle based on the fuel level. In other examples, where the agricultural machine 103 is a sprayer, as the material in the sprayer is dispersed, the mass of the machine decreases as the machine operates. Furthermore, if the agricultural machine 103 is a planting machine, as seeds are planted, the mass of the vehicle may also change. If the agricultural machine 103 is a harvester, then the mass of the machine 103 increases between unloading operations. Therefore, the vehicle mass identifier 478 can detect the application spray rate or the seeding rate or yield to identify how the vehicle mass is changing. The vehicle mass identifier 478 may also identify the mass of the vehicle in other ways.

[00147] Other criteria detector 482 can identify other criteria that can be used to generate a path. Based on the criteria detected by various criteria identifiers in the path planning generator 458, the path generator 480 generates a path output indicative of a path that the agricultural vehicle 103 took or a path that is recommended for the agricultural vehicle 103, or both. Additionally, path planning generator 458 may receive input from operator 110 indicating the criteria that operator 110 has specified for use in identifying the path. Criteria can be time, fuel efficiency, compaction criteria and/or other criteria. The path planning generator 458 generates output indicative of the path based on the desired criteria.

[00148] Section/field boundary data generator 450 may identify the boundaries of the field in which the agricultural machine 103 is traveling, the boundary of rows in the field, or the boundary of machine sections or other boundaries. The overlap locator 486 identifies when different sections or line units are operating to determine when there has been an overlap. For example, if a planter can be turned on and off in sections, it may be that during one pass one or more sections plant seeds that overlap seeds planted during a previous pass, so that the overlapping portion of the field is planted twice. This overlap can be identified by the overlap locator 486. The section/field boundary data generator 460 can also generate output indicative of other section or field boundaries.

[00149] Anomaly identification system 500 (which is described in more detail below in relation to Figure 9B) identifies expected data (such as an expected condition or state) based on a previous state and a previous condition and compares that condition or state expected with the actual detected data (an actual detected condition or state) to determine whether an anomaly exists. The anomaly can be any of a wide variety of different anomalies, such as a difference between a plant's expected state of development and its actual state of development; the state of a weed that has already been sprayed and the actual state of the weed; the difference between expected soil moisture and actual soil moisture; among a wide variety of other anomalies.

[00150] The seed-to-moisture contact data generator 501 generates data indicative of the quality of contact between the seed and moist soil. For example, the seed may have good seed-to-soil contact, but the soil may not be moist, in which case the seed-to-moisture metric may be relatively low compared to a scenario in which seed-to-soil contact is good and the soil has a desired moisture level. The seed-to-moisture contact metric can be used alone or can be used to generate other data such as estimated plant development, estimated yield, estimated emergence data, etc. To generate a seed-to-moisture metric, the seed-to-moisture contact data generator 501 may receive a sensor signal indicative of seed-to-soil contact (such as an image of the seed as it is placed in the furrow, information indicative of furrow quality, etc. other things), as well as a soil moisture metric indicative of soil moisture, and calculate the seed-to-moisture contact metric based on that information or other information.

[00151] Trench structure data generator 502 can receive data indicative of trench formation and generate a trench structure metric that indicates the quality of the trench structure. For example, a visual or optical sensor may generate a signal showing the ditch or furrow as it is generated. Sidewall quality, trench depth, trench shape, and other trench characteristics can be extracted from the image using image processing and used by the trench structure data generator 502 to generate a metric indicative of the quality of the trench. ditch.

[00152] The composite agricultural score generator 503 can take a wide variety of different types of information and generate a composite agricultural score based on that information. For example, a composite planting score can be generated as a function of the seed-to-moisture metric, the trench structure metric, one or more residue coverage metrics (described below), the temperature of the planted soil (which can be detected or estimated during planting), the quality of the as-planted fertilizer or pesticide application or other chemical application, a planter rocking metric indicative of whether the planter row unit is rocking during planting, a planter speed metric indicative of the planter speed during planting and/or any of a wide variety of other metrics. The composite agricultural score generator 503 may use a function that weights the various inputs to the function or other mechanisms to generate the composite agricultural score. The score can be georeferenced to a specific geographic location and used in an augmented display, or it can be pre-indexed or tagged in other ways.

[00153] The waste data generator 504 can receive signals from sensors indicative of waste on an agricultural surface and generate any of a wide variety of different types of waste data. For example, the residue data generator 504 may process an image showing residues to identify a residue coverage metric indicative of the percentage of the field that is covered by residues. The residue data generator 504 can generate a metric indicative of the uniformity with which the resident covers the field, different residue coverage metrics for different geographic locations, a metric indicative of how likely the residue is to interfere with crop emergence in different geographic locations, or a wide variety of other waste data. Waste data can be used to generate other information, can be used in an augmented reality display or in other ways.

[00154] Intercropping data processing system 505 can obtain and process information indicative of an agricultural field in which intercropping occurs. Intercropping can include a crop that is planted in a field that also has a cover crop planted (such as alfalfa planted in a field covered with wheat as a cover crop), fields that have multiple crops in alternating geographic areas (such as corn intercropped with soybeans, crops planted with a weed suppressing cover, crops planted together with a field crop such as radishes, etc., or other intercropped scenarios. The intercropped data processing system 505 can generate metrics indicative of the effectiveness of a cover crop, the effectiveness of a tillage crop, the effectiveness of the weed suppression crop, soil attributes based on live sensor information from the cover crop or other crop, various productivity characteristics (such as yield) for the various crops and a wide variety of other information. Intercrop data can be used to generate additional information, can be used in an augmented reality display, or can be used in other ways as well.

[00155] Figure 9B shows a block diagram of an example anomaly identification system 500 in more detail. Figure 9B shows that the anomaly identification system 500 may include previous state identification system 506, previous condition identification system 508, expected object/condition identification system 509, comparison system 507, anomaly identifier 510, anomaly output system 511 and other items 512. The anomaly identification system 500 first identifies an expected object or condition based on a previous state and/or a previous condition and compares the expected object or condition with an object or condition or current state to determine if an anomaly exists. An anomaly is a difference between what is expected (based on prior information) and what is actually occurring. The difference can be quantified and compared to a threshold to determine whether the anomaly exists, the status or severity of the anomaly, etc. The anomaly can be output along with any other desired data (such as the magnitude of the anomaly, etc.). ) for further processing, to provide automated control, to present recommendations to one or more users, or for other reasons. The anomaly can also be georeferenced to a particular geographic location in the agricultural field, or indexed or referenced to other data.

[00156] More specifically, the previous state identification system 506 identifies a previous state about which an anomaly should be identified. The previous state may be indicative of a previous state of a geographic location of a field, a previous state of a plant (such as a weed or crop plant), a previous state of the soil (such as a previous moisture content), a previous state level of compaction, a previous slope or elevation, a previous nutrient level, or other previous state. The previous condition identification system 508 can identify a previous condition, such as previous weather conditions, previous characteristics or conditions corresponding to a plant or field, previous location of weeds, rocks, obstacles, etc., or any other previous condition. The expected state/object/condition identification system 509 generates an expected state, object, and/or condition. The expected state, object, or condition may be obtained from another data store or may be derived using a model, a lookup table, a machine learning system, a rules-based system, an artificial intelligence system, a neural network, Bayesian classifier, etc., based on previous information from systems 506 and 508, or can also be derived in combination with additional information.

[00157] For example, a previous state can be identified as a previous developmental state of a particular plant. A previous condition may be indicative if this has occurred since the previous developmental state of the plant was identified. The expected state generated by the expected state identification system 509 may be a currently expected developmental state of the same plant generated by a plant developmental model, given its prior state and the intervening prior climate conditions. The expected state, object or condition may be the expected state of a weed when its previous state of development is known and the previous condition indicates that it was sprayed with an herbicide. The expected state, object or condition may be the erosion state of a geographic location of a field generated by a soil erosion model, given the previous state of the terrain at that location and given previous climatic conditions such as previous precipitation, etc. . These are just a few examples of an expected state, object or condition and how they can be generated.

[00158] The comparison system 507 receives information from the expected state/object/condition identification system 509 that identifies the expected state, object and/or condition. The comparison system 507 also receives data indicative of a current state, object and/or condition. The current state, object and/or condition may be represented by a current sensor signal, by currently derived data, or may be obtained in another way. By way of example, where the state output expected by system 509 is the expected developmental state of a plant, then the current state of the plant may be a sensor signal indicative of the current developmental state of the plant (such as a state of plant development identified by an image processor based on a current image of the plant, etc.). The comparison system 507 compares the two data items (the expected state and the current state) and generates output indicative of the difference. Anomaly identifier 510 receives comparison output from system 507 and identifies an anomaly based on that information. The anomaly identifier 510 then generates output indicating the type of anomaly. For example, in the example where the expected and actual state is the plant development state, the anomaly identifier may indicate that a plant is less developed than expected. In an example where the expected and current state are the presence or absence of an obstacle, the anomaly identifier may identify the anomaly as one in which an obstacle (such as a rock or other obstacle) exists where none was expected. If the expected state and the actual state are the erosion state, the anomaly identifier can identify an anomaly as one in which a ditch (or other erosion effect) exists where it was not expected. In an example where the expected and current state are the state of a weed, the anomaly identifier may identify an anomaly as one in which a weed exists where it has already been sprayed. These are examples of the types of outputs that can be generated by the anomaly identifier 510.

[00159] The anomaly output system 511 then generates an output indicative of the identified anomaly, a severity level of the anomaly, etc. The output may be used to display or otherwise present the anomaly to a user, to generate control signals to perform automated control based on the anomaly, or to display the anomaly in a display as part of an augmented reality display, or in a wide variety of other ways.

[00160] When identifying anomalies, the previous state and previous action can be used to generate predictive information indicative of anomalies (such as unexpected objects or conditions) that can be used for visual augmentation. The predictive information used to generate the anomalies may be relative to images or data from a current time or relative to data, including images, collected at a passing time. In the latter case, there may be a predictive image or map layer of the past and an actual image of the present. These two sets of data can be analyzed to identify objects (such as weeds or other objects) or other features such as crop development or other features such as anomalies where the predictive and actual data differ by a threshold value. Anomalies can be highlighted visually and the workflow can seek further description, diagnosis and/or prognosis of the condition.

[00161] Where anomalies exist, different workflows can be initiated depending on the specific anomaly. For example, a diagnostic-related workflow can be triggered when an anomaly is identified that indicates that weeds are present where a spring operation has already occurred. In this example, a workflow might be related to a diagnosis that weeds are resistant to glyphosate. In another example, the workflow might perform a diagnostic to identify that weeds are not being treated due to a faulty spray nozzle. These are just examples.

[00162] Figure 9C is a block diagram showing an example intercrop data processing system 505 in more detail. System 505 may include plant identification system 513, per-plant metric calculation system 514, visual overlay generation system 515, visual representation identifier 516, sustainability metric generator (e.g., nitrogen/carbon/other) 517 , living sensor analysis system 518 , crop tillage effect metric generator 519 , other intercrop data processing functionality 520 .

[00163] The plant identification system 513 illustratively processes video images or receives the result of such processing and identifies plants in the different crops in an intercropped field. System 513 can compare the characteristics of crops identified in an image with those of different intercrop plants planted in the field to identify which are which or identify the plants in other ways. The per-plant metrics calculation system 514 can illustratively perform processing to identify per-plant metrics for each of the plants in the different intercropping systems. Per-plant metrics can be things like yield per plant, estimated forage biomass, etc. A wide variety of other per-plant metrics can also be calculated.

[00164] The sustainability metric generator 517 can also receive information indicative of intercrop varieties and generate a variety of different sustainability metrics, such as an estimate of carbon credits, species carbon credits, nitrogen credits, among others. things.

[00165] The live sensor analysis system 518 analyzes live sensors and generates output indicative of this analysis. For example, plants may have different colors based on their vigor, based on the various nutrients that are accessible to the plant or nutrients that are not available to the plant, soil pH level, based on soil moisture, or based on a wide variety of other criteria.

[00166] Visual representation identifier 516 identifies a specific visual representation for each of the different types of crops in the interleaved crop field. Therefore, the visual representation can be superimposed over an augmented reality display showing where different cultures will emerge, showing different populations of cultures, or showing different properties or characteristics corresponding to the cultures. The visual overlay generation system 515 may generate an overlay indicative of any of the items or metrics calculated by the various systems and items in the interleaved crop data processing system 505, over the field as an augmented reality display.

[00167] The crop tillage effect metric generator 519 generates a crop tillage effect metric that indicates a tillage effect that some crops may have had on the soil. For example, radishes or other similar plants may be planted to carry out a farming operation. A metric indicative of the effectiveness of the tillage operation may be generated by the crop tillage effect metric generator 519.

[00168] A wide variety of interleaved culture data can be generated by other interleaved culture data processing functionality 520 as well.

[00169] In one example, different tile or plant icon attributes may be used to designate different varieties or species of plants in the displayed image. In other examples, increased display may depend more strongly on plant species or function. For example, if corn and soybeans are intercropped, the augmented display might annotate corn plants with estimated yield per plant, while annotating grains with acreage yield in two-meter segments (or other segments). In another example, where corn is intercropped with a cover crop, then the corn can be scored for estimated yield per plant and the cover crop can be scored for estimated forage biomass at a present time or a future time. , as after corn harvest. Other cover crop data can include things like grain yield; biomass; nitrogen credit for legumes; carbon credit for species, vigor or living roots; effect of crop cultivation such as radish vigor and size; weed suppression effectiveness; soil attributes based on live sensor analysis; and other information.

[00170] Additionally, precision intercropped multicrop planters can collect location information for each seed of each species. When performing the analysis, the intercrop data processing system 505 may generate the overlays, thresholds (such as planting depth error discussed above), formulas (such as crop stage as a function of planting and growing degree days), exclusively in relation to each type of plant. Different cultures can have different overlaps.

[00171] Likewise, a single culture from a mixture of interspersed cultures can be augmented or represented. For example, where corn is intercropped with soybeans or cover crops (such as rye, vetch, radish, etc.), the other crops can be subtracted from the augmented display to provide greater clarity of the corn data. Subtraction may be performed by the intercrop data processing system 505 by subtracting the plant data in the image and replacing it with an artificial background (such as black pixels), a corresponding real background (such as an image of the ground surface for the field collected at the time of planting), a pseudo-realistic background (which can be real pixels or synthesized for the local soil type), among others. Additionally, the optional subtraction and substitution discussed above can be performed on monocropped images to highlight individual lines, groups of plants, individual plants, weeds, etc.

[00172] It will also be noted that augmented displays may be related to sustainability parameters such as net carbon footprint, crop input carbon footprint, gross carbon sequestration, or other sustainability parameters. The crop's input carbon footprint can be the per-plant sum of carbon emissions associated with seed, applied fertilizer, equipment operations, and the like. Carbon sequestration per plant can be estimated from crop and soil attributes, current growing environment, future growing environment, growing history, etc. The net carbon footprint can be the difference between the carbon footprint of crop input and carbon sequestration.

[00173] Likewise, these values can be calculated based on the area, where different crops have different levels of carbon footprint and carbon sequestration. In other examples, instead of a per-plant or per-area calculation, environmental impact values may be calculated based on bushels, tons, or other metric of material removed. The impact of livestock grazing can be considered, such as manure, biomass consumption/removal, carbon storage impacts on the root system, and other information.

[00174] Figure 10 shows a block diagram of an example of a display device 107. The display device 107 may include alert monitor generator 490, geographic correction system 491, mobile device display generator 492, data projection device 494 and any of a wide variety of other display devices 496. Alert monitor generator 490 can generate an alert monitor on wearable vision equipment (such as glasses) on the front windshield of the agricultural machine , on the front or side windows of the agricultural machine and/or on the rear windows of the agricultural machine.

[00175] Figure 10 is a block diagram showing an example display device 107 (which may be similar to or different from the display generator 159 that may generate a display on the mobile device 104) in more detail. The display device 107 includes alert monitor generator 490, geographic correction system 491, mobile device display generator 492, data projection generator 494, and other items 496. The alert monitor generator 490 may be used in the compartment of the operator of the agricultural machine 103 to generate an alert monitor. The alert monitor may be generated on the windshield of the agricultural machine 103, on one or both side windows and/or on the rear window or other surfaces. For example, the alert monitor may generate a display on the front windshield of agricultural machine 103 displaying alphanumeric, color-coded, bar graph, or other metrics overlaid at different geographic locations in front of the machine 103 to show the different metrics or georeferenced graphics to those locations. Some examples will be described in more detail below in relation to Figures 15-24. In another example, the alert monitor generator 490 may generate an alert monitor in the rear window of a tractor or other towing vehicle that is towing an implement, such as a planter with row units. The alert monitor can overlay, over each row unit, various information corresponding to that row unit, such as seeding rate, downforce, travel quality, or any of several other metrics for the specific row unit. In this way, when operator 110 turns to view the towed implement, operator 110 can easily see a wide variety of different information corresponding to each of the queue units on the implement. The same can be done for other implements such as tillage implements, spraying implements, harvesting implements or other implements. The alert monitor generator 490 may also be used to generate an alert monitor on a wearable display device, such as smart glasses or a head-mounted optical display engine, or in other ways.

[00176] The geographic correction system 491 may be used to correct the geographic position signal indicative of the geographic location of the machine 103 or display device 107. Such correction may be performed to more accurately display the georeferenced data. Some examples of geocorrection are described elsewhere in this document.

[00177] The mobile device display generator 492 may be a display generator on the mobile device 104, where the mobile device 104 is mounted or resides in the operator compartment of the agricultural machine 103. For example, the mobile device display generator 492 may be one or more of the components or items described above in relation to the mobile device 104 shown in Figure 1B, or other display generators.

[00178] The data projection generator 494 illustratively projects visual cues onto the field or other surface over which the agricultural machine 103 is traveling. For example, the data projection generator 494 may be laser units mounted in or near the headlights of the agricultural machine 103. The projection may be a color projection, an alphanumeric projection, a graphical projection, or other projection that is indicative of georeferenced metrics. corresponding to the geographic location over which the projection is displayed.

[00179] Figure 11 is a block diagram, similar to Figure 4, except showing additional data sets 498 that may be retrieved and/or generated and correlated to a geographic location and stored in the geotagged pre-indexed data store 138, or elsewhere. In an example, additional data sets 498 may include georeferenced planting data 550. The georeferenced planting data 550 may include specific seed location data 552, seed singulation data 554, seed population/spacing/depth data 521, planter swing data 522, planter speed data 523, obstacle encountered data 556 and other data 558. Specific seed location data 552 can be generated based on the planting machine data. Specific seed location data 552 may indicate the locations of individual seeds detected during planting and transmitted back to the hosted agricultural data service 112 where they may be stored in the geotagged pre-indexed data store 138 and/or used by the data generation system 139 or other items to generate additional data.

[00180] Seed population/spacing/depth data 521 may indicate the seed population over a given geographic area of a field, the seed spacing over that area or a different area, and seed depth data indicating the depth in that a seed in a particular geographic location has been planted. Data 521 may be detected from a planter during the planting operation, may be estimated before or after the planting operation, or may be obtained in other ways.

[00181] Planter rocking data 522 may be indicative of the quality of travel of a planter row unit or other data indicative of whether a planter was rocking excessively (such as leaving the ground or compromising the desired planting depth, etc. .) during the planting operation. Planter swing data may be detected by a sensor 111 which may be an accelerometer, inertial measurement unit, or other sensor that may detect the quality of displacement of a row unit or other planter mechanism.

[00182] Planter speed data 523 can be detected during the planting operation from a ground speed sensor that can be incorporated into one of the sensors 111 on the agricultural machine 103. The speed sensor can detect the speed of displacement of machine 103 or a planting mechanism in machine 103. The speed sensor may also detect the speed of a seed distribution system, a seed dosing system, or the speed of other parts of a planter or planting implement.

[00183] Seed segmentation data 554 may be indicative of seed skips or duplications or other segmentation events that are detected in seeding machines during the seeding operation. The singulation data 554 can identify when a singulation event occurred and the precise geographic location of that singulation event.

[00184] Obstacle encountered data 556 may indicate the geographic location where machine 103 encountered an obstacle. The obstacle may be detected using a mechanical deflection sensor, a torque sensor, an accelerometer or other sensor indicating that the planter or row unit or other machine or implement has encountered an obstacle. For example, some planting mechanisms have a rod or other item that opens a furrow. When the rod encounters a rock or other obstacle, this can impart force to the rod that can be detected by a sensor. The geographic location of that encounter, along with an indicator indicating what the encounter was (such as hitting a rock or other obstacle, etc.) may also be sent to the hosted agricultural data service 112 of the agricultural machine 103.

[00185] Additional data sets 498 may also include processed/derived data 560. Data 560 may include ROI by location data 562, path planning data 564, section/field boundary data 566, compression data 568, soil erosion data 570, animal presence/tracking data 572, pest/disease data 574, seed moisture contact data 529, trench structure data 530, or other processed or derived data 576. The generation of each of data items 562-574 was described above in relation to Figure 9A. The geographic location of each of the metrics generated and reflected in the data 562-574 may be sent from the machine 103 to the hosted agricultural data service 112 for storage in the data store 138 or for further processing or for storage elsewhere.

[00186] Additional data sets 498 may also include sprayer data 578. In one example, a sprayer has components that can recognize weeds and spray individual weed locations or other plant locations or pest locations that are detected by the sprayer. during runtime. Thus, spray data 578 may include discrete spray locations and rates 580 that identify individual locations where chemicals were sprayed, the content of the chemical, the rate at which the chemical was sprayed (or the amount sprayed at each local) and/or a wide variety of other data 582.

[00187] Additional data sets 498 may include harvester data 584 which may include yield, crop moisture, crop quality (such as whether the harvested material is clean, cracked, etc.) and any of a wide variety of other data of combine 584. Data from combine 584 may be correlated with one or more geographic locations and sent from the combine to hosted agricultural data service 112 for storage and/or further processing.

[00188] Additional data sets 498 may include tillage data 586 which may include tillage depth, various soil characteristics detected during tillage, contour data reflecting the contour of the field after tillage, and a wide variety of other data of farming. The data sets 498 may include diagnostic/prognostic data 587 generated or used by the diagnostic/prognostic system 141.

[00189] Additional data sets 498 may also include previous operation maps 701. Such maps may include coverage maps 703 that indicate the coverage of a particular agricultural operation over a field. The applied maps 705 may be material maps such as chemicals, herbicides, pesticides, fertilizers, etc., as they were applied. As-applied maps may indicate the geographic location where the material was applied, the amount applied, the rate at which it was applied, among other things. The wayfinding maps 709 may map items that are of particular importance to the operator or other user. For example, signage maps may be maps of the location of rocks or other obstacles, drainage duct entrances, or other locations that should be marked for special attention. Previous operation maps 701 may include other maps 711 that are also generated during previous cross-field operations.

[00190] Additional data sets 498 may also include intercrop data 433 which may show maps of where multiple crops were planted in a field, estimated yields for individual crops, nitrogen and/or carbon credits, and/or other chemicals for individual crops, percentage plant yield or other characteristics per plant for the intercrop field, tillage effect, forage biomass or other information for the intercrop field, or any of a wide variety of other data for a field of intercropped crops.

[00191] Additional data sets 498 may also include a wide variety of other data 588. The other data 588 may include machine level data 589 (which may be such as tire pressure, machine mass or weight, level(s) ) of tank filling, etc.), other locations and rates of chemical application (or material applied/or operation) 590, waste coverage percentage 592 which indicates the percentage of a given geographic area that is covered by waste and soil temperature data 594. Other data 588 may also include residue/seed impact, sprout blockage data 531. Data 531 may be a metric or other indication of how residue in a particular geographic location may affect development of the seed. For example, if the residue is heavy, it may affect soil temperature or soil moisture, may physically block sprout or plant emergence, or may affect seed development in other ways. The data 531 may be data geographically referenced to a field and may include one or more different indicators that indicate the impact of the residue on a seed. Other data 588 may also include one or more composite agricultural scores 532. Some examples of how to generate composite agricultural scores 532 are discussed elsewhere in this document and any of a wide variety of other georeferenced data 596.

[00192] Figure 12 is a flowchart illustrating an example of the operation of the agricultural system 100 in generating geotagged data, receiving a request for data or other data of interest from the mobile device 104 or agricultural machine 103 or other requesting device and providing data responsive, which responds to the request or which is based on the data received. It is first assumed that the hosted agricultural data service 112 obtains data of interest to the service, as indicated by block 620 in the flowchart of Figure 12. The data of interest may be obtained, as discussed elsewhere, from a wide variety of sources. For example, data may be obtained from sensors 111 on agricultural machine 103, which may be a planter, a sprayer, a harvester, a tillage implement or other farming equipment, or a wide variety of other machines, as indicated by block 622 The data may be obtained from other sources, such as weather sources or vendors, or other data sources or services, as indicated by block 624.

[00193] The hosted agricultural data service 112 may then perform any desired processing on the data, as indicated by block 626. The processing may be performed by the processed/derived data generation system 139 or diagnostic/prognostic system 141 or other systems or components. For example, the data may be processed by the anomaly identification system 500 of the ROI generator 450, soil erosion data generator 452, animal presence/tracking data generator 454, pest/disease data generator 456, seed/moisture contact data generator 501, trench structure data generator 502, agricultural composite score generator 503, residue data generator 504, intercrop data processing system 505, path planning generator 458 or section/field boundary data generator 460 or other data generator 462. The data may be processed to obtain section/field boundary data 566 or compression data 568. The data may be animal presence/tracking data 572, soil erosion data 570, seed-to-soil contact data 529, or pest/disease data 574. The data may be processed by a wide variety of other components and may include other data 638, some of which are shown above in Figures 4 and 11 and/or elsewhere here.

[00194] The data generation system 139 then pre-indexes and geotags the data and stores it in data storage 138, as indicated by block 640 in the flowchart of Figure 12.

[00195] At some point, the hosted agricultural data service 112 will receive a request for data from a requesting device, such as mobile device 104, display device 107, another component of agricultural machinery 103, etc. The data request may be a request for data that can be displayed, or may include an image that must be identified or otherwise processed, where the processed data must be sent back to the requesting device. Receipt of a data request is indicated by block 642 in the flowchart of Figure 12.

[00196] Application server 136 receives the request and determines whether the requested data is already in the pre-indexed and geotagged data store 138, as indicated by block 644. If so, the data is retrieved, as indicated by block 646, so that they can be formatted and sent to the requesting device. If the requested data is not already in the data store 138 but needs to be obtained from another source, the runtime search component 132 generates a search request and performs a runtime search to obtain the data from another source. . The processed/derived data generation system 139 may also perform additional processing to obtain the requested data. Additionally, the diagnostic/prognostic system 141 may perform other processing and also generate diagnostic and/or prognostic data. Performing a run-time search or other processing or interaction to obtain the requested data is indicated by block 648 in the flowchart of Figure 12. Application server 136 receives the responsive data (search results or processed or derived data) as indicated by block 650 and formats this data for rendering on the requesting device, as indicated by block 652. The application server 136 then sends the data to the requesting device, as indicated by block 654. The requesting device may be a mobile device 104, an in-cab display device 107 or any of a wide variety of other devices 656.

[00197] Figure 13 is a flowchart illustrating an example of how the requesting device processes data, requested by an operator and received from application server 136, or in another example, obtained from a local data store that is local to the requesting device. First, it is assumed that the requesting device detects operator input requesting the display of information, as indicated by block 658 in the flowchart of Figure 13. The input may be a touch gesture, a menu-driven input, or input. speech, or other input on a device, requesting the display of data. In one example, the requested data is geotagged data so that it can be displayed over terrain for viewing by an operator. The requesting device (which may be display device 107, mobile device 104, etc.) or other device 659 then receives the geotagged data as indicated by block 660. The geotagged data may be obtained from a local data store as indicated. by block 662, or from a remote storage, such as from the hosted agricultural service 112, as indicated by block 664, or otherwise, as indicated by block 666.

[00198] In one example, the display generator uses geographic correction system 491 (in display device 107) or position correction system 167 (in mobile device 104) to correct geographic information that identifies the location of the display device. exhibition. For example, when the display device is a mobile device 104, then the geographic location of that device can be identified using a relatively imprecise GPS receiver or other location system. This inaccurate localization can be corrected by using other higher accuracy data, such as RTK data. Correction of geographic location signals is indicated by block 668 in the flowchart of Figure 13 and the use of RTK data is indicated by block 669. Higher precision location data may be, for example, based on line or seed location where this location was generated during planting or at another time by a higher precision positioning system. Correction of the display device's geographic location signal based on higher precision line/seed location information is indicated by block 670 in the flowchart of Figure 13. The correction may be based on a known location of a marker that is already in the field as indicated by block 670. The correction may be based on a user correction or alignment inputs as indicated by block 673. For example, a field boundary may be overlaid on a display of a field boundary. However, the superimposed boundary may be misaligned with the actual field boundary. The operator can then drag the overlapping boundary so that it aligns in the display with the field's actual boundary. Based on the realignment input by the operator, the correction system 491 or 167 can determine the magnitude and direction of the misalignment and use this information to also correct the location of other displayed items. Correction may be based on other known location information, as indicated by block 674. Additionally, position correction may be performed using phytoreferencing where, during planting, a controller controls a device to vary an attribute of a plurality of individual plants (such as plant spacing, plant location - e.g., a distance by which seeds are displaced laterally from a row, etc.) so as to form a pattern of location correction information or other information.

[00199] The requesting device then generates a display of the geotagged data as indicated by block 676. The alert monitor generator 490 may generate an alert monitor in a window or glasses as indicated by block 678. The generator data projection device 494 may generate a laser projected image (or otherwise generated projection) onto the real field or other projection as indicated by block 680. Mobile device display generator 492 may generate an overlay on a camera viewer of mobile device 104, as indicated by block 682. The geotagged data may also be displayed in other ways, as indicated by block 684.

[00200] Figures 14A and 14B show a flowchart that illustrates, in more detail, different ways in which geotagged or georeferenced data can be displayed. The various data and display items shown in Figures 14A and 14B may be similar to that shown in Figures 4 and 11 above or different. And numbered differently just as an example. The display of geotagged data is generally indicated by block 686 in the flowchart of Figure 14A. The displayed data may be 688 planting data. For example, the individual seed location may be displayed in a georeferenced form so that as the operator is viewing the data through some type of viewer (data monitor device) alert, camera view locator device, etc.), the locations of individual seeds are marked on the terrain that the operator is viewing. The display of the individual seed location is indicated by block 690.

[00201] Instead of the individual seed location, or in addition to that location, singulation data can be displayed indicating the locations of duplicates or skips that were found during the seeding process. The display of singulation data is indicated by block 692.

[00202] Obstacle location data may be displayed to indicate the geographic location where an obstacle was encountered by the planter, as indicated by block 694. Other planting data may also be displayed in a georeferenced form.

[00203] Planting data 688 may also include machine level data 733, chemical application locations and rates 735, residue/seed impact, shoot blocking information 737, and composite crop score 739. Some examples of these are described in more detail above in relation to Figure 11.

[00204] In another example, viewing and spraying vehicles identify weeds or pests or diseases or other indications that indicate that a chemical should be sprayed as the vehicle is traveling through the field and then sprays the weeds, pests, diseases or other identified locations. The specific location where the chemical was sprayed or otherwise administered (the discrete see and spray locations) is recorded and the discrete see and spray locations may be displayed in a georeferenced manner, as indicated by block 696.

[00205] In another example, the georeferenced data that is displayed in a georeferenced manner may be processed or derived data generated by the processed/derived data generation system 139, as indicated by block 698 in the flowchart of Figure 14A. The processed data may be ROI data 700, path data 702, boundary/planting overlay data 704, compression data 706, or a wide variety of other processed or derived data 698. The process/drive data 698 may also be as shown in Figure 14B, include data indicative of soil erosion 723, data indicative of animal presence or tracking 725, data indicative of pests or diseases 727, data indicative of seed moisture contact 729, and data indicative of trench structure 731 In another example, the georeferenced data displayed may be crop data 708.

[00206] The georeferenced data that can be displayed in georeferenced form may be any of a wide variety of other data 710, such as waste coverage data 712, soil temperature data 714, or other data 716. The other data 710 may include 733 machine-level data, 735 chemical application locations and rates, residue/seed impact, 737 shoot blockage data, and 739 composite crop scores.

[00207] In some examples, the display may be generated interactively so that the operator can interact with the display. For example, the display can be generated with user-actionable mechanisms, such as bindings, buttons, icons, or other mechanisms. These mechanisms can be triggered by the operator to interact with the display. Detection of operator interaction with the display is indicated by block 718 in the flowchart of Figure 14A. Some interactions include a tap and swipe input 141 in which the operator slides along a displayed crop line. The interaction may include a coarse/low granularity interaction 743 in which the operator provides a decreasing granularity interaction to show additional detailed information or an increasing granularity interaction to show less detailed (more general) information. Figure 14A, operator interactions also include a zoom in/out interaction 720 to zoom in or out of the information view, respectively. Another interaction may be a panoramic interaction 722 in which the view may be panned from side to side or up and down. In yet another example, the interaction might be to display a menu or other selector. The operator can trigger one of the input mechanisms to toggle the requested data geotagged and displayed in the geotagged view. For example, the operator may choose to switch from displaying the seed's georeferenced location to displaying the georeferenced downforce. The operator can choose to aggregate the display to show downforce and seed location or toggle the requested data in other ways. Detection of an operator interaction to switch the requested data is indicated by block 724 in Figure 14A. Other interactions may also be detected, as indicated by block 726.

[00208] The display is then processed by the display generator based on detected operator interaction, as indicated by block 728. By way of example, the display generator may switch the display to show a display based on a swipe input , information with high/low granularity, an enlarged or reduced version, a panoramic version, different data, etc.

[00209] Figures 15-24 show different examples of georeferenced augmented (or enhanced) reality displays that can be generated for an operator on a display device, such as on a mobile device 104 or other display device 107. Figure 15 shows an enhanced display 730 that shows terrain 732 with a plurality of different grid segments 734. Each grid segment 734 may be color-coded or shaded, or otherwise visually distinguishable from other grid segments 734 to identify a particular metric represented by the grid section. Figure 15 also shows a set of actuators shown generally at 731 that can be activated by the user to choose the metric that will be represented by the various grid segments 734. For example, in Figure 15 the actuator labeled “singulation” was activated so that the grid sections represent singulation data indicating whether jumps, doubles, or singles were planted in those grid segments. The other 731 actuators include “downforce”, “soil temperature”, “days to emergence” and “yield”. The actuators 731 shown in Figure 15 are examples only, and other metrics or data may be represented by the actuators so that when an actuator is actuated, the corresponding data may be displayed as augmented reality data on display 730.

[00210] In the example shown in Figure 15, display 730 also shows a plurality of individual seed locations 736. Segments 734 represent singulation data. Therefore, display 730 is displaying a singularization metric as well as the locations of individual seeds in the terrain 732 being shown in the display. Different tones or other visual cues of segment 734 may indicate whether singulation events have occurred and whether these events are skips, doubles, etc. Of course, singulation data is just one example of the metric that can be geotagged and displayed, and many others can be displayed as well.

[00211] Figure 16 is an enhanced display similar to Figure 15 and similar items are numbered in a similar manner. In Figure 16, the actuator 731 that was selected by the operator is the “yield” actuator so that the grid segments 734 shown in Figure 16 represent yield values corresponding to those grid segments. Thus, display 738 shows that the augmented reality blocks represent a per-plant metric in which the per-plant metric is a per-plant yield metric. Unlike Figure 15, in the example shown in Figure 16, grid segments 734 are displayed in relation to the already emerged plants that are shown in display 738. Thus, the grid segments can represent the expected yield at the locations of the grid segments. , soil temperature, downforce, planting depth or metrics or different combinations of different metrics.

[00212] Figure 17 shows an example of an enhanced display that is similar to Figure 16 and similar items are numbered in a similar manner. In Figure 17, the user activated the “yield” actuator, but also provided a drill-down input requesting that more detailed information be shown so that data display 740 is shown to display the values of all metrics (singulation, downforce , soil temperature, emergence days, and yield) corresponding to a selected grid section 742. Therefore, each grid section 734 is visually used to display the yield metric corresponding to that grid section, and the data display 740 shows values for a specific selected grid section 742. Additionally, Figure 17 shows that the view has been enlarged.

[00213] Figure 18 is an enhanced display similar to Figure 17 and similar items are numbered in a similar manner. However, in FIG. 18, the grid segment 744 has been selected by the operator (through a user interaction such as tapping the grid segment 744, clicking it, or other interactions) so that the values in the data display 740 are updated. to show the values corresponding to grid segment 744.

[00214] Figure 19 shows another enhanced display 750 that displays georeferenced information in a different manner. The enhanced display 750 displays the locations of individual seeds 752, as well as an information display 754 that shows planting information such as planted data, temperature when planted, soil moisture level, planting depth, and seed composition. fertilizer that was used during planting. Figure 20 is an enhanced view similar to Figure 19, except that Figure 20 shows a wider (zoomed) view and the data display 754 has been removed.

[00215] Figure 21A shows a projected yield per plant in the bottom line of Figure 21A. Yield per plant is an example of a metric, and other metrics per plant can also be shown. Yield per plant is based on the planting parameters illustrated in the top rows in Figure 21A. Figure 21B shows an enhanced display 760 in which individual rows of seeds 762 are shown and all rows are indicated by alignment projections 764. Graphical display elements 766 are displayed in an augmented reality display, on top of the seeds in the rows. . The graphical display elements 766 visually identify the projected yield per plant for the seeds over which they are displayed. Projected yield can be based on a wide variety of criteria such as seed-to-moisture contact, moisture availability, trench structure (which may include seed-to-soil contact, compaction, etc.), residue coverage that can affect soil temperature or shoot blocking, the quality of the planted fertilizer and pesticide application, planter balance, planter speed or other items.

[00216] As can be seen, the height and/or color of the graphical display element 766 may be proportional to a plant metric, such as plant height, plant vigor (e.g., an NDVI value corresponding to the plant), estimated yield per plant, etc.

[00217] Figure 22 is another example of an enhanced display 768, which is similar to the enhanced display 760 shown in Figure 21B, and similar items are numbered similarly. However, the enhanced display 768 has additional augmented reality display elements in addition to those shown in Figure 21B. For example, each of the display elements 766 has an associated lowercase letter a-j. Display elements 766 may also have corresponding circles 770. The circles may represent yield per plant or other data, such as a composite planting score. Differences in values represented by display element 766 and circles 770 may be represented visually using shape, color, size, pattern, flashing cues, intensity, or other visual cues. Figure 22 also shows that some of the display elements 766 have graph display elements (e.g., bar graph) 772 that can show a plurality of different parameters based on the height and color of the bars in each of the graphs 772. Other visual attributes, such as those listed above or others, may be used in display elements 772 to represent different values. Furthermore, display elements 772 may, in addition to bar charts, be line graphs, pie charts, spider diagrams, tables, text, and other display elements. The increase can be shown in three dimensions or two dimensions or another way. Some information may also be communicated using speech synthesis or other elements.

[00218] It should be noted that increasing or enhancing the display can also be sequenced in several different ways. For example, the display engine may sequence the different augmentations automatically, or through menu interaction, or through speech interaction, or in other ways. Additionally, in one example, only some of the augmented display elements are sequenced, while others remain constant. For example, the example shown in Figure 22, all display elements may remain constant, except display elements 772, which may be replaced with other display elements representing other variables, according to a desired sequence. The sequence through which display elements are displayed can be a predefined sequence or a user-selected sequence.

[00219] Additionally, the agricultural machine 103 and/or mobile device 104 or hosted agricultural data service 112 may have speech processing functionality so that speech inputs may be received from a user or operator and used to control the system agricultural 100. The speech processing functionality may include speech recognition functionality and speech synthesis functionality so that voice commands can be received and processed and the processing results can be output using spoken language.

[00220] As an example of speech interaction with a system, it is assumed that mobile device 104 has speech processing functionality. Figure 23 shows an enhanced display 774 in which individual seed locations are shown and line and passage numbers corresponding to individual seed locations are also shown. For example, in rows 776, 778, and 780 in Figure 23, the seeds in each of these rows were planted during a specific pass (pass 1 represented by P1) with a 12-row planter. Therefore, lines 776, 778 and 780 are represented by P1, R12; P1, R11; and P1, R10, respectively. Line 776 is therefore the 12th line in pass 1 for a planter. Line 778 is the 11th line in passage 1 and line 780 is the 10th line in passage 1. An example speech interaction might be as follows (assuming the word “Jonny” is activation work for the processing functionality so that when the speech processing functionality hears an operator pronounce that word, the speech processing functionality starts listening for speech commands that it should execute. Thus, the speech interaction might be as follows:

[00221] Operator: “Jonny, show the predicted yield per plant for pass 1, line 12.”

[00222] The speech recognition system identifies the command and generates output identifying a specific display that should be displayed on the mobile device. The display may correspond to the enhanced display 760 shown in Figure 21B. Then the operator can say “Jonny, show the composite planting score for each plant in pass 1, line 12”. In this example, the speech recognition system recognizes the command to show the composite planting score and increases the display shown in Figure 21B to that shown in Figure 22, for plants labeled “a”. In another example, the composite planting score may be displayed elsewhere in display 774.

[00223] Likewise, the user can say “Jonny, divide the composite planting score for pass 1, line 12.”. The speech recognition functionality then recognizes the command to break the composite planting score into its individual elements and display a bar graph similar to the graphs 772 shown in Figure 22, adjacent to the seeds displayed in passage 1, line 12 in display 774 ( in Figure 23). Each bar in the bar chart can represent the value of a variable used to generate the composite score.

[00224] In another example, mobile applications are touch-enabled to recognize touch gestures. As an example, the user can hover their finger over a set of seed locations. Figure 24 shows an example of this. It can be seen that the user has swiped over two different sets of seeds represented by set 782 and set 784. In this example, mobile application 160 may aggregate values corresponding to the seeds in set 782 to obtain an average or other aggregated value. Likewise, the values corresponding to the seeds in the set 784 can also be aggregated to obtain an aggregated value. The seed locations themselves can be displayed to represent one or more variables, while the added value can be represented by the color of the line connecting these seed locations (which the user has hovered over). In the example shown in Figure 24, dots displayed at individual seed locations can be colored to show plant locations and to represent per-plant location data (such as planting depth error, the difference between a seed's target depth and the actual detected depth), with the color bars connecting these individual seed locations being used to represent data for the seed set, as an average value for those plants in the set.

[00225] Additionally, colors can be selected based on error threshold values. For example, a color may indicate that a certain value is acceptable, moderate, or unacceptable. By way of example, the color green may indicate an acceptable depth error that can be distinguished by a depth error of less than 1 centimeter, while the color yellow indicates a moderate depth error that indicates a depth error between 1 and 2 centimeters. , and the red color may indicate an unacceptable depth error indicating that the depth error exceeds 2 centimeters. Thresholds can be default thresholds, predefined thresholds, or user-defined thresholds.

[00226] The combination of per-plant and per-row data can be used by the diagnostic/prognostic system 141 in diagnosing planter problems such as, without limitation, unsanitary seed meter operation, unsanitary wiper or trash opener operation, Unhealthy closing wheel or downforce setting values, excessive planting speed, etc.

[00227] Table 1 shows some examples of different information that may include, alone or in combination, as a graph of one or several elements, information that is represented as an augmented reality display: TABLE 1 • Type and/or rate of pesticide • Impact of drift on actual application versus planned/intended chemical distribution • Applicator speed • Sprayer boom swings over plant • Sprayer application quality score • Fertilizer type and/or rate • Estimated soil nutrient level /measured before or after operation • Fertilizer application job quality score • Crop quality metric • Residue coverage • Tillage tool depth • Tillage tool angle (e.g. disc angle) • Pitch attribute soil, structure or other soil health metric • Crop quality score • Plant health metric such as height, NDVI value, others • Crop stage, lodging magnitude, other crop development • Site-specific precipitation, evapotranspiration , other weather conditions • Other

[00228] In the examples of enhanced or enlarged displays shown in the present Figures, the grids displayed around the locations of individual plants and other locations are shown as square grids. However, checkerboards can also have other shapes, such as circles, hexagons, octagons or another shape. In some examples, the format may convey information. For example, formats may convey information that can be understood by a colorblind person who may not necessarily be able to see all the different shades of color. Likewise, the format can identify the specific data that can be aggregated into a metric. For example, a square grid may indicate that a composite value was generated using data from four neighboring orthogonally located points (such as four neighboring orthogonally located plants or other locations), while an octagon may indicate that a composite value was generated using data from eight points. neighbors, with four located orthogonally and four located diagonally in relation to the subject point.

[00229] Furthermore, although some examples of enhanced or enlarged displays in this document show that the formats are centered on plan locations, in other examples the displayed formats may be centered on spaces between lines, on a geographic grid of latitude and longitude, etc. In some examples, shapes can identify where material was applied to the field. For example, the shape may be a rectangle covering areas where a floor engagement element, application nozzle, or other machine component was in a specific configuration. The two ends of the rectangle can define the start and end of a specific configuration. The sides of the rectangle can be defined by a row width or a fraction of the row width, a nozzle spray pattern width, a bed or trench width, estimated root zone width, or other soil, plant, or soil criteria. machine.

[00230] It can therefore be seen that the current system allows the farmer to obtain information throughout an entire year that will assist him in making many decisions that need to be made for a successful harvest. In the pre-planting season, the farmer can obtain information that will help him decide which crop to plant and in making tillage, fertilizer, crop and other decisions. Information can be displayed on an alert monitor or on a mobile device. The system can be an image or other data request and receive georeferenced data for display. These are just examples. Thus, the present system reduces the need to perform multiple database access operations when displaying data in different views. Instead, the data can be overlaid or otherwise displayed in a georeferenced augmented reality display. This reduces rendering overhead and network bandwidth. Improves the presentation, completeness and speed of information, thereby improving the operation of the display device.

[00231] The present discussion mentioned processors and servers. In one example, the processors and servers include computer processors with memory and a set of associated timing circuits not separately shown. Processors and servers are functional parts of the systems or devices to which they belong and are enabled by, and facilitate the functionality of, other components or items in those systems.

[00232] Processors and servers are functional parts of the systems or devices to which they belong and are activated and facilitate the functionality of other components or items of these systems. If the implementation comprises multiple processors, the processors may be local or remote, or some may be local while others are remote. Processors can share information through wired, wireless, or a mix of communication systems. Processors can fixedly or dynamically assign portions of computation to processors. Processors can perform their tasks with varying degrees of human supervision or intervention. Humans can be located at any appropriate processor or communication node of a distributed system. Humans may be physically located on a work machine or in some other location. Examples of human interaction devices without limitation include displays, touch displays, wearable displays, audio or speech output such as headphones or speakers, microphones, haptic output such as vibration or thermal devices, brainwave sensors , eye trackers, heart rate and other physiological sensors, or cameras for facial, gesture or other body monitoring. In some examples, processors may include systems on a chip, embedded processors, servers, desktop computers, tablets, or cell phones. In some examples, unauthorized monitoring, alteration or replacement of data communications is mitigated. Without limitation, the examples may partially or fully implement authentication of nodes sending or receiving data. Authentication systems may include physically unclonable functions (PUFs), encryption of data sent between nodes, use of a distributed, immutable ledger of data updates (e.g., Block Chain), or other mechanisms or systems or functionalities. Additionally, various user interface views were discussed. Displays can take a wide variety of different forms and can have a wide variety of different user-actuable input mechanisms arranged therein. For example, user-actuable input mechanisms can be text boxes, check boxes, icons, hypertext links, drop-down menus, search boxes, etc. Mechanisms can also be actuated in a wide variety of different ways. For example, the mechanisms can be actuated using a point-and-click device (such as a trackball or mouse). Mechanisms can be actuated using hardware buttons, switches, a joystick or keyboard, free levers or thumb panels, etc. The mechanisms can also be activated using a virtual keyboard or other virtual triggers. Additionally, where the display on which the mechanisms are displayed is a touch-sensitive display, the mechanisms may be actuated using touch gestures. Furthermore, where the device displaying them has speech recognition components, the mechanisms can be actuated using speech commands.

[00233] Numerous data stores were also discussed. It will be noted that the data stores can each be broken down into multiple data stores. All may be local to the systems accessing them, all may be remote, or some may be local while others are remote. All these configurations are covered in this document.

[00234] Additionally, the Figures show a number of blocks with functionality assigned to each block. Note that fewer blocks can be used so that the functionality is performed by fewer components. Additionally, more blocks can be used with functionality distributed across more components.

[00235] It will also be noted that system 100, or portions thereof, may be arranged in a wide variety of different devices. Some of these devices include servers, desktop computers, laptop-type computers, tablet-type computers, or other mobile devices such as palm top computers, cell phones, smart phones, multimedia players, personal digital assistants, etc.

[00236] Figure 25 is a simplified block diagram of an illustrative example of a portable or mobile computing device 16 that can be used as a user or client mobile device 104, in which the present system (or parts thereof) can be implemented. It should be noted that the device 16 can be portable, mounted in an agricultural equipment cabin, or also in other places. It can be a relatively small device (when it is portable, for example) or larger (when mounted in a cabin, for example). 26-27 are examples of portable or mobile devices.

[00237] Figure 25 provides a general block diagram of the components of a client device 16 that can execute components of system 100 or that interact with system 100, or both. In device 16, a communications link 13 is provided, which allows the portable device to communicate with other computing devices and, in some examples, provides a channel for receiving information automatically, such as by scanning. Examples of communication link 13 include an infrared port, a serial/USB port, a wired network port such as an Ethernet port, and a wireless network port that allows communication via one or more communication protocols, including General Packet Radio Services (GPRS), LTE, HSPA, HSPA+ and other 3G and 4G radio protocols, 1Xrtt and Short Message Service, which are wireless services used to provide cellular access to a network, as well as 802.11 protocols and 802.11b (Wi-Fi) and the Bluetooth protocol, which provide local wireless connections for networks.

[00238] In other examples, applications or systems (such as mobile agricultural application 160) are received on a removable Secure Digital (SD) card that is connected to an SD card interface 15. The SD card interface 15 and communication links 13 communicate with a processor 17 (which may also incorporate processors or servers from the previous figures) over a bus 19 that is also connected to memory 21 and input/output (I/O) components 23, as well as clock 25 and location system 27.

[00239] I/O components 23, in one example, are provided to facilitate input and output operations. I/O components 23 for various examples of device 16 may include input components such as buttons, touch sensors, multi-touch sensors, optical or video sensors, voice sensors, touch displays, proximity sensors, microphones, sensors tilt and gravity switch, and output components such as a display device, a speaker, and/or a printer port. Other I/O components 23 may also be used.

[00240] Clock 25 illustratively comprises a real-time clock component that outputs a time and date. It may also, illustratively, provide timing functions for the processor 17.

[00241] Location system 27 illustratively includes a component that generates a current geographic location of device 16. This may include, for example, a global positioning system (GPS) receiver, a LORAN system, an estimated navigation system , a cell phone triangulation system or other positioning system. It may also include, for example, mapping software or navigation software that generates maps, navigation routes and other desired geographic functions.

[00242] Memory 21 stores operating system 29, network settings 31, applications 33, application configuration settings 35, data storage 37, communication units 39, and communication configuration settings 41. Memory 21 may include all the types of tangible volatile and non-volatile computer-readable memory devices. It may also include computer storage media (described below). Memory 21 stores computer-readable instructions that, when executed by processor 17, cause the processor to perform computer-implemented steps or functions in accordance with the instructions. Likewise, device 16 may have a client system 24 that may run various applications or incorporate portions of functionality into service 112 or mobile device 104. Processor 17 may be activated by other components to facilitate its functionality as well.

[00243] Examples of settings 31 include such things as proxy information, Internet connection information, and mappings. Settings 35 include settings that tailor the application to a specific user. Settings 41 provide parameters for communicating with other computers and include items such as GPRS parameters, SMS parameters, connection usernames and passwords.

[00244] Applications 33 may be applications that were previously stored on device 16 or applications that are installed during use, although they may also be part of operating system 29 or hosted external to device 16.

[00245] Figure 26 shows an example in which the device 16 is a tablet-type computer 600. In Figure 26, the computer 600 is shown with the user interface display screen 602. The screen 602 may be a touch screen touch (so touch gestures from a user's finger can be used to interact with the application) or a pen-enabled interface that receives input from a pen or stylus. The computer 600 may also use a virtual on-screen keyboard. Of course, the computer 600 may also be connected to a keyboard or other user input device via a suitable connection mechanism, such as a wireless connection or USB port, for example. Computer 600 may also illustratively receive voice input.

[00246] Figure 27 shows that the device may be a smart phone 71. The smart phone 71 has a touch screen 73, which displays icons or tiles or other user input mechanisms 75. The mechanisms 75 may be used by a user to run applications, make calls, perform data transfer operations, etc. In general, the smart phone 71 is embedded in a mobile operating system and offers more advanced connectivity and computing capabilities than a regular phone.

[00247] Note that other shapes of the devices 16 are possible.

[00248] Figure 28 is an example of a computing environment in which system 100, or parts thereof, (for example) can be deployed. Referring to Figures 28, an exemplary system for implementing some embodiments includes a computing device in the form of a computer 810 programmed to operate as discussed above. Components of computer 810 may include, but are not limited to, a processing unit 820 (which may comprise processors or servers from the preceding figures), a system memory 830, and a system bus 821 that couples various system components including the system memory to processing unit 820. System bus 821 may be any of several types of bus structures using any of several bus architectures. These architectures may include Industrial Standard Architecture (ISA) bus, Microchannel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronic Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus. . Memory and programs described in relation to previous Figures can be implemented in portions corresponding to Figure 28.

[00249] Computer 810 typically includes a variety of computer-readable media. Computer-readable media can be any media (or a combination of media) that can be accessed by computer 810 and includes volatile and non-volatile media. The media can be removable or non-removable. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer-readable storage media is different and does not include a modulated data signal or carrier wave. Includes hardware storage media, including volatile and non-volatile, removable and non-removable media implemented by any method or in any technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. For example, computer storage media includes RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile discs (DVD) or other optical disk storage, magnetic disk storage, or other storage devices magnetic or any other medium that can be used to store the desired information and that can be accessed by the computer 810. The communication medium comprises computer-readable instructions, data structures, program modules or other data on any information delivery medium . The term “modulated data signal” means a signal that has one or more of its characteristics adjusted or altered in such a way as to encode information in the signal. By way of example, communication media include wired network or direct wired connection and wireless media such as acoustic, RF, infrared and other wireless media.

[00250] System memory 830 may include computer storage media in the form of read-only memory (ROM) 831 and random access memory (RAM) 832. A basic input/output system 833 (BIOS), containing the routines Basic information that helps transfer information between elements within the computer 810, such as during startup, is typically stored in ROM 831. RAM 832 can store data and/or program modules that are immediately accessible or being operated on by processing unit 820 , such as operating system 834 , applications 835 , other program modules 836 , and other data 837 .

[00251] Computer 810 may also include other computer storage media. Figure 28 shows a hard disk drive 841 that reads from or writes to magnetic media and an optical disk drive 855 that reads from or writes to a removable optical disk 856, such as a CD ROM or other optical media. The hard disk drive 841 may be connected to the system bus 821 via a memory interface, such as interface 840, and the optical disk drive 855 may be connected to the system bus 821 via a memory interface, such as the 850 interface.

[00252] Alternatively or additionally, the functionality described herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that may be used include Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (e.g., ASICs), Application-Specific Standard Products (e.g., ASSPs), System-on-a-Chip Systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[00253] The units and their associated computer storage media discussed above and illustrated in Figure 28, provide storage of computer-readable instructions, data structures, program modules, and other data for the computer 810.

[00254] A user can enter commands and information into computer 810 through input devices, such as a keyboard 862, a microphone 863, and a pointing device 861, such as a mouse, ball mouse, or touch control. Other input devices (not shown) may include a joystick, game controller, satellite dish, digitizer, or the like. These and other input devices are often connected to the processing unit 820 through a user input interface 860 that is coupled to the system bus, but may be connected through other interface and bus structures, such as a parallel port, port game console or a Universal Serial Bus (USB). A visual display 891 or other type of display device is also connected to system bus 821 through an interface, such as a video interface 890. In addition to the monitor, computers may also include other peripheral output devices, such as speakers. speakers 897 and printer 896, which can be connected via a peripheral output interface 895.

[00255] The computer 810 may be operated in a network environment using logical connections to one or more remote computers, such as a remote computer 880. The remote computer 880 may be a personal computer, a portable device, a server, a router, a network PC, peer device, or other common network node and typically includes many or all of the elements described above in connection with computer 810. The logical connections depicted in Figure 28 include a local area network (LAN) 871, a controller area network (CAN) and a wide area network (WAN) 873, but may also include other networks.

[00256] When used in a LAN network environment, the computer 810 is connected to the LAN 871 via a network interface or adapter 870. When used in a WAN network environment, the computer 810 typically includes a modem 872 or other means of establishing communications over the WAN 873, such as the Internet. Modem 872, which may be internal or external, may be connected to system bus 821 via user input interface 860 or other appropriate mechanism. In a network environment, program modules represented relative to computer 810, or portions thereof, may be stored in the remote memory storage device. For example, Figure 28 illustrates remote application programs 885 as residing on remote computer 880. It will be recognized that the network connections shown are examples.

[00257] It should also be noted that the different examples described here can be combined in different ways. That is, parts of one or more examples can be combined with parts of one or more other examples. All of this is covered here.

[00258] Although the matter has been described in specific language for structural characteristics and/or methodological acts, it is understood that the matter defined in the attached claims is not necessarily limited to the specific characteristics or acts described above. Instead, the specific features and acts described above are described as exemplary forms of implementing the claims.

Claims (15)

Computer-implemented method, characterized in that it comprises: displaying (686), on a display device, a display of an agricultural surface; receiving georeferenced interspersed crop data (715) indicative of a characteristic of a plurality of different crop species planted on the agricultural surface and corresponding geographically to the agricultural surface; and generating (320) an enhanced display on the display device by enhancing the display of the agricultural surface with visual cues, the visual cues being indicative of the georeferenced interleaved crop data (715). The computer-implemented method of claim 1, wherein receiving georeferenced intercrop data comprises: generating (513) plant location data indicative of plant locations of the intercrop plants and plant species data plants indicative of a plant species at plant locations. The computer-implemented method of claim 2, wherein generating an enhanced display comprises: displaying (515), such as visual cues, plant species indicators, plant species data indicators, at locations of plants. Computer-implemented method according to claim 1, characterized by the fact that receiving georeferenced interleaved crop data comprises: generating a per-plant metric (514) for a given plant at a given plant location. The computer-implemented method of claim 4, wherein generating an enhanced display comprises: displaying (515), as the visual cues, the per-plant metric at plant locations. Computer-implemented method according to claim 1, characterized by the fact that receiving georeferenced intercrop data comprises: generating (517) a location-specific sustainability metric corresponding to a geographic location of the plants in intercrop. The computer-implemented method of claim 6, wherein generating an enhanced display comprises: displaying (515), as the visual cues, a sustainability metric indicator, indicative of the sustainability metric, at geographic locations The computer-implemented method of claim 7, wherein generating a site-specific sustainability metric comprises: generating (517), as the site-specific sustainability metric, at least one of a carbon footprint crop input footprint, a carbon sequestration value, or a difference between the crop input carbon footprint and the carbon sequestration value. The computer-implemented method of claim 1, wherein receiving georeferenced interleaved crop data comprises: generating (519) a crop tillage effect metric indicative of a corresponding site-specific crop tillage effect to a geographic location of plants in intercropped culture. The computer-implemented method of claim 9, wherein generating an enhanced display comprises: displaying (515), as the visual cues, a crop tillage effect indicator indicative of the crop tillage effect metric. culture in geographic location. The computer-implemented method of claim 1, wherein the georeferenced interleaved crop data is indicative of a geographic location of a first crop species and a geographic location of a second crop species and wherein the generation of an improved display comprises: identifying a first visual representation (516) corresponding to the first crop species; identify (515) a second visual representation (516) corresponding to the second kind of culture; display (515) the first visual representation in the geographical location of the first species of culture; and displaying (515) the second visual representation in the geographic location of the second kind of culture. The computer-implemented method of claim 11, further comprising: detecting a user selection input (724) selecting the first crop species; and improving the display by displaying, at the geographic location (552) of the second crop species, a replacement display that replaces the display of the second visual representation. The computer-implemented method of claim 12, wherein the display enhancement comprises: displaying, at the geographic location of the second crop species, a display indicative of the location (552) of the second crop species prior to the emergence of the second kind of culture. The computer-implemented method of claim 12, wherein the display enhancement comprises: displaying, at the geographic location (552) of the second crop species, a simulated display of the location of the second crop species without the second crop species of culture. Agricultural system, characterized by the fact that it comprises: at least one processor (154); and a data store (156) storing computer-executable instructions that, when executed by the at least one processor, cause the at least one processor to perform steps comprising: displaying (676), on a display device, a display of a agricultural surface; receiving georeferenced interspersed crop data (715) indicative of a characteristic of a plurality of different crop species planted on the agricultural surface and corresponding geographically to the agricultural surface; and generating (676) an enhanced display on the display device by enhancing the display of the agricultural surface with visual cues, the visual cues being indicative of the georeferenced interleaved crop data (715).

Images