AU2021104829A4 - Method and Apparatus for Estimating Tree Fruit Load and Harvesting Time - Google Patents

Abstract

A method for estimating tree fruit load and harvesting time, the method comprising the steps of: a) Collecting, using a first one or more cameras, an initial measurement of a number and/or maturity stage of flowers on a plurality of fruit trees and, on one or more subsequent occasions, a subsequent measurement of the number and/or maturity stage of flowers on the plurality of fruit trees; b) Collecting, using a second one or more cameras, an initial measurement of a number and/or size of fruit on the plurality of fruit trees and, on one or more subsequent occasions, a subsequent measurement of the number and/or size of fruit on the plurality of fruit trees; c) Collecting, using one or more temperature sensors, ambient temperature measurements at one or more locations among the plurality of fruit trees; d) Calculating, based on the initial measurement and the subsequent measurement of the number of flowers, the initial measurement and the subsequent measurement of the number and/or size of fruit and the ambient temperature measurements the tree fruit load in the plurality of fruit trees and the harvesting time of at least a portion of the fruit. 1/4 DRAWINGS Flower Measurements Fruit Measurements Flower Number Fruit Number Temperature Flower Maturity Fruit Size Measurements 11 12 Dry Matter Content 10 /SSC Measurements 13 17 14 FIG 1158 19 020 Harvest Time Estimate Fruit Load Estimate

Description

1/4

DRAWINGS

Flower Measurements Fruit Measurements Flower Number Fruit Number Temperature Flower Maturity Fruit Size Measurements

11 12

Dry Matter Content /SSC Measurements

13 17

14

FIG 1158

19 020

Harvest Time Estimate Fruit Load Estimate

METHOD AND APPARATUS FOR ESTIMATING TREE FRUIT LOAD AND HARVESTING TIME TECHNICAL FIELD

[0001] The present invention relates to a method and apparatus for estimating tree fruit load and harvesting time.

BACKGROUND

[0002] In agriculture, an estimation of tree fruit load (fruit number and fruit size) is important to allow farmers to allocate appropriate resources, such as labour and materials (vehicles, field bins, packing cartons and so on), to the harvest. Underestimating the tree fruit load may result in an inability to harvest the entire crop, while overestimating the tree fruit load results in a waste of resources and additional cost.

[0003] In addition, estimating the harvesting time of the fruit is important not just so that resources can be allocated at an appropriate time, but also to allow for critical marketing decisions, such as the choice of market, the booking of advertising and so on. This is particularly important in situations with long supply chains (such as where the fruit is to be exported) or where the fruit is destined for further processing (such as wine making).

[0004] Traditionally, forecast of harvest timing relied on a manual estimate of the time of peak flowering(s), and estimating the tree fruit load has been performed manually, such as by estimating the load on a representative tree within a block and then multiplying the result by the number of trees, or by using a counting frame to determine the volume of fruit in a given area and then extrapolating the fruit yield based on addition information, such as tree density and size. However, these techniques are often inaccurate, being dependent on operator performance and failing to sample an adequate representation of the orchard. Further, existing practices have a poor integration of the various measures towards forecasting the harvest time profile (i.e., number and size of fruit by week).

[0005] In more recent times, automated systems for determining fruit number have been developed. In one embodiment, LIDAR devices are used to scan fruit trees and estimate the fruit load. However, LIDAR devices are typically expensive, meaning that these systems are not particularly cost effective. Further, these systems typically deliver only one or two elements of that required for prediction of harvest load by harvest time.

[0006] Thus, there would be an advantage if it were possible to provided a method and apparatus for estimating tree fruit number and size and harvesting time that was both fast and accurate, but that was also more cost effective than existing automated systems.

[0007] It will be clearly understood that, if a prior art publication is referred to herein, this reference does not constitute an admission that the publication forms part of the common general knowledge in the art in Australia or in any other country.

SUMMARY OF INVENTION

[0008] Embodiments of the present invention provide a method and apparatus for estimating tree fruit number and size and harvesting time, which may at least partially address one or more of the problems or deficiencies mentioned above or which may provide the public with a useful or commercial choice.

[0009] With the foregoing in view, the present invention resides broadly in a first aspect in a method for estimating tree fruit load and harvesting time, the method comprising the steps of: a) Collecting, using a first one or more cameras, an initial measurement of a number and/or maturity stage of flowers on a plurality of fruit trees and, on one or more subsequent occasions, a subsequent measurement of the number and/or maturity stage of flowers on the plurality of fruit trees; b) Collecting, using a second one or more cameras, an initial measurement of a number and/or size of fruit on the plurality of fruit trees and, on one or more subsequent occasions, a subsequent measurement of the number and/or size of fruit on the plurality of fruit trees; c) Collecting, using one or more temperature sensors, ambient temperature measurements at one or more locations among the plurality of fruit trees; d) Calculating, based on the initial measurement and the subsequent measurement of the number of flowers, the initial measurement and the subsequent measurement of the number and/or size of fruit and the ambient temperature measurements the tree fruit load in the plurality of fruit trees and the harvesting time of at least a portion of the fruit.

[0010] The first one or more cameras and the second one or more cameras may collect measurements of the respective parameters directly. In other embodiments of the invention, the first one or more cameras and the second one or more cameras may collect data from which the measurements of the respective parameters may be determined. For instance, data collected by the first one or more cameras and the second one or more cameras may undergo processing (particularly electronic processing using a computing device) in order to determine the measurements of the respective parameters.

[0011] The first one or more cameras used to collect the initial and subsequent measurements of the number of flowers may be of any suitable form. In some embodiments, different types of cameras may be used for collecting the initial measurement and the subsequent measurement. More preferably, however, the same type of camera is used to collect the initial and subsequent measurement of the number of flowers. In a preferred embodiment of the invention, the first one or more cameras may include one or more red green blue (RGB) cameras.

[0012] The initial measurement and the subsequent measurement may be collected using the first one or more cameras in any suitable manner. Preferably, however, the first one or more cameras may be moved relative to the plurality of fruits trees. It is envisaged that, as the one or more cameras are moved relative to the plurality of fruit trees (preferably, by travelling past the plurality of fruit trees), the first one or more cameras may collect the initial measurement and/or the subsequent measurement.

[0013] The first one or more cameras may be moved relative to the plurality of fruit trees using any suitable technique. For instance, the first one or more cameras may be carried manually. More preferably, however, the first one or more cameras may be mounted to a vehicle configured to move relative to the plurality of fruit trees. Any suitable powered or unpowered vehicle may be used, and the vehicle may be manned or unmanned. The vehicle may comprise a ground vehicle or an aerial vehicle, although preferably the vehicle comprises a land vehicle.

[0014] In a preferred embodiment of the invention, the first one or more cameras may collect additional measurements. In some embodiments of the invention, the first one or more cameras may collect measurements relating to the density of flowers on each of the plurality of fruit trees, the distribution of flowers on each of the plurality of fruit trees, the size of the flowers, the colour of the fruit and so on.

[0015] In some embodiments of the invention, the first one or more cameras may collect still images of the plurality of fruit trees. More preferably, the first one or more cameras may collect video of the plurality of fruit trees.

[0016] In a preferred embodiment of the invention, the plurality of fruit trees may comprise a portion of an orchard or similar plantation. In some embodiments, the plurality of fruit trees may comprise the entirety of an orchard or similar plantation.

[0017] In some embodiments of the invention, the first one or more cameras may comprise a pair of RGB cameras. In this embodiment, it is envisaged that the pair of RGB cameras may be configured to be located at approximately 180° to one another. In this way, each of the cameras in the pair of cameras may collect measurements of different fruit trees, such as fruit trees in adjacent rows within an orchard. Thus, each of the pair of RGB cameras may be positioned approximately 90° to the direction of travel of the cameras (or the vehicle to which they are mounted) relative to the fruit trees.

[0018] The initial measurement of the number of flowers and/or the flower maturity may be collected at any suitable time. Preferably, however, there must be at least some flowers on the plurality of fruit trees for the initial measurement to be collected. Thus, the initial measurement is preferably collected after the commencement of flowering of the plurality of fruit trees.

[0019] The subsequent measurement of the number of flowers and/or the flower maturity may be collected at any suitable time interval following the initial measurement. Preferably, however, the subsequent measurement is collected soon enough following the initial measurement that meaningful data may be obtained, but not so soon after the initial measurement that there is no or negligible change in the number of flowers.

[0020] The maturity of a flower may be used to assist in estimating the harvest time of the fruit. In particular, it will be understood that as flowering ends, pollination occurs and the fruit are set. Thus, by taking measurements of the flower maturity (for instance, the time since the initial flowering), the present invention may be used to accurately estimate the harvest time of the fruit. In some embodiments, the time since the initial flowering (i.e. the maturity of the flower) may be used in conjunction with the measurements of the heat units to determine the harvest time of the fruit.

[0021] It will be understood that fruit trees experience one or more flowering events during the cycle of fruit production. These flowering events may be spread over days, weeks or even months. In a preferred embodiment of the invention, subsequent measurements may be collected at sufficient frequency to estimate a peak of the (or each) flowering event. Further, the subsequent measurements may allow for the estimation of the spread of fruit harvest (both in terms of time and location) during the harvest period.

[0022] It is envisaged that the extent of flowering in the (or each) flowering event may be used as an initial point in estimating the proportion of fruit ready for harvest at different harvest times. For instance, the proportion of flowers in the different flowering events may be determined, and those proportions may be applied to the count of total fruit number. In this way, an estimate of fruit load associated to the different flowering events may be determined.

[0023] In addition, another technique to determine the proportion of fruit ready for harvest may be to determine the size profile of the fruit (i.e. number of fruit in each of two or more size categories) in order to determine an estimate of fruit in different harvest time classes.

[0024] These estimates may be adjusted as the season progresses (for instance, a rain event may result in at least some of the flowers in a flowering event dropping off). Thus, the collection of subsequent measurements is advantageous in that counting the same fruit on more than one occasion allows for an estimation not just of the harvesting time, but the spread (in terms of time) of the harvesting period.

[0025] In some embodiments of the invention, the subsequent measurement may be collected between 1 day and 1 month following the initial measurement. More preferably, the subsequent measurement may be collected between 3 days and 3 weeks following the initial measurement. Still more preferably, the subsequent measurement may be collected between days and 2 weeks following the initial measurement. Most preferably, the subsequent measurement may be collected approximately one week following the initial measurement.

[0026] It is envisaged that a plurality of subsequent measurements may be collected. Thus, the subsequent measurements may comprise a first subsequent measurement, a second subsequent measurement, a third subsequent measurement and so on. It is envisaged that the time interval between the initial measurement and the first subsequent measurement may be the same as the time interval between each of the subsequent measurements, although it is also possible that the time interval may vary.

[0027] The second one or more cameras used to collect the initial and subsequent measurements of the number and/or size of fruit on the plurality of fruits trees may be of any suitable form. In some embodiments, different types of cameras may be used for collecting the initial measurement and the subsequent measurement. More preferably, however, the same type of camera is used to collect the initial and subsequent measurement of the number and/or size of fruit on the plurality of fruits trees. Ina preferred embodiment of the invention, the second one or more cameras may include one or more depth cameras. Any suitable depth cameras may be used, such as time-of-flight (ToF) cameras, Active InfraRed Spectroscopy (AIRS) cameras and the like. It is envisaged that the ToF cameras may be used to enable the measurement of at least the size of the fruit. The ToF camera may also be used to determine the number of fruit, although the measurement of the number of fruit may also be collected using one or more RGB cameras.

[0028] It will be understood that the term "fruit size" may refer not just to the physical dimensions of the fruit, but may also refer to the mass of the fruit (as the mass of the fruit may correlate to the lineal dimensions of the fruit).

[0029] In some embodiments of the invention, the first one or more cameras may include a camera that also comprises one or the second one or more cameras. Thus, a single camera may perform more than one function.

[0030] The initial measurement and the subsequent measurement may be collected using the second one or more cameras in any suitable manner. Preferably, however, the second one or more cameras may be moved relative to the plurality of fruits trees. It is envisaged that, as the second one or more cameras are moved relative to the plurality of fruit trees (preferably, by travelling past the plurality of fruit trees), the second one or more cameras may collect the initial measurement and/or the subsequent measurement.

[0031] The second one or more cameras may be moved relative to the plurality of fruit trees using any suitable technique. For instance, the second one or more cameras may be carried manually. More preferably, however, the second one or more cameras may be mounted to a vehicle configured to move relative to the plurality of fruit trees. Any suitable powered or unpowered vehicle may be used, and the vehicle may be manned or unmanned. Preferably, however, the vehicle may comprise a ground vehicle. In a preferred embodiment of the invention, the first one or more cameras and the second one or more cameras may be mounted to the same vehicle. In this embodiment, the first one or more cameras and the second one or more cameras may be moved relative to the plurality of fruit trees simultaneously. However, not all of the cameras may be operated each time the vehicle is moved relative to the plurality of fruit trees.

[0032] In a preferred embodiment of the invention, the first one or more cameras and/or the second one or more cameras may collect additional measurements. In some embodiments of the invention, the second one or more cameras may collect measurements relating to colour of the fruit, characteristics of the fruit tree canopy and so on.

[0033] In some embodiments of the invention, the second one or more cameras may collect still images of the plurality of fruit trees. More preferably, the second one or more cameras may collect video of the plurality of fruit trees.

[0034] The initial measurement of the number and/or size of fruit may be collected at any suitable time. Preferably, however, there must be at least some fruit on the plurality of fruit trees for the initial measurement to be collected. Thus, the initial measurement is preferably collected after initial fruit set on the plurality of fruit trees. However, typically after the initial fruit set there is a period of growth in which some of the fruit drop off the tree, thereby thinning out the number of fruit in a tree. Thus, in a preferred embodiment, the initial measurement of the number and/or size of fruit may be collected after the fruit drop period. It will be understood that the initial measurement of the number and/or size of fruit may preferably be collected a number of weeks prior to the harvesting of the fruit.

[0035] The subsequent measurements of the number and/or size of fruit may be collected at any suitable time interval following the initial measurement, and it will be understood that the timing of the subsequent measurements may be guided by the number and spread of the flowering events. However, in some embodiments of the invention, the subsequent measurement may be collected between 1 day and 1 month following the initial measurement. More preferably, the subsequent measurement may be collected between 3 days and 3 weeks following the initial measurement. Still more preferably, the subsequent measurement may be collected between 5 days and 2 weeks following the initial measurement. Most preferably, the subsequent measurement may be collected approximately one week following the initial measurement.

[0036] It is envisaged that a plurality of subsequent measurements may be collected. Thus, the subsequent measurements may comprise a first subsequent measurement, a second subsequent measurement, a third subsequent measurement and so on. It is envisaged that the time interval between the initial measurement and the first subsequent measurement may be the same as the time interval between each of the subsequent measurements, although it is also possible that the time interval may vary.

[0037] Any suitable temperature sensors may be used to measure the ambient temperatures within the orchard, and, more preferably, the ambient temperatures outside of the canopy of the tree. In this way, the ambient temperature measurements may be relevant to flower and fruit development. Preferably, the one or more temperature sensors are located among the plurality of fruit trees and may not be movable relative thereto. More preferably, the one or more temperature sensors adjacent to a group of trees. Specifically, it is envisaged that the one or more temperature sensors may not be located under the foliage of the trees. Any suitable number of temperature sensors may be located among the plurality of fruit trees, and it will be understood that the number of temperature sensors may depend on a number of factors, such as the area of land on which the plurality of fruit trees is planted, the topography of the land, the number of fruit trees, the density of planting of the fruit trees and so on.

[0038] In a preferred embodiment of the invention, however, it is envisaged that sufficient temperature sensors may be provided so that a relatively accurate determination of the ambient temperature across the plurality of fruit trees may be achieved.

[0039] In a preferred embodiment the ambient temperatures measurements collected by the one or more temperature sensors may be used to calculate the heat units generated within the plurality of trees. The calculation of heat units from temperature measurements is conventional, and no additional discussion of this is required. However, it is envisaged that the calculation of the heat units may be conducted using one or more computing devices.

[0040] The one or more temperature sensors may collect ambient temperature measurements substantially continuously, or may collect ambient temperature measurements at regular (or irregular) intervals of time throughout a day. Preferably, the temperatures sensors may collect measurements at sufficient regularity so that an accurate measurement of the daily maximum and minimum temperatures may be determined.

[0041] In some embodiments of the invention, a determination of the accumulated heat units over a period of time may be made. The determination of the accumulated heat units may be used to estimate the harvest maturity (and therefore the harvest time). In some embodiments, an average of historical temperature measurements for a particular location may be used to predict how heat units will accumulate, and thus estimate the harvest time. It is envisaged that the historical records may be updated with new temperature measurements as they are taken. Thus, the harvest time estimate may be updated regularly (for instance on a daily basis or as new temperature measurements are taken). It is envisaged that, while the initial estimates of the harvest time may be based substantially entirely on the historical temperature measurements, updates to the measurements (as new temperature measurements are taken) throughout the season may improve the accuracy of the estimate of the harvest time.

[0042] It is envisaged that, in some embodiments, one or more additional measurements may be taken to assist in calculating the tree fruit load and/or the harvesting time of at least a portion of the fruit and/or to validate the calculation of the tree fruit load and/or harvesting time based on the other measurements. Any suitable additional measurements may be taken, and it will be understood that the additional measurements may depend on a number of factors, such as the type of fruit. In some embodiments, measurements of the colour of the fruit (and/or the change in the colour of the fruit over time) may be used to assist in calculating the tree fruit load and/or the harvesting time and/or to validate the calculation of the tree fruit load and/or harvesting time based on the other measurements. The colourof the fruit may include the colour of the skin of the fruit, the flesh of the fruit, or a combination of the two.

[0043] In one embodiment, one or more measurements of the content of the fruit (and/or the change in dry matter content of the fruit over time) may be used to assist in calculating the harvesting time and/or to validate the calculation of the harvesting time based on the other measurements, in particular the heat units. The measurements of the contents of the fruit may be of any suitable form, and it will be understood that the measurements may depend on the type of fruit. For instance, in some embodiments of the invention, the dry matter content of the fruit may be measured. This may be particularly the case for certain fruit, such as mangoes, apples, avocadoes and kiwifruit. On the other hand, in some fruit that does not store starch (such as plums), a measurement of the soluble solids content (SSC) of the fruit may be taken.

[0044] The dry matter content or SSC of the fruit (and/or the change in dry matter content of the fruit over time) may be used to assist in calculating the harvesting time and/or to validate the calculation of the harvesting time based on the other measurements, in particular the heat units. The dry matter content or SSC may be measured using any suitable technique, although in a preferred embodiment of the invention, the dry matter content or SSC of the fruit may be measured using a non-destructive technique. Any suitable non-destructive technique may be used, such as, but not limited to near infrared spectroscopy (NIRS). It is envisaged that the non destructive measurement of the dry matter content or SSC of the fruit may be conducted manually, or a NIRS device (such as a NIRS scanner) may be mounted to the vehicle with the first one or more cameras and the second one or more cameras. In a preferred embodiment of the invention, the measurement of the dry matter content or SSC of the fruit may be conducted in the period leading up to the harvesting of the fruit. Thus, the measurement of the dry matter content or SSC of the fruit may only be conducted from a few days to a few weeks prior to harvest.

[0045] In one embodiment of the invention, measurements of the dry matter content or SSC of the fruit may be used to determine a rate of change (and particularly, a rate of increase) of the dry matter in the fruit.

[0046] In some embodiments of the invention, a computing device may be used to determine the harvest date of the fruit based on the measurements. This will be discussed in more detail later. However, it is envisaged that a desired harvesting condition may be set within the computing device. The computing device may then calculate a harvest date based on when the desired harvesting condition may be met or achieved. Any suitable desired harvesting condition may be set, although in one embodiment of the invention the desired harvesting condition may be that a proportion of the fruit have a predetermined dry matter content. As a specific example, a desired harvesting condition may be that 90% of the fruit have a dry matter content of at least 15% by weight. Thus, measurements of the current dry matter content of the fruit, as well as a calculation of the rate of increase of dry matter in the fruit, may be used to estimate a harvest date at which the desired harvesting condition is likely to be met.

[0047] As previously stated, the present invention includes the step of calculating the tree fruit load in the plurality of fruit trees and the harvesting time of at least a portion of the fruit. The tree fruit load and the harvesting time may be calculated based on the initial measurement and the subsequent measurement of the number of flowers, the initial measurement and the subsequent measurement of the number and/or size of fruit and the ambient temperature measurements (particularly once heat units are calculated based on the ambient temperature measurements). As previously stated, the dry matter or SSC measurements may also be used in the calculation of the harvesting time. Thus, in a specific embodiment, the tree fruit load and the harvesting time may be calculated using a combination of tree imagery (i.e. the measurements taken by the cameras), the temperature measurements and the dry matter or SSC measurements.

[0048] The tree fruit load and the harvesting time may be calculated using any suitable technique. Preferably, however, the tree fruit load and harvesting time may be calculated using one or more computing devices. Any suitable computing device may be used, such as, but not limited to, one or more computers, servers or the like, or any suitable combination thereof. The computing device may be a device on a network associated with a user, or may be provided via cloud computing.

[0049] In a preferred embodiment, the one or more computing devices may be configured to receive the measurements from the first one or more cameras, the second one or more cameras, the one or more temperature sensors and the dry matter or SSC measurement devices (where used). The computing devices may be configured to receive the measurements in any suitable manner. For instance, the measurements may be transmitted electronically from the first one or more cameras, the second one or more cameras and the one or more temperature sensors to the computing device via a physical connection (such as a cord, wire, cable or the like) or via a physical medium (such as a disc, CD, memory drive or the like). More preferably, however, the computing device may be configured to receive the measurements wirelessly from the first one or more cameras, the second one or more cameras and the one or more temperature sensors. Thus, in this embodiment of the invention, at least some of the first one or more cameras, the second one or more cameras and the one or more temperature sensors may be provided with a transmission portion configured to transmit the measurements to the computing device. The transmission portion may transmit the measurements to the computing device using any suitable technique, such as a WiFi connection, Bluetooth connection or the like.

[0050] In some embodiments of the invention, large volumes of data (such as image or video data) may be processed on a local computing device to measurements of the number of flowers and/or the number and/or size of the fruit. The measurements may then be transmitted to a further computing device for the calculation of the tree fruit load and harvesting time. In some embodiments, the calculation of the tree fruit load may be displayed on a map of the orchard (or portion thereof) to provide a heat map of the fruit density.

[0051] In a preferred embodiment of the invention, the computing device may include a calculation portion in the form of a processor. The calculation portion may be in electronic communication with a data storage portion such as a database or similar form of electronic storage. It is envisaged that the data storage portion may receive and store measurements received from the first one or more cameras, the second one or more cameras and the one or more temperature sensors. The data storage portion may also store historical data from the plurality of fruit trees. In some embodiments, the data storage portion may also include data received from other pluralities of fruit trees where the method of the present invention is, or has been,used.

[0052] In some embodiments of the invention, the computing device may further comprise one or more machine learning modules. In this embodiment of the invention, it is envisaged that the machine learning modules may be configured to assist in calculating the harvesting time and/or tree fruit load. This may be done using any suitable technique, although it is envisaged that the machine learning modules may be configured to determine the measurements of the number of flowers and/or the number and/or size of the fruit.

[0053] The estimate of tree fruit load and harvesting time determined by the computing device may be presented to a user in any suitable format. For instance, the estimate may be provided in the form of a number of fruit in the plurality of trees, or within blocks of trees within the plurality of trees, or even in individual trees. The harvesting time estimate may be provided in the form of a date or range of dates. In some embodiments, the estimate may be displayed on a map of the orchard (or portion thereof) to provide a heat map of the fruit or flower density.

[0054] In some embodiments of the invention, the computing device may be configured to generate a harvest forecast of fruit over a period of time. For instance, the computing device may be configured to generate a harvest forecast of fruit number by week. Preferably, the harvest forecast may also include a size profile of the fruit and/or a dry matter content profile of the fruit.

[0055] More preferably, however, the computing device may be configured to generate a map of the plurality of trees that represents any suitable parameter. For instance, the map may represent tree fruit load, flower distribution, flower to fruit ratio, variation in the number of fruit per tree and so on. The parameters may be displayed graphically (such as in different colours representing different tree parameters) and/or numerically.

[0056] It will be understood that the use of maps to represent the parameters may be used in any suitable manner. For instance, a map showing fruit load may be used to guide picking crews to parts of the orchard where fruit is ready for harvest. Maps showing the fruit to flower ratio or the variation in number of fruit per tree may be useful to an agronomist or orchard manager for future development or optimisation of planting within the orchard.

[0057] It is envisaged that the fruit in the plurality of trees will not all be ready to harvest at the same time. Instead, fruit in different regions of the plurality of trees may be produced and/or grow at different rates. In addition, as previously stated, multiple flowering events may take place over a period of time, which may result in the same tree carrying both fruit and flowers at the same time. Thus, it is envisaged that the estimate of harvest time may be updated over time as flowers blossom and fruit grows in different regions of the plurality of trees. Thus, in a preferred embodiment of the invention, the estimate of the tree fruit load and the harvesting time may be periodically updated (based on the subsequent measurements) as fruit is produced and grows in different regions of the plurality of trees.

[0058] In a second aspect, the invention resides broadly in a system for estimating tree fruit load and harvesting time, the system comprising:

A first one or more cameras configured to collect initial and subsequent measurements of a number and/or maturity stage of flowers on a plurality of fruit trees;

A second one or more cameras configured to collect initial and subsequent measurements of a number and/or size of fruit on the plurality of fruits trees;

One or more temperature sensors configured to collect ambient temperature measurements at one or more locations among the plurality of fruit trees; and

A calculation portion configured to receive the measurements from the first one or more cameras, the second one or more cameras and the one or more temperature sensors, wherein the calculation portion is configured to calculate the tree fruit load in the plurality of fruit trees and the harvesting time of at least a portion of the fruit based on a combination of the initial measurement and the subsequent measurements of the number of flowers and/or the flower maturity, the initial measurement and the subsequent measurements of the number and/or size of fruit and the ambient temperature measurements.

[0059] Preferably, the first one or more cameras and the second one or more cameras may be mounted to a vehicle configured to move relative to the plurality of fruit trees.

[00601 The present invention provides numerous advantages over the prior art, in that the present invention provides for the simultaneous determination of both the harvest time of fruit and the fruit load. This not just allows farmers to allocate appropriate resources, such as labour and materials (vehicles, field bins, packing cartons and so on), to the harvest, but allows farmers to accurately determine the time at which the resources will be required.

[0061] By accurately determining both the correct quantity of resources required as well as when the resources are required, wastage of resources (in terms of idle labour or excess quantities of materials) may be reduced or eliminated. Similarly, an accurate determine of the required resources may result in losses that can occur if insufficient resources are allocated to the harvest.

[0062] Any of the features described herein can be combined in any combination with any one or more of the other features described herein within the scope of the invention.

[0063] The reference to any prior art in this specification is not, and should not be taken as an acknowledgement or any form of suggestion that the prior art forms part of the common general knowledge.

BRIEF DESCRIPTION OF DRAWINGS

[0064] Preferred features, embodiments and variations of the invention may be discerned from the following Detailed Description which provides sufficient information for those skilled in the art to perform the invention. The Detailed Description is not to be regarded as limiting the scope of the preceding Summary of Invention in any way. The Detailed Description will make reference to a number of drawings as follows:

[0065] Figure 1 illustrates a schematic view of a method and system for estimating tree fruit load and harvesting time according to an embodiment of the present invention.

[0066] Figure 2 illustrates an apparatus for collecting measurements in a method and system for estimating tree fruit load and harvesting time according to an embodiment of the present invention.

[0067] Figure 3 illustrates a map of fruit load density in an orchard generated using the method and system for estimating tree fruit load and harvesting time according to an embodiment of the present invention.

[0068] Figure 4 illustrates a fruit size estimation generated using the method and system for estimating tree fruit load and harvesting time according to an embodiment of the present invention.

DETAILED DESCRIPTION

[0069] Figure 1 illustrates a schematic view of a method and system 10 for estimating tree fruit load and harvesting time according to an embodiment of the present invention.

[0070] In Figure 1, a first one or more cameras 11 in the form of a pair of RGB cameras are used to collect measurements of the number of flowers on fruit trees within an orchard (or portion of an orchard), as well as the flower maturity. A second one or more cameras 12 is used to collect measurements of the number of fruit on the fruit trees, as well as the size of the fruit. The second one or more cameras 12 comprises at least a ToF camera to measure the fruit size. The ToF camera could also be used to collect measurements of the number of fruit, although an RGB camera (such as one from the first one or more cameras 11) could also be used for this purpose.

[0071] In the embodiment of the invention illustrated in Figure 1, the measurements collected by the first one or more cameras 11 and the second one or more cameras 12 are in the form of video footage. This video footage is analysed using a local computer 13 in order to provide data regarding the number and maturity of flowers, as well as the number and size of fruit. This may be done to reduce the size of the data to be transmitted via the Cloud 14 to a computing device 15 located remotely to the local computer 13.

[0072] Temperature sensors 16 located within the orchard are used to collect measurements of the ambient temperature. These temperature measurements are also transmitted via the Cloud 14 to the processing device 15.

[0073] Finally, as the fruit matures on the trees, measurements of the dry matter content of the fruit using an NIRS scanner 17. Measurements of the dry matter content of the fruit are transmitted via the Cloud 14 to the processing device 15.

[0074] The computing device 15 comprises a server, one or more machine learning modules and electronic storage in the form of a database 18. Based on the data received from the Cloud 14, the computing device calculates an estimate of the harvest time 19 of the fruit and an estimate of the fruit load 20.

[0075] In the present invention, while an initial estimate of the harvest time 19 and the fruit load 20 can potentially be calculated based on a single measurement of the various parameters, the first one or more cameras 11, the second one or more cameras 12, the temperature sensors 16 and the NIRS scanner 17 are used to collect one or more subsequent measurements of the various parameters. Data collected from these subsequent measurements is transmitted via the Cloud 14 to the computing device 15 so that new estimates of the harvest time 19 and the fruit load 20 can be determined. The new estimates may be updated estimates of the original harvest time 19 and fruit load 20 based on additional data collected as the flowers and/or fruit mature. Alternatively (or in addition to), the updated estimates may be estimates of the harvest time 19 and fruit load 20 related to subsequent flowering events in the orchard.

[0076] Figure 2 illustrates an apparatus 21 for collecting measurements in a method and system for estimating tree fruit load and harvesting time according to an embodiment of the present invention.

[0077] The apparatus 21 illustrated in Figure 2 is a powered vehicle configured to be driven between adjacent rows of trees within an orchard. A pair of RGB cameras 11 is mounted at 180° to each other so that the RGB cameras are directed at approximately 90° to the direction of the travel of the vehicle 21. The cameras 11 capture images relating to the number and maturity of flowers in the trees on either side of the vehicle 21 as the vehicle 21 is driven between adjacent rows of trees.

[0078] A ToF camera 12 is also mounted to the vehicle 21, with the ToF camera 12 being configured to collect measurements of the number and/or size of fruit as the vehicle 21 is driven past trees in the orchard.

[0079] Also mounted to the vehicle is a computer 13 configured to process data received from the cameras 11, 12 to determine the measurements of the respective parameters.

[0080] Figure 3 illustrates a map 22 of fruit load density in an orchard generated using the method and system for estimating tree fruit load and harvesting time according to an embodiment of the present invention. The colours applied to sections of the orchard in the map 22 represent the fruit load at that location. Areas shown in red have relatively high fruit density, areas in yellow have moderate fruit density, while areas in green have a relatively low fruit density. This information allows for improved or effective farm management in terms of allocating adequate harvesting resources to different parts of the orchard based on the fruit load and/or identifying (and optionally implementing management practices) underperforming areas within the orchard.

[0081] Figure 4 illustrates a fruit size estimation generated using the method and system for estimating tree fruit load and harvesting time according to an embodiment of the present invention.

[0082] In Figure 4, a graph 23 of fruit size distribution is illustrated. It is envisaged that an estimate of the harvest time may be based on the fruit size distribution reaching a predetermined distribution and/or predetermined median and/or a predetermined mean.

[0083] In the present specification and claims (if any), the word 'comprising' and its derivatives including 'comprises'and 'comprise'include each of the stated integers but does not exclude the inclusion of one or more further integers.

[0084] Reference throughout this specification to 'one embodiment' or 'an embodiment' means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearance of the phrases 'in one embodiment' or 'in an embodiment' in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more combinations.

[0085] In compliance with the statute, the invention has been described in language more or less specific to structural or methodical features. It is to be understood that the invention is not limited to specific features shown or described since the means herein described comprises preferred forms of putting the invention into effect. The invention is, therefore, claimed in any of its forms or modifications within the proper scope of the appended claims (if any) appropriately interpreted by those skilled in the art.

Claims (5)

1. A method for estimating tree fruit load and harvesting time, the method comprising the steps of: a) Collecting, using a first one or more cameras, an initial measurement of a number and/or maturity stage of flowers on a plurality of fruit trees and, on one or more subsequent occasions, a subsequent measurement of the number and/or maturity stage of flowers on the plurality of fruit trees; b) Collecting, using a second one or more cameras, an initial measurement of a number and/or size of fruit on the plurality of fruit trees and, on one or more subsequent occasions, a subsequent measurement of the number and/or size of fruit on the plurality of fruit trees; c) Collecting, using one or more temperature sensors, ambient temperature measurements at one or more locations among the plurality of fruit trees; d) Calculating, based on the initial measurement and the subsequent measurement of the number of flowers, the initial measurement and the subsequent measurement of the number and/or size of fruit and the ambient temperature measurements the tree fruit load in the plurality of fruit trees and the harvesting time of at least a portion of the fruit.

2. A method according to claim 1, wherein the first one or more cameras or the second one or more cameras include at least one RGB camera.

3. A method according to claim 1 or claim 2 wherein the first one or more cameras or the second one or more cameras include at least one depth camera.

4. A method according to any one of the preceding claims wherein one or more measurements of the dry matter content or SSC of the fruit are also used to calculate the tree fruit load in the plurality of fruit trees and the harvesting time of at least a portion of the fruit.

5. A system for estimating tree fruit load and harvesting time, the system comprising:

A first one or more cameras configured to collect initial and subsequent measurements of a number and/or maturity stage of flowers on a plurality of fruit trees;

A second one or more cameras configured to collect initial and subsequent measurements of a number and/or size of fruit on the plurality of fruits trees;

One or more temperature sensors configured to collect ambient temperature measurements at one or more locations among the plurality of fruit trees; and

A calculation portion configured to receive the measurements from the first one or more cameras, the second one or more cameras and the one or more temperature sensors, wherein the calculation portion is configured to calculate the tree fruit load in the plurality of fruit trees and the harvesting time of at least a portion of the fruit based on a combination of the initial measurement and the subsequent measurements of the number of flowers and/or the flower maturity, the initial measurement and the subsequent measurements of the number and/or size of fruit and the ambient temperature measurements.