NL2029485B1

NL2029485B1 - Agricultural operations monitoring system

Info

Publication number: NL2029485B1
Application number: NL2029485A
Authority: NL
Inventors: Hendricus Persoon Stephanus; Henricus Sebastiaan Zwinkels Cornelis
Original assignee: Novum Est B V; Cohese Holding B V
Priority date: 2021-10-21
Filing date: 2021-10-21
Publication date: 2023-05-16

Abstract

Method for monitoring operations in a cultivation environment, such as a greenhouse, comprising the steps of providing a worker tracking device; determining a location of the 5 worker tracking device with respect to the cultivation environment, wherein the worker tracking device comprises a wrist movement measurement device and that the method further comprises the steps of measuring wrist movement data with the wrist movement measurement device during repetitive agricultural operations; and identifying the repetitive agricultural operations performed with the wrist movement on the basis of the measured wrist 10 movement data and pre-stored operation data.

Description

P35247NLOO/MBA/RR

Title: Agricultural operations monitoring system

Field of the invention

The present invention relates to method for monitoring operations in a cultivation environment, such as a greenhouse, an agricultural operations monitoring system, a cultivation environment provided with the agricultural operations monitoring system and a non-volatile storage medium for an agricultural operations monitoring system.

Background of the invention

In agriculture, for example in horticulture, various time recording systems are known for keeping track of a worker's activities. Such systems are configured to register the time spent by a worker on the basis of manual inputs and/or by reading a tag when the worker starts or stops performing a particular activity. The registered time between starts and stops may be linked to the worker to determine worker productivity.

Other measures of yield, such as amount of product harvested are often determined retrospectively on the basis of sales numbers and/or may be determined by weighing or counting products after harvesting.

Analysis is often performed batch-wise, i.e. per order or per patch of land as products are usually harvested in batches and then processed together, e.g. packaged at a central point in or nearby the cultivation sub-environment. Harvested crops may then be counted or weighted at the central location.

As a result, there is only a rough picture of the production or yield per worker or per batch. Variations in production between patches of land, or within a patch of land are therefore not known, especially not within a batch of harvested crops. The same applies to the provided insight per worker, the type of activity of a worker, or the production variations of a worker over time.

It would be possible to separately administrate all operations performed by a worker and/or the crop individually. For example by harvesting and weighing the harvested crop separately for each batch, but it has been found that for providing practically valuable insights, the process and infrastructure for doing so would be too expensive and too many administrative labour would be necessary.

Therefore, information provided with current activity registration systems remains relatively course, fragmented and lagging in time as productivity numbers are determined after harvesting, after time spent by the workers has been registered and a after the harvested batch of products has been weighted. Thus, a relatively large number of manual actions is required and the human factor may inevitably lead to mistakes.

SG10201502246PA discloses an agricultural worker management system comprising a data acquisition unit having location information associated with a worker. An IMU may be provided to supplement GPS location tracking of the worker using sensor fusion techniques.

A worker is assigned a selected task to perform based on at least one management rule and location information. Upon performing the task, information may be received from a load sensor on a receptacle used to contain the agricultural product to determine an amount of agricultural product being handled by the worker, such as fertilizer, pesticide or harvested produce to indicate the performance of the worker on the assigned task.

A disadvantage of the system disclosed in SG10201502246PA is that tasks need to be assigned to workers based on pre-defined management rules, whereas the worker may perform other tasks, such as a non-assigned task. Further, performance indication relies on an external load sensor for weighing the harvested crop. Harvested crop may disappear after harvesting, i.e. before weighing of the harvested crop, causing inaccuracies in calculated production numbers.

As such, presently no user-friendly systems are known for reliable and user-friendly monitoring of operations in a cultivation environment.

Object of the invention

It is therefore an object of the present invention to provide a method for monitoring operations in a cultivation environment that lacks one or more of the disadvantages mentioned above, or at least to provide an alternative method, for example to provide a method that can be applied more conveniently or that provides for very precise, real-time monitoring of agricultural operations, with respect to specific patches of land and with respect to specific workers.

The present invention

The present invention provides a method for monitoring operations in a cultivation environment according to claim 1. The method comprises the steps of providing a worker tracking device and determining a location of the worker tracking device with respect to the cultivation environment;

The worker tracking device comprises a wrist movement measurement device and the method further comprises the steps of measuring wrist movement data with the wrist movement measurement device during repetitive agricultural operations and identifying the repetitive agricultural operations performed with the wrist movement on the basis of the measured wrist movement data and pre-stored operation data.

The applicant has found that wrist movements may exhibit a movement path that depends on the agricultural operation performed by a worker and that can be distinguished relatively clearly for different types of agricultural operations. Therefore, wrist movements may provide a relatively clear indication of the operation performed.

As repetitive operations are performed, it has been found to be possible to distil common features from the wrist movement data in such a way that operation data can be stored on the basis of which agricultural operations may be identified.

By tracking wrist movements, specific operations may be identified and distinguished from each other on the basis of wrist movement data, whereby the determined location provides information where the operation is performed. Having the combination of wrist movement data and determined location provides several advantages:

Firstly, insights may be provided into the productivity of a worker and/or into locally varying conditions in the cultivation environment. Not only a number of repetitions is determined automatically, but also different agricultural operation types may be distinguished from each other, and a location thereof may be determined.

Therewith, variations in productivity of a worker over time or with location may become apparent, and multiple types of agricultural activities may be monitored, potentially without using external and central sensors as in the prior art.

Also, for agricultural operations that would otherwise not lead to a centrally measurable result, such as the cutting off parts of the crop, e.g. leaves, that locally fall to the ground of the cultivation environment could only be measured by means of time or batch number. It may now be possible to perform monitoring of the actual number and type of operations performed, instead of monitoring based on time or estimated crop numbers.

By measuring actual wrist movement while an agricultural operation is being performed, the agricultural operations may be identified relatively quickly, such that near-real- time monitoring of operations and locations may be possible. Monitoring may be possible before finishing the agricultural operation on all targeted crop, for example before competition of a whole a batch of harvested product and while the repetitive actions are being performed.

Further, due to the combination of operation and location other valuable information may potentially be derived, such as type of crop, age of the crop, and environmental conditions at the site of the crop, for example when the agricultural operations are only possible for a certain type or age of crop, or when the agricultural operations are performed differently on such crop.

In particular, when performing an agricultural operation locally, such as watering, fertilising, applying pesticides, or cutting off crop parts, the advantageous combination of identification of the agricultural operation and the determined location may provide valuable insights in the conditions of certain crop within the cultivation environment, for example when pests are occurring more frequently at specific crop, or when bad leaves are occurring more frequently at specific locations.

As such, the agricultural operations performed by a worker can be monitored relatively precisely and the invention may enable growers to truly apply precision farming in their cultivation environment.

The cultivation environment may be any environment in which crop can be cultivated.

The cultivation environment may be a horticulture environment, for example a greenhouse.

As the agricultural operation is repetitive, the same agricultural operation may be performed multiple times in succession, the same agricultural operation type may be performed on multiple crop. As the movement is a repetitive movement, the movement does not have to be exactly the same, but may instead still be identified if variations are present.

The agricultural operation may be performed on each crop. For example, the repetitive operation may typically be performed at least 500 times in a row by a worker. The repetitive operation may be performed once on each crop in the cultivation environment during cultivation, or once on each crop in a sub-environment in the cultivation environment.

The repetitive agricultural operations may be identified relatively quickly, for example a repetitive operation may be identified the same agricultural operation is performed another time in succession.

The determined location may for example be related to a specific cultivation environment, such as a patch of land, or a sub-environment of the cultivation environment, e.g. a specific section or path number in a greenhouse.

The worker tracking device may be configured to determine the location on the basis of global navigation satellite systems, such as GPS or other GNSS systems. The worker tracking device may alternatively or additionally be configured to determine the location based on signals from transmitters placed in or near the cultivation environment, e.g. via triangulation of such signals.

The wrist movement measurement device may be configured to be worn on a wrist, e.g. as a watch. By wearing on a wrist, the wrist movement measurement device may be worn comfortably. In addition or as alternative, the wrist movement measurement device may be worn at a different location on the forearm, or attached to the hand of a worker. The wrist movement measurement device may be worn or carried such that movements of the wrist of monitored worker may be measured.

The wrist movement measurement device may comprise an inertial measurement unit configured to measure wrist movement data on the basis of wrist movements. Alternatively,

the wrist movement measurement device may comprise other sensors, such as optical

Sensors.

Wrist movements, as used herein, may be the movements of a wrist with respect to the cultivation environment, i.e. as a result of, inter alia, movements of an under arm due to rotation around the elbow and shoulder joints. Movements of the wrist with respect to the cultivation environment may also be induced due to movement of the body of a worker monitored with the wrist movement measurement device, for example during walking.

The wrist movement data may is representative for the wrist movements, and may comprise translation movements and/or rotation movements around multiple axes, e.g. three or six axes.

The wrist movement measurement device may comprise at least one of a three-axis accelerometer, a three-axis magnetometer, a three-axis gyroscope and/or a barometer, wherein the wrist movement data includes one or more of accelerometer data, magnetometer data, gyroscope data and/or barometer data.

The repetitive action may be identified by an identification unit. The identification unit may, for example be a computer system, such as a cloud computing system. This way, a large amount of computational power may be available.

The identification unit may be configured to identify repetitive agricultural operations on the basis of the measured wrist movement data and pre-stored operation data. The pre- stored operation may comprise all types of data that can be stored, and thus may not only include movement data or characteristics thereof, but may also comprise a computer programme or an algorithm.

In an embodiment, the step of identifying the repetitive agricultural operation on the basis of the measured wrist movement data and pre-stored operation data is performed by an identification unit comprising an artificial intelligence algorithm.

The identification unit may comprise an artificial intelligence algorithm. The identification unit may be configured to be trained with machine learning. The artificial intelligence algorithm may comprise a neural network, support vector machine and/or hidden markov model In the past, it has been found to be difficult to identify agricultural operations, as such operations may be performed differently depending on a worker, and even a worker may perform the same agricultural operation differently using a limb movement that may vary from time to time. Therefore, identification of agricultural operations by simply comparing wrist movement data with earlier movement data on the basis of based on human-made comparison criteria does not always lead to the desired recognition accuracy.

It has been found that an artificial intelligence algorithm may be particularly suited for identifying repetitive agricultural operations on the basis of wrist movements. As machine learning may be able to identify agricultural operations on the basis of criteria that may not be noticed by humans, identification may be performed relatively accurately.

As the agricultural operations are repetitive, it may be relatively easy to measure relatively large numbers of wrist movement data on the basis of which the machine learning can be trained. This way, the identification unit may be relatively reliable in identifying repetitive agricultural operations.

Further, by training the identification unit, identification accuracy of agricultural operations may be improved further.

In addition, it may be possible to train the identification unit with new types of agricultural operations, so that the pre-stored operation data can be updated to be able to also identify new agricultural operations.

In an embodiment, the identification unit may be configured to identify the repetitive agricultural operations on the basis of both the determined location, the measured wrist movement data and the pre-stored operation data. For example, when different agricultural operations would lead to similar wrist movement data, the agricultural operations may be distinguished on the basis of the determined location.

The worker tracking may be configured to transmit the wrist movement data to the identification unit, for example via a wireless internet connection. Wrist movement data may be transmitted as raw sensor data, or be pre-processed. Alternatively, the identification unit may be integrated in the worker tracking device. In addition to the wrist movement data, the determined location and/or other data, such as time, may be transmitted.

The method may comprise recording the time at which the location was determined and/or at which the wrist movement data was measured. For example, the worker tracker may be configured to record time and to transmit the recorded time to the identification unit.

Recording time may allow for more accurate identification of performed agricultural operations.

In an embodiment, the worker tracker is provided with a real-time system clock that records the time since boot of the worker tracker. As such, a relatively monotonous and reliable clock may be provided, even when the worker tracker in a power saving mode. The real-time system clock may be used to record an exact time interval between measurements of the wrist movement measurement device. It has been found that an ordinary time value, such as can usually be used by apps in an operating system, may be insufficiently accurate for identifying agricultural operations, in particular using artificial intelligence. By using a real- time system clock, the identification may be more reliable.

In an embodiment, the repetitive agricultural operations comprise processing of a harvested crop cultivated in the cultivation environment. This may include, for example,

packing operations, such as placing the harvested crop in a package or transportation container. By monitoring the processed harvested crop, e.g. processed harvested crop, an agricultural yield, such as amount of harvested crop, may be determined without weighing the harvested crop.

In an embodiment, the repetitive agricultural operations comprise operations performed on a cultivated crop. This may include all cultivation operations carried out on or to the crop. Previously, it would be necessary to manually indicate a location and/or an agricultural operation in order to perform monitoring. By determining location and identifying agricultural operation, it may become possible to perform monitoring of operations performed on a cultivated crop while simultaneously performing the operation in the cultivation environment, potentially without manual inputs.

In this way, for example, it can be determined at which location an operation was performed on a crop. In particular, if the location is sufficiently specific, the exact location of an agricultural operation on an individual plant or other crop can be monitored for improved precision farming.

In an embodiment, the pre-stored operation data is representative for harvesting operations performed on a cultivated crop, such as cutting, trimming or picking crop, wherein the agricultural operations are harvesting operations. By monitoring harvesting operations, a production of the cultivation environment can be determined, using the location such that local variations can be compared.

In an embodiment, the method further comprises the step of determining a location- specific performance in a predetermined period by counting a number of the identified agricultural operations in the predetermined period. A central data display may be configured to determine and/or to display the location specific performance.

A predetermine period may be pre-programmed or provided by a worker or another user. The predefined period is a time period, which may be a day or a part of a day, or a week, or a month, or a year. The predefined period can also be real time or a period from the past date/time to the present.

In a further embodiment, the method comprises the step of comparing determined location-specific performances of different locations in the predetermined period.

In an embodiment, the determined location-specific performance represents a location-specific quantity of harvested crop.

In an embodiment, the location-specific performance is updated in real time during the repetitive agricultural operations on the basis of the measured wrist movement data.

In an embodiment, the method further comprises the steps of subdividing the cultivation environment into multiple cultivation sub-environments to be monitored; and associating the determined location with one of the multiple cultivation sub-environments to be monitored; wherein the step of determining a location-specific performance comprises counting a number of the identified agricultural operations in the associated cultivation sub- environment. The identification unit, a central data display and/or another system may be configured to perform this step.

This way, the sub-environments of the cultivation environment may be compared with respect to each other. For example, production, e.g. harvested crop may be determined for each sub-environment, such that local differences in the cultivation environment may be evaluated by comparing sub-environments.

A sub-environment may for example be a section of a greenhouse, e.g. a path number, or a specific section of a patch of land. Advantageously, the sub-environments are virtual sub-environments. The virtual sub-environments may be smaller than the actual physical sub-environments. As such, monitoring may be relatively location-specific for a relatively small virtual sub-environment and may be performed more precise than with a batch in the prior art, which was usually defined by physical constraints of the cultivation environment.

Each sub-environment may have an equal size, for example the size of the cultivation environment divided by the number of sub-environments.

In an embodiment, the step of subdividing the cultivation sub-environment into multiple cultivation sub-environments to be monitored comprises the steps of storing perimeter location points of the cultivation environment and interpolating the cultivation environment into multiple cultivation sub-environments based on the stored perimeter location points, wherein the identified agricultural operations are counted in the interpolated multiple cultivation sub-environments. This way, sub-environments may be programmed relatively easily.

Perimeter location points may be location points located on the perimeter of the cultivation environment, for example at boundaries of a patch of land or at the facades of a greenhouse.

The perimeter points may be stored by determining a location with the worker tracker upon manual triggering of a storage mode.

In an embodiment, the cultivation sub-environments have a surface area of less than 5 m2. Owing to the invention, relatively precise monitoring of agricultural operations may be possible. The minimum surface area may be limited by the precision of the determined location only. As such, insight may be provided into cultivation performance at a relatively high level of detail.

In an embodiment, agricultural operations performed on an individual crop can be identified. In this way, the condition and history of an individual crop may be monitored.

In an embodiment, the method further comprises the steps of providing a central data display unit; transmitting the determined location and the movement data and/or the identified repetitive agricultural operations to the central data display unit; classifying the identified repetitive operations according to worker, agricultural operation type and/or sub-environment; displaying, on the central data display unit, the classified identified repetitive operations of the at least one crop or group of crops.

By providing a central data display unit, data from multiple wrist movement measurement devices may be combined in one display. The central data display unit may for example be a screen providing access to a cloud portal, a portable device and/or a local computer. A central data display unit may be configured to display the identified repetitive operations according to worker, agricultural operation type, and/or sub-environment on the basis of determined location, wrist movement data, identified agricultural operation, worker information, transportation information and/or other criteria.

The worker tracking device may be configured to transmit the wrist movement data to the central data display unit and/or to the identification unit, for example via a wireless internet connection. Wrist movement data may be transmitted as raw sensor data, or be pre- processed. The central data display unit may be configured to process the determined location and/or the wrist movement data.

The identification unit may be configured to transmit identified agricultural operations and locations to the central data display unit. The central data display unit may comprise the identification unit.

The central data display unit may be configured to display a heat map of harvested crops. On the heat map, the identified agricultural operations and a location thereof may be shown. The 'heat map' may additionally or alternatively be displayed for other relevant data such as diseases and pests.

In an embodiment, the worker tracking device comprises a location selector, such as a touch screen, and wherein the step of determining the location includes selecting one of the cultivation sub-environments with the location selector.

In addition or alternative to the determination of the location using GNSS, the location may be determined by manual input a location. The location selector may comprise a graphic user interface (GUI) through which a location may be selected.

In an embodiment, the worker tracking device is configured to determine an approximate location, to present a list of precise locations on the basis of the determined approximate location, and to receive a selected list entry from the list of precise locations to determine the location. For example, a list may be presented by audio or graphically on a

GUI, whereby the worker may conveniently select a precise location. By adapting the presented list based on the approximate location, a worker may conveniently choose a precise location without scrolling through all possible locations. This allows smart position selection, such that a location may be determined quickly and efficiently, even if a technical determination of the location, e.g. on the basis of GNSS, would be insufficiently accurate.

In an embodiment, the wrist movement measurement device comprises an information reader, such as an NFC reader or QR reader, wherein the process further comprises step of: reading location information using the information reader, from an information tag attached to the location, wherein the step of determining the location is performed in dependence on the location information read; and/or reading worker information using the information reader, from an information tag associated with the worker, wherein the step of comparing the measured movement data with pre-stored operation data to determine repetitive agricultural operations performed with the wrist movement further comprises determining an executing worker and/or reading transportation information using the information reader, from a transportation tag attached to a transportation container in which objects are placed or from which objects are taken out during the agricultural operation.

The information reader may also be another type of RFID reader and/or optical reader.

The information reader may be configured to read information tags arranged in the cultivation environment.

The location selector may be operatively connected to the information reader for selection of the location on the basis of the read location information.

The wrist movement measurement device may be configured to transmit location information, worker information and/or transportation information to the identification unit and/or to the central data display, for example together with the wrist movement data.

In particular, when the worker tracker is carried by a worker, for example worn on the wrist, a reader may allow reading an NFC tag or QR tag before, after and/or during performing agricultural operations to add information to the identified agricultural operations relatively easily. The wrist movement measurement device may be configured to transmit this information to an identification unit and/or central data display unit.

In an embodiment, the wrist movement measurement device comprises a speech recognition unit, further comprising the steps of recording a message with the speech recognition unit; coupling the recorded message to the determined location of the wrist movement measurement device with respect to the cultivation environment and/or to the measured movement data.

Using speech recognition, commands and/or observations may be provided to the wrist movement measurement device by a worker.

The location selector may be operatively connected to the speech recognition unit for spoken selection of the location.

Speech recognition is also relatively suitable for making observations in the cultivation environment in relation to the determined location. As such, speech recognition may be used to record crop observations such as counts, pests and/or diseases.

The speech recognition unit may be configured to record messages and to transmit the recorded messages, for example to a central data display unit.

In an embodiment, the method further comprises the steps of labelling the measured wrist movement data with an agricultural operation type, training an artificial intelligence algorithm to identify agricultural operations on the basis of the labelled measured wrist movement data and storing the trained artificial intelligence algorithm in the pre-stored operation data.

Labelling the measured wrist movement data may be performed centrally, for example in a cloud system, on the central data display unit, or locally on the worker tracker, for example by pressing a button on the wrist movement measurement device, via a GUI and/or via speech recognition.

Measured wrist movement data may be temporarily stored to be labelled and then be provided to the artificial intelligence algorithm with the label to train the artificial intelligence algorithm, which may be subsequently stored in the pre-stored operation data (e.g. in the identification unit).

In order to train the identification unit, the wrist movement data is labelled with an agricultural operation type at moments when each individual repetitive agricultural operations is performed.

As such, machine learning may be performed by training with the labelled wrist movement data

The measured wrist movement data may be labelled through speech recognition. This way, it may not be necessary to label the respective data afterwards, but labelling may be performed simultaneously when the operations are performed by linking a word to a specific label (such as "flower" for harvesting a flower).

According to another aspect, the invention provides a worker tracking device for performing the method according to any of the embodiments described herein, for example according to any of the claims 1-15.

The worker tracking device may comprise a wrist movement measurement device configured to measure wrist movement data during repetitive agricultural operations; and a location tracker configured to determine a location of the worker tracking device with respect to the cultivation environment.

The worker tracking device may be configured to transmit the determined location and the measure wrist movement data to an identification unit provided with pre-stored operation data and configured to identify the repetitive agricultural operations performed with the wrist movement on the basis of the measured wrist movement data and pre-stored operation data.

The worker tracking device may provide similar benefits as with the method, as explained herein. The worker tracking device may be configured to perform embodiments of the method, as explained herein.

According to another aspect, the invention provides an agricultural operations monitoring system, comprising the worker tracking device according to any of the embodiments described herein, for example to claim 16; data receivers installed in the cultivation environment and configured to receive movement data from the wrist movement measuring device; wherein the wrist movement sensor is equipped to send movement data to the data receiver after and/or during repetitive agricultural operations.

The data receivers may be WIFI-receivers, but other receivers may also be possible.

In large-scale situations mobile internet receivers may be provided, such as LTE or 5G- receivers.

According to another aspect, the invention provides a cultivation environment, for example a greenhouse, provided with the agricultural operations monitoring system according to any of the embodiments described herein, for example according to claim 17.

According to another aspect, the invention provides a non-volatile storage medium for an agricultural operations monitoring system according to any embodiment described herein, for example according to claim 17, storing a machine readable instruction to perform the method according to any of the embodiments described herein, for example according to claims 1-15.

The agricultural operations monitoring system, the cultivation environment, and the non-volatile storage medium for an agricultural operations monitoring system may provide similar benefits as with the method, as explained herein.

Brief description of drawings

Further characteristics of the invention will be explained below, with reference to embodiments, which are displayed in the appended drawings, in which:

Figure 1 schematically depicts a perspective view of a cultivation environment in which an embodiment of the method according to the present invention is performed;

Figure 2 schematically depicts a perspective view a greenhouse provided with an agricultural operations monitoring system according to an embodiment of the present invention; and

Figure 3 schematically depicts an agricultural operations monitoring system comprising a worker tracker used in the method of the embodiment of Figure 1.

Detailed description of embodiments

Figure 1 schematically depicts a perspective view of a cultivation environment 90 divided in multiple sections 91 that form sub-environments and are provided with crop 92.

Workers 93 may walk between or in the sections 91 to perform agricultural operations, for example cutting off parts of the crop 92 or harvesting the crop 92 by picking.

The method comprises the steps of providing a worker tracking device 1, comprising a wrist movement measurement device 2 and a location tracker 3.

The wrist movement measurement device 2 is configured to be worn on a wrist of a worker 93, e.g. as a watch, and comprises an inertial measurement unit 21 configured to measure wrist movement data on the basis of wrist movements, as depicted in Fig. 3.. In particular, the inertial measurement unit 21 of the wrist movement measurement device 2 comprises a three-axis accelerometer, a three-axis magnetometer, a three-axis gyroscope and a barometer.

The location tracker 3 comprises a GPS-receiver configured to determine a location of the wrist movement measurement device 1.

The worker tracking device 1 is provided with a real-time system clock 11 that records the time since boot of the worker tracker 1. The worker tracker 1 is configured to record the time at which the location was determined and/or at which the wrist movement data was measured and to transmit the recorded time to the identification unit 4.

The wrist movement data includes accelerometer data, magnetometer data, gyroscope data and barometer data and is representative for the wrist movements, and comprises translation movements and rotation movements around six axes.

As shown in Fig. 3, an external identification unit 4 is provided, which is an external cloud computing system. The identification unit 4 is configured to identify repetitive agricultural operations on the basis of the measured wrist movement data and pre-stored operation data, and optionally the location determined by the location tracker 3. The identification unit 4 comprises an artificial intelligence algorithm stored in the pre-stored operation data.

The artificial intelligence algorithm comprises one or more of a neural network, support vector machine and a hidden markov model, which may be used jointly to improve identification.

The artificial intelligence algorithm is trained with agricultural operations, such as harvesting operations performed on a cultivated crop, such that the pre-stored agricultural operation data are presentative for harvesting operations performed on a cultivated crop.

The worker tracking device 1 is configured to transmit the wrist movement data and the determined location to the identification unit 4 via a wireless internet connection as raw sensor data.

A central data display unit 5 is provided, configured to receive the identified agricultural operations from the identification unit 4, to determine a location-specific performance in a predetermined period by counting a number of the identified agricultural operations in the predetermined period and to display the location specific performance representing a location-specific quantity of harvested crop 92.

The predetermined period is pre-programmed by a worker 93 or another user, such as a supervisor. The predefined period is a time period, in this case a day. The central data display 5 is configured to determine location-specific performances of different sub- environments 91 in the predetermined period.

The central data display unit 5 is configured to subdivide the cultivation environment 90 into multiple equally-sized virtual cultivation sub-environments 21’ that are smaller than the physical sub-environments 91, to associate a determined location with one of the multiple virtual cultivation sub-environments 21’ and to determining a location-specific performance by counting a number of the identified agricultural operations in the associated virtual cultivation sub-environment 91°. The virtual cultivation sub-environments have a surface area of less than 5 m2.

The central data display unit 5 is configured to display a heat map of harvested crop and to display the identified repetitive operations according to worker, agricultural operation type, and/or sub-environment on the basis of determined location, wrist movement data, identified agricultural operation, worker information, transportation information and/or other criteria.

In use, agricultural operations may be performed by the worker 93, such as packing operations, such as placing the harvested crop 92 in a package or transportation container, or cutting, trimming or picking crop 92.

The location of the worker tracking device 1 is determined with respect to the cultivation environment 91 with the location tracker 3 to monitor a location of the worker 93.

The location is related to a specific patch of land, or a sub-environment 91 of the cultivation environment 90, e.g. a specific path number in a greenhouse.

Wrist movement data is measured with the wrist movement measurement device 2 during repetitive agricultural operations and is thus is representative for the wrist movements.

The wrist movement data, determined location and recorded time are transmitted to the identification unit 4 via a wireless internet connection. A repetitive operation may be repeated for each crop 92, for example at least 500 times when 500 crop are provided in the sub-environment 91. Repetitive agricultural operations performed with the wrist movement are identified by the identification unit 4 on the basis of the recorded time, measured wrist movement data and pre-stored operation data, and optionally, the determined location.

The determined location and the identified repetitive agricultural operations are then transmitted to the central data display unit 5 by the identification unit 4. Alternatively, the central data display unit 5 and the identification unit 4 may be combined and/or the measured wrist movement data may be transmitted. Further, the measured wrist movement data may be transmitted to the identification unit 4 and/or central data display unit 5 via data receivers 7 or other wireless or wired connections, as depicted in Fig. 3. Although the arrows are indicated in one direction, transmitting and communication may be performed in both directions. On the central data display unit 5, the identified repetitive operations are displayed near-real time according to worker, agricultural operation type, location and/or sub- environment for example via a heat map on which each individual crop can be identified.

Further, a number of harvested crop is counted for each virtual sub-environment and displayed by the central data display 5, such that local differences in the cultivation environment may be evaluated by comparing sub-environments.

For training the artificial intelligence algorithm, wrist movement measurement data may be labelled manually on the central data display unit 5, by providing an agricultural operation type to entries in the wrist movement data representing a respective agricultural operation. The trained artificial intelligence algorithm may then be stored in the pre-stored operation data.

Figure 2 schematically depicts a perspective view a greenhouse provided with an agricultural operations monitoring system according to an embodiment of the present invention. The agricultural operations monitoring system 6 comprises a worker tracker having a wrist movement measuring device 2 worn by a worker 93, data receivers 7 installed in the cultivation environment 90 and configured to receive movement data from the wrist movement measuring device 2 and to transmit the wrist movement data to central data display unit 5.

The wrist movement measurement device 1 is configured to transmit wrist movement data to the data receivers 7 after and/or during repetitive agricultural operations.

The data receivers 7 are WIFI-receivers, but other receivers may also be possible. In large-scale situations mobile internet receivers may be provided, such as LTE or 5G- receivers.

The central data display unit 5 is shown to be located in the cultivation environment 90, but may also be located elsewhere, for example in an office. Alternatively or additionally, the central data display unit may be cloud computing system or a portable device. The central data display unit 5 is configured to subdivide the cultivation sub-environment into multiple cultivation sub-environments to be monitored 91’ by storing perimeter location points 94 of the cultivation environment 90, i.e. at the outer ends of pathway 95 nearby the facades of the greenhouse, and interpolating the cultivation environment 90 into multiple cultivation sub- environments 91 based on the stored perimeter location points. Additionally or alternatively, the cultivation environment 90 may be interpolated into smaller cultivation sub-environments 91’ that are virtually separated.

The perimeter points are stored by determining a location with the worker tracker 1 upon manual triggering of a storage mode by the worker 93 by pressing a button

The identified agricultural operations are counted by the central data display unit 5 in each of the interpolated multiple cultivation sub-environments 91 91°.

The wrist movement measurement device 1 comprises an information reader configured to read information tags arranged in the cultivation environment, in this case an

NFC-reader, and a speech recognition unit.

The worker tracking device 1 comprises touch screen with a GUI which forms a location selector (not shown). The worker tracking device 1 is configured to determine an approximate location, to present a list of precise locations on the basis of the determined approximate location on the location selector, and to receive a selected list entry from the list of precise locations to determine the location.

Before, after or during performing agricultural operations, the worker 93 selects one of the cultivation sub-environments 91 in a graphically presented list in the GUI. The list is adapted based on the approximate location, such that a location may be determined even if a technical determination of the location, e.g. on the basis of GNSS, would be insufficiently accurate.

In addition or alternative to selecting a cultivation sub-environment 91, information may be read by passing the information reader along an NFC-tag comprising location information to determine a location. The NFC-tag (not shown) is arranged nearby the pathway

95 for each sub-environment 91. Further, worker information may be read using the information reader, for example from a worker NFC-tag carried by the worker 93, or transportation information may be read from a transportation NFC-tag attached to a transportation container for harvested crop.

Further, a location may be selected by a speech command to the speech recognition unit. Also, observations may be done by the worker 93, for example the presence of pests or diseases, by speaking a message to the speech recognition unit and training of the identification unit may be performed by speaking agricultural operation types to label the wrist movement data.

Besides the shown and described embodiments, numerous variants are possible. For example the dimensions and shapes of the various parts can be altered. Also it is possible to make combinations between advantageous aspects of the shown embodiments.

It should be understood that various changes and modifications to the described embodiments can be made without departing from the scope of the invention, and therefore will be apparent to those skilled in the art. It is therefore intended that such changes and modifications be covered by the appended claims.

Claims

CONCLUSIONS

CLAIMS 1. Method for tracking actions in a cultivation environment, such as a greenhouse, comprising the steps of: - providing an employee tracking device (1) - determining a location of the employee tracking device relative to the cultivation environment (90); characterized in that the employee tracking device comprises a wrist movement measuring device (2) and the method further comprises the steps of: - measuring wrist movement data with the wrist movement measuring device during repetitive agricultural operations; and - identifying the repetitive agricultural operations performed with the wrist movement on the basis of the measured wrist movement data and pre-stored operation data.

The method of claim 1, wherein the repetitive agricultural operations include processing a harvested crop grown in the growing environment.

A method according to claim 1 or 2, wherein the repetitive agricultural operations comprise operations on a cultivated crop, such as clipping, pruning or harvesting of crop.

A method according to any one of the preceding claims, wherein the step of identifying the repetitive agricultural operations on the basis of the measured wrist movement data and pre-stored operation data is performed by an identification unit (4) comprising an artificial intelligence algorithm.

A method according to any one of the preceding claims, further comprising the step of: - determining a location-specific performance in a predetermined period by counting a number of the identified agricultural operations in the predetermined period.

The method of claim 5, wherein the calculated performance represents a location-specific amount of harvested agricultural products.

A method according to claim 5 or 6, wherein the location-specific yield is updated in real time during the repetitive agricultural operations on the basis of the measured wrist movement data.

A method according to any one of claims 5-7, further comprising the steps of: - dividing the cultivation environment into several cultivation sub-environments (91, 91°) to be monitored; and - associating the determined location with one of the multiple cultivation sub-environments to be monitored; wherein the step of determining a location-specific performance comprises counting a number of the identified agricultural operations in the associated crop subenvironment.

A method according to claim 8, wherein the step of dividing the cultivation environment into several cultivation sub-environments to be monitored comprises the steps of: - storing outline points (94) of the cultivation environment; and - based on the stored contour points, interpolating the crop environment in multiple crop subenvironments, counting the identified repetitive agricultural operations in the interpolated multiple crop subenvironments.

A method according to claim 8 or 9, wherein the cultivation sub-environments have an area of less than 5 m2.

A method according to any one of the preceding claims, further comprising the steps of: - providing a central data display unit; - transmitting the determined location and the movement data and/or the identified agricultural operation to the central data display unit; — classifying the identified repetitive agricultural operations by worker, type of agricultural operation and/or cultivation sub-environment; - displaying on the central data display unit the classified identified repetitive agricultural operations on the at least one crop or group of crops.

A method according to any one of the preceding claims, wherein the worker tracking device comprises a location selector, such as a touch screen, and wherein the location determination step comprises selecting one of the growing sub-environments with the location selector.

13. Method as claimed in any of the foregoing claims, wherein the wrist movement measuring device comprises an information reader, such as an NFC reader or QR reader, wherein the method further comprises the step of: - using the information reader to read location information from an on-site confirmed information tag, wherein the location determining step is performed in dependence on the read location information; and/or - reading employee information from an information tag associated with the employee with the information reader, wherein the step of comparing the measured movement data with pre-stored action data to determine repetitive agricultural actions performed with the wrist movement further comprises determining an employee performing and/or - reading out transport information with the information reader from a transport tag applied to a transport container in which objects are placed or from which objects are taken during the agricultural operation, the method further comprising the step of storing the read data with the particular agricultural operation. location information and/or employee information and/or transportation information.

A method according to any one of the preceding claims, wherein the wrist movement measuring device comprises a speech recognition unit, further comprising the steps of - recording a message with the speech recognition unit; - linking the recorded message to the determined location of the wrist movement measuring device in relation to the cultivation environment and/or to the measured movement data.

A method according to any one of the preceding claims, further comprising the steps of: - labeling the measured wrist movement data with a type of agricultural action; — training an artificial intelligence algorithm to identify the farming operations based on the labeled measured wrist motion measurement data; and

- storing the trained artificial intelligence algorithm in the pre-stored operation data.

16. Employee tracking device for performing the method according to one of the preceding claims.

17. Agricultural action tracking system, comprising: - the wrist movement measuring device according to claim 16; - data receivers arranged in the cultivation environment and arranged to receive movement data from the wrist movement measuring device, wherein the wrist movement measuring device is adapted to send movement data to the data receivers after and/or during repetitive agricultural operations.

18. Cultivation environment, for instance a greenhouse, provided with the agricultural action tracking system of claim 17.

A non-volatile storage medium for an agricultural operation tracking system according to claim 17, which stores a machine-readable instruction for performing the method according to any one of claims 1-15.