CA3021755A1

CA3021755A1 - Self-powered remote camera probes for automated horticultural pest detection

Info

Publication number: CA3021755A1
Application number: CA3021755A
Authority: CA
Inventors: David T. Southwell
Original assignee: Individual
Current assignee: Individual
Priority date: 2018-10-22
Filing date: 2018-10-22
Publication date: 2020-04-22
Also published as: WO2020082175A1

Abstract

A remote wireless sensor that operates from harvested energy and communicates with wireless infrasturcture comprising a wireless communications mechanism and antenna, microprocessor, camera, light source, photovoltaic cell, electrical energy storage, connective electronics, an insect trap and a frame to connect the components together.
The device accumulates energy from the environment and periodically activates to take image samples from its trap and wirelessly relay them to servers for analysis. The device is intended to be deployed in indoor horticultural environments to provide automated early warning of pest insects when connected to wireless networks and servers analysing the relayed images for pests. The device is small, lightweight, low cost and designed to be attached directly to the crop being monitored. Using properties of the radio communications signals, individual devices can be automatically located in space.

Description

TITLE: SELF-POWERED REMOTE CAMERA PROBES FOR AUTOMATED
HORTICULTURAL PEST DETECTION
FIELD OF THE INVENTION
[1] The present invention relates to low cost remote camera probes that can be deployed in great numbers in greenhouse facilities to provide early warning of insect pest infestations.
BACKGROUND

[2] In the field of indoor horticulture, insect pests are a continuous threat to the health, yield and quality of a crop growing in a controlled environment. Due to air circulation systems actively rotating air throughout a facility infestations can spread very quickly. The trend towards increasingly large scale greenhouses for reasons of cost efficiency at scale increases the net value of crops at risk of infestation, especially if just a single crop type is grown across the entire facility.

[3] For certain crops that are grown organically there are few options to limit the spread of insect pests once detected and it is necessary to remove affected plants as quickly as possible in the hope of avoiding widespread infestation. It is not uncommon for entire crops to be lost in large growing facilities due to uncontrollable insect populations that were detected too late to contain.

[4] Due to the size of the growing facilities, direct visual inspection of such a large surface of stems, leaves, flowers and produce can be extremely labour intensive and therefore expensive. As crops mature the at-risk investment in terms of time, energy, labour and raw materials increases but so does the difficulty of effectively inspecting the plants for signs of infestation. More mature crops have a larger surface area, more of that area is obscured or hard to access and older plant tissue may be more susceptible to attack.

[5] There is therefore a need for a system that can be cost-effectively deployed to provide automated early detection of pest insects. A system that could directly detect and identify pest insects, their eggs or the damage they cause in a timely fashion could be a valuable line of defence against catastrophic crop loss, but would need to be sensitive, accurate, easy to deploy and inexpensive enough to provide effective coverage across a large physical area on a commercially viable basis.
BRIEF SUMMARY OF INVENTION

[6] The present invention comprises a small, low-cost, self-powered camera probe capable of directly imaging an integrated sticky insect trap or exposed leaf and wirelessly communicating with a back-end comprising standard wireless networking and server infrastructure hosting custom image analysis software.

[7] The probes themselves feature an SOC (System on a Chip ¨ a highly integrated microprocessor complex) equipped with wireless communication, an interface to a digital camera and the ability to control a Light Emitting Diode (LED) light source to illuminate the trap in a controlled way. The probe is powered by an integrated photovoltaic cell, microwave antenna or a primary battery that charges an energy storage device such as a supercapacitor or rechargable secondary battery, or by direct electrical connections. Probe activity is periodically triggered by a timer mechanism.

[8] At one end of the device the body of the probe supports the Printed Circuit Board (PCB) containing all the elements except the photovoltaic cell and the trap.
The camera and LED are oriented towards the trap which is mounted at the other end of the device and the photovoltaic cell is mounted between the two ends.
Correct orientation of the camera and trap may be assured by thin wires that attach the top of the two ends together under slight tension to maintain rigidity. The main body of the probe may be built from stamped sheet metal such as aluminum or plastic or a wire frame.

[9] In operation the energy harvested from the photovoltaic cell accumulates in the storage device. A very low power timer device continuously operates, "waking up"
the SOC at regular intervals. On wake up, the SOC illuminates the trap, captures a still image and wirelessly transmits it to waiting server infrastructure. Once at the server, the image is processed and analysed for content. If the system determines that a new feature of interest (typically a trapped flying insect) has been found, it records the possible infestation in a central database and the system decides if prompt human intervention may be requested.

[10] Each probe has a unique serial number for identification, so an insect warning report can be associated with its originating probe. By analysing various properties of measured radio signals reported from multiple wireless base stations at known positions within the building, automatic location of each probe in all three dimensions is possible, faciliting this process.

[11] The system is designed in such a way that a great many probes could be deployed in a growing facility, allowing high quality coverage of the crop during its entire life cycle.
BRIEF DESCRIPTION OF DRAWINGS

[12] Some embodiments of the present invention are illustrated as an example and are not limited by the figures of the accompanying drawings, in which like references may indicate similar elements and in which:

[13] FIG. 1 ¨ depicts a side elevation of one example of a probe according to various embodiments of the present invention.

[14] FIG. 2 ¨ depicts a top view of one example of a probe according to various embodiments of the present invention.

[15] FIG. 3 ¨ depicts an end elevation of one example of a probe according to various embodiments of the present invention.

[16] FIG. 4A ¨ depicts a perspective representation of a probe using a wire-based frame.

[17] FIG 4B ¨ depicts a perspective representation of a probe using an alternative wire frame configuration suited to attaching the trap with tape.

[18] FIG 5 ¨ depicts a high level electrical schematic of a probe.

[19] FIG 6 ¨ depicts a probe, the wireless access points and downstream infrastructure.
DETAILED DESCRIPTION OF THE INVENTION

[20] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well as the singular forms, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises"
and/or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and/
or groups thereof

[21] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one having ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[22] In describing the invention, it will be understood that a number of techniques and steps are disclosed. Each of these has individual benefit and each can also be used in conjunction with one or more, or in some cases all, of the other disclosed techniques. Accordingly, for the sake of clarity, this description will refrain from repeating every possible combination of the individual steps in an unnecessary fashion. Nevertheless, the specification and claims should be read with the understanding that such combinations are entirely within the scope of the invention and the claims.

[23] New remote pest detection probes and analytics systems are discussed herein.
In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be evident, however, to one skilled in the art that the present invention may be practiced without these specific details.

[24] The present disclosure is to be considered as an exemplification of the invention, and is not intended to limit the invention to the specific embodiments illustrated by the figures or description below.

[25] The present invention will now be described by referencing the appended figures representing preferred embodiments.

[26] FIG 1, FIG 2 and FIG 3 respresent side, plan and end elevations views respectively of the physical configuration of the remote camera probe in one variation of the present invention.

[27] Referring to FIGs 1, 2 & 3:

[28] Printed Circuit Board (PCB) 1 connects all electronic elements of the probe.
All elements except the Photo Voltaic (PV) cells 3 are physically soldered onto PCB
1. PV cells 3 are attached to the floor of the device and are connected by two wires to PCB 1. Multilayer PCB 1 measures approximately 20mm wide by 30mm tall by 0.5mm thick.

[29] Frame 2 supports and interconnects PCB 1, PV cells 3 and trap 4. The frame may be constructed from several different materials:

[30] In one embodiment, the frame is fashioned from stamped and folded aluminum sheet. The thin sheet may be stamped with ridges and folds to increase rigidity. The frame may be attached to the PCB 1 with screws or adhesive tape or glue. The frame may be attached to PV cells 3 with adhesive tape or glue. The frame may be attached to trap 4 using bendable tabs or the adhesive properties of the trap paper itself.

[31] In a second embodiment, the frame is constructed from a molded plastic part.
In this case the PCB 1, PV cells 3 and trap paper 4 may be attached using the same methods as for the aluminum foil frame case, or may be each clipped into place using slots and tabs built into the plastic frame molding.

[32] In the case of these first two construction choices for the frame 2, tension wires 6 may be used to stablise the frame. These wires connect the top two corners of PCB
1 with the top two corners of the part of the frame that supports the trap 4.
These wires are thin filaments which are in tension when installed into place and have the effect of both stiffening the structure and pulling the camera and trap ends of the frame so they are parallel and thus optimally positioned for imaging the entire surface of the trap from the camera avoiding focus errors and trapezoidal image distortion.

[33] In order to improve wireless communication efficiency (data rate, range or energy consumption of the radios), tension wires 6 may be connected as antenna for the integrated radios.

[34] Referring to FIGs 4 A, 4B: In a third embodiment, the frame is constructed of one or more segments of wire 11 that are soldered onto or otherwise attached directly to the PCB 1. The PV cells 3 may be attached in this case to the top of the device and serve as the alignment and stiffening element in place of the tensioning wires used in FIGs 1, 2 and 3. Stainless steel would be a suitable material for wire 11. As shown in FIG 4A, trap 4 could be attached to wire 11 using adhesive tape along each vertical side. Alternatively as shown in FIG 4B, the vertical elements of the wire frame supporting the trap could be angled allowing the rear surface of the trap to be attached with adhesive tape 12.

[35] In each case, the wire frame 11 could be formed in such a way as to clip under compression into holes and slots created in PCB 1, permitting the assembly of the probe from component parts without permanently attaching the PCB 1 with solder, glue or tape.

[36] In the case of a FIG 4B, a stiff card-like trap 4 the wire frame 11 could be designed to engage curved slots cut into the trap 4 to allow the spring of the wire to lock the trap in place.

[37] In the case of wire frame construction as shown in FIGs 4A and 4B the wire frame 11 itself may also serve as the wireless communications antenna for improved radio performance.

[38] Referring again to FIGs 1, 2 & 3:

[39] The trap paper 4 is derived from commercially available insect trap paper, sized for this application. Such trap paper is coated with a sticky material that traps insects 5 that fly or walk onto it. The paper may also contain chemical attractants to improve the rate at which insects are caught. The paper may also be coloured in such a way as to attract targeted insects.

[40] In some indoor horticultural operations, desirable insects performing useful functions such as pollination (example: bees/bumblebees) or preying upon pest species (example: ladybugs) may be deliberately introduced. In these cases, a coarse mesh of wire or plastic filaments may be deployed some distance away from the sticky surface of the trap 4 to prevent desirable species of insect from becoming trapped on it. This strategy works since predators and pollinators are often significantly larger than the pest species the trap 4 is intended for.

[41] The probe is designed to be installed for the duration of the crop's life cycle which means multiple months of continuous operation. During this time the trap paper 4 may be filled with insects or contaminated with obscuring material such as dust and soil and so become unable to continue to further generate useful data. In this case, the system decides to generate a request for human assistance in replacing the paper trap on the probe. This process is facilitated by a convenient mechanism for removing and refreshing the trap paper 4, and also the automatic location system based on radio telemetry.

[42] Referring to FIG 5 we now describe the principle of operation from an electrical perspective:

[43] The probe requires electrical energy to function and this energy is generally harvested from the PV cells 3 but they cannot be relied upon to produce sufficient instantaeous power to operate the SOC 10 and associated peripherals under all light conditions. When installed under the growth canopy or when operating during times of darkness (when insects may be most active) there will certainly be insufficient direct power. The PV cells instead produce energy for the energy storage subsystem 7 which charges up and has the power delivery capacity to operate the probe in short bursts. Even under low light conditions, the PV cell 3 is able to make progress charging the system when it is only used infrequently and periodically. The energy storage system in the preferred embodiment is built around supercapacitors.
Supercapacitors have the advantage of tolerating virtually unlimited numbers of charge/discharge cycles without suffering significant capacity, leakage or internal impedance degradation. Other embodiments may select rechargeable batteries such as those based on Lithium-Ion chemistries. Such batteries have price, volumetric energy and volumetric power density advantages over supercapacitors however they are more volatile and less durable. Supercapacitors also have a significantly higher internal impedance than most battery types which makes it easier to design systems that are intrinsically safer and specifically may be fundamentally unable to malfunction in such a way as to serve as ignition sources. The energy storage subsystem is sized to support operational eletrical requirements in the most compact and cost effective way. In practice this means that during the longest period of darkness the probe must be able to execute a minimum number of bursts to meet plant surveillance requirements. This requirement informs the energy storage capacity and also the probe's self-discharge rate while it is waiting between activity bursts.

[44] An energy management circuit ensuring positive charging from the PV cells and minimisation of leakage paths ensures the storage system 7 operates as efficiently as possible. When the system is idling between activity bursts it is able to accumulate electrical energy. It is important that during these intervals the power consumption of the probe is as low as possible. For this reason during these times, power is electrically isolated from the main power draws (the SOC 10, camera 8, LED 9, radio 15) using a power transistor and only a dedicated timer device 14 is allowed to run. The timer 14 is configured to wait for a specified period of time measured in minutes or even hours, and then to trigger a wake up of the SOC 10 to initiate an activity burst.

[45] An activity burst should be executed as efficiently as possible to minimise the drain on the energy storage subsystem 7. Each burst comprises the following steps:

[46] 1. Timer 14 enables power to the SOC 10 and associated peripherals and triggers the SOC 10 to boot from internal non-volatile storage ¨ typically flash memory.

[47] 2. SOC 10 boots into its Real Time Operating System (RTOS) and loads its application executable.

[48] 3. The application reads a wake counter from non-volatile storage, increments it and writes it back to non-volatile storage. If the counter is still below a fixed threshold, the probe decides this is not the time to wake up and jumps directly to step 10 below. However if the counter has reached the threshold then an image capture and transmission will occur. This mechanism is designed to allow finer control over the effective wake up interval. For example, the wakeup timer may be configured by a resistor value to wake up the SOC every 10 minutes which is very likely too frequent since it would drain the energy storage subsystem 7 too quickly, but by using the counter mechanism longer effective intervals can be implemented with minute granularity with only a slight energy budget penalty.

[49] 4. A TCP/IP or UDP session is opened with the wireless network access point it was associated with when first deployed.

[50] 5. The camera 8 is initialised for still image capture.

[51] 6. A series of frames is captured with ramping LED illumination durations until a correctly exposed image is received. This approach is necessary because the ambient illumination conditions will vary dramatically from full sunlight in a greenhouse during the day to complete darkness during the night or certain phases of crop maturation which call for darkness.

[52] 7. The optimally exposed image is optionally compressed and then transmitted to the server infrastructure over the wireless network.

[53] 8. Metadata such as the probe's unique ID and other readings that may be available are appended to the transmitted data package along with the image itself.

[54] 9. The SOC 10 resets its wake counter in non-volatile memory to zero.

[55] 10. The SOC 10 initiates a shutdown which causes all power to be removed from the SOC 10 subsystem until the next wake up event.

[56] Each activity burst lasts well under 1 second and therefore consumes a minimal amount of energy. Due to the time and energy constraints the probes are limited to data capture and transmission ¨ all processing, archival and analysis occurs within much faster computer systems downstream of the probes.

[57] Referring to FIG 6:

[58] A probe 16 is within wireless communication range of several wireless access points 17(i), 17(ii) & 17(iii). In turn the access points 17(i-iii) are connected using wired network links to a standard network switch/router 18. Switch 18 also connects local server 19 and provides a link 20 to remote data centres across the Internet or dedicated circuits.

[59] When a probe 16 is first deployed and activated, it examines the access points it can detect and ranks them in order of signal strength, identifying an affinity to its preferred access point which is generally the one located closest to it. This information is stored in the probe's non-volatile memory so it can automatically connect to it during an activity burst.

[60] When the probe 16 is sending broadcast signals to all access points 17, each can measure the signal strength and so collectively the access points 17 can estimate the physical location of each particular probe as identified by a unique ID or even the MAC address embedded in the broadcast packets.

[61] Since signal strength scales inversely with distance squared between two communicating nodes, the access points 17 can model a volumetric region in which each probe 16 resides. Various factors such as signal attenuation due to obscuring objects, reflections and anisotropic antenna characteristics will distort this estimate so a heuristic algorithm complete with the ability to discard outlier readings that are inconsistent with near concensus among large populations of access points 17 is suggested.

[62] The more access points 17 that each probe 16 can reach with radio signals, the more accurate the location fix will be.

[63] Possible alternative methods for establishing probe 16 location fixes include radio flight time measurements and direct optical detection of strobed lights from the probe.

[64] As depicted in FIG 6, local analysis of the images transmitted by populations of probes is performed in the local server(s) 19. Local servers 19 collect probe data from the access points and decide if the images reported by each probe 16 show significant differences from the previous image reported by that probe, after normalising for global image changes due to varying illumination levels. If the probe 16 is deemed to have new data to report, the image and associated metadata are in turn transmitted to a remote data centre for detailed analysis. Such analysis may include conventional image processing techniques for normalisation and feature enhancement, as well as pattern recognition algorithms designed or trained to identify insects 5 caught in the traps 4 (FIG 2). This information may simply be archived or may also trigger alarms that would result in further investigations by the operators of the facility to see if intervention is required.

Claims

The embodiments of the invention in which an exclusive property or privilege is claimed are defined as follows:

1. An apparatus for a probe capable of the remote detection of horticultural pests and transmitting images of said pests to server infrastructure for analysis comprising :
¨ A component for receiving electrical power;
¨ An energy storage component;
¨ An energy management component coupling the harvester to an energy storage component and the main electronics payload;
¨ A microprocessor complex component;
¨ A wake up timer component;
¨ An image sensor component;
¨ An illumination source component;
¨ A pest trap component;
¨ A communications component; and ¨ Structural elements to integrate the above components into a single device.

2. The apparatus of claim 1 wherein electrical power is received by a single or plurality of photovoltaic cells.

3. The apparatus of claim 1 wherein electrical power is derived from beamed microwave energy converted by an antenna.

4. The apparatus of claim 1 wherein electrical power is received directly via a cable.

5. The apparatus of claim 1 wherein electrical power is received from a fixed or replacable battery.

6. The apparatus of claim 1 wherein the energy storage component is based on high capacity capacitors such as supercapacitors.

7. The apparatus of claim 1 wherein the energy storage component is based on rechargable chemical cells or batteries such as lithium polymer or lithium ion types.

8. The apparatus of claim 1 wherein the energy control component consists of a transistor circuit to prevent over-voltage.

9. The apparatus of claim 1 wherein the energy control component consists of a power management integrated circuit.

10. The apparatus of claim 1 wherein the microprocessor complex consists of a System On a Chip (SOC) with integrated volatile and non-volatile storage, interfaces for connection to cameras and networks.

11. The apparatus of claim 1 wherein the microprocessor complex consists of a Field Programmable Gate Array (FPGA) implementing SOC functions.

12. The apparatus of claim 1 wherein the microprocessor complex is implemented using several integrated circuits to separately provide processor, memory, camera interface and network interface functions.

13. The apparatus of claim 1 wherein the image sensor is an integrated camera device comprising Complimentary Metal Oxide Semiconductor (CMOS) or Charge Couple Device (CCD) image sensors and associated optics.

14. The apparatus of claim 1 wherein the image sensor is a Complimentary Metal Oxide Semiconductor (CMOS) or Charge Couple Device (CCD) image sensor paired with discrete optics.

15. The apparatus of claim 1 wherein the network interface implements the IEEE

802.[abng] WiFi standard protocol.

16. The apparatus of claim 1 wherein the network interface implements the Bluetooth standard wireless link protocol.

17. The apparatus of claim 1 wherein the network interface implements a proprietary wireless communications protocol.

18. The apparatus of claim 1 wherein the network interface implements a wired Ethernet network protocol.

19. The apparatus of claim 1 wherein the pest trap comprises a sticky paper sheet.

20. The apparatus of claim 19 wherein the pest trap is coloured in such a way as to attract pests.

21. The apparatus of claim 19 wherein the sheet includes chemical pest attractants.

22. The apparatus of claim 19 wherein the sheet is illuminated by the camera Light Emitting Diode (LED) under software control to attract pests.

23. The apparatus of claim 19 wherein the sticky side of the sheet is overlaid with a relatively coarse mesh of fine filaments to prevent insects above a certain size from reaching and being trapped on its surface.

24. The apparatus of claim I wherein the network interface is wireless and part or all of a wire frame supporting all the operational elements of the probe also serve as a wireless antenna.

25. A System for automatically detecting and identifying horticultural pests comprising remote probes, network access points and local servers connected to remote data centres.

26. The system of claim 25 wherein a plurality of remote probes are deployed throughout a horticultural facility.

27. The system of claim 25 wherein software running on the probes is able to estimate the probe's location using radio telemetry from signals generated by the network access points.

28. The system of claim 25 wherein software running on the access points or local servers is able to estimate each probe's location using radio telemetry from signals generated by the probes.

29. The system of claim 25 wherein the local servers are connected to remote data centres across the Internet.

30. The system of claim 25 wherein the local servers are connected to remote data centres across dedicated optical network links.

31. The system of claim 25 wherein software running on the local servers and/or the remote data centre servers performs image analysis and pattern recognition functions to automatically detect, identify and count suspected pests captured in images transmitted from the probes.