Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/50—Depth or shape recovery
  • G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/10—Terrestrial scenes
• • A—HUMAN NECESSITIES
  • A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
  • A01B—SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL
  • A01B39/00—Other machines specially adapted for working soil on which crops are growing
  • A01B39/12—Other machines specially adapted for working soil on which crops are growing for special purposes, e.g. for special culture
  • A01B39/18—Other machines specially adapted for working soil on which crops are growing for special purposes, e.g. for special culture for weeding
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30181—Earth observation
  • G06T2207/30188—Vegetation; Agriculture
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/20—Image signal generators
  • H04N13/204—Image signal generators using stereoscopic image cameras
  • H04N13/254—Image signal generators using stereoscopic image cameras in combination with electromagnetic radiation sources for illuminating objects
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N2013/0074—Stereoscopic image analysis
  • H04N2013/0081—Depth or disparity estimation from stereoscopic image signals

Definitions

• the present invention relates to automated weed detection and/or destruction.
• One aim of this invention is to improve the accuracy of the identification of weeds or of the differentiation between crops and weeds, particularly in the case of crops sown in bands in a substantially random distribution within the band with weeds present within the band of crops.
• Other aims will be clear from the following description.
• the present invention provides a method of differentiating between weeds and crops as defined in claim 1 , particularly with a view to determining the position of weeds growing in soil amongst crops.
• the present invention provides a method of selectively destroying weeds, preferably individually, using such a method to determine the position of weeds.
• the invention also extends to equipment adapted to identify and/or destroy weeds using such methods.
• an automated in-row weeding device requires identifying the precise position of weeds. Once the position of a weed has been determined, an automated system may be used to destroy the weed, for example, by delivering or spraying a chemical at the precise location of the weed (for example by controlling an array of micro-sprayers rather than blanket spraying of chemicals), by thermal destruction for example using a flame or heated probe, or by using a robotic arm (for example using a mechanical cutter having a width of perhaps 1-2 cm).
• the possibility of thermal or non-chemical destruction of individual weeds has particular application for organic agriculture.
• the position of weeds may be generated in the form of a weed map or spray map.
• the invention is based on the premise that improved accuracy in the automated identification of weeds and/or in the differentiation between weeds and crops and/or in the identification of their position can be achieved by estimating the height of plants protruding from the soil rather than the distance between the top of a plant and a fixed measurement device. This is particularly the case when the level of the soil is not even or planar and/or the soil has a non- planar profile or surface roughness. It is even more the case in such conditions when the plants are young and therefore of small size compared with the irregularities of the ground.
• the invention is particularly applicable where the difference in height between weeds and crops is of the same order of magnitude as the surface roughness of the soils or where the ratio a/b between (a) the variation in the soil height, for example within the area of a captured image, and (b) the average crop height is greater than 1/8, 1/5, % or 1/3. This may be the case during the early stages of growth of the crops.
• the height of the crops being analysed may be greater than 2 mm, 5 mm, 10 mm or 20 mm; it may be less than 200 mm, 150 mm, 100 mm, 80 mm or 50 mm.
• the invention is of particular use in relation to (i) crops sown in a line or band along a ridge or raised strip of soil, for example a ridge of a ploughed ridge and furrow and/or (ii) crops sown in rows or bands having a width of less than 100 cm, particularly less than 70 cm, less than 60 cm or less than 50 cm.
• the average density of crops within a band may be greater than 10 crop plants per m 2 ; it may be greater than 20, 50, 100 or 150 crop plants per m 2 .
• This average density may be assessed, for example for carrots, by considering ten separate areas of 30 cm long by 5 cm wide at which crops have been sown along the centre line of the band, counting the number of crop plants in each of these areas and calculating the mean average density of crops per m 2 .
• the average crop density particularly when seeds are sown and/or in the initial part of their growth cycle may be greater, for example greater than 500 or 1000 per m 2 .
• Early weeding is particularly beneficial for horticultural crops, for example carrots: some common annual weeds have their peak of germination around the same times as crop sowing and affect the crop growth. It has been shown that there is a significant effect of weed removal timing on the yield of e.g.
• carrots the 3-week and 5-week after sowing weeded plots have a significantly greater yield than 7-week treatment. Furthermore, carrots are sown in a relatively dense irregular pattern. Consequently, the invention is particularly applicable to differentiation between plants comprising mixed weeds and crops early in the crop growing cycle. This may be within the time period starting 7 days, 10 days or 14 days after sowing of the crops and/or ending 60 days, 90 days or 100 days after sowing of the crops.
• the invention is particularly applicable to identification of weeds and/or determination of weed position where the weeds are mixed with crops, particularly where the weeds are within a crop line or band. This is often the case with horticultural crops.
• the invention may be of particular application for use with one of more of the following crops: one or more apiaceae (also called umbellifers); carrots; celery; fennel; medicinal plants; cumin; parsley; coriander; parsnip; common beans.
• the apiaceae class is intended to include: Anethum graveolens - Dill, Anthriscus cerefolium - Chervil, Angelica spp.
• the invention may also be used in relation to: beans; potatoes; chicory; beetroot.
• Automated weeding equipment may be arranged to work its way along a line or band of crops; it may be self propelling. It may operate on a step by step basis so that it remains stationary at a first position along a line a crops whilst detecting the position of weeds and dealing with them before move to a second position further along the line of crops and repeating this procedure. Alternatively, the weeding equipment may function whilst it is moving.
• the weeding equipment may be arranged to travel along a line of crops several times during a critical weeding period. Thus, for example, if a weed is not correctly identified as such during a first passage, it may be correctly identified during a second passage, for example 2, 3, 4, 5 or more days later. Such a system may be used to reduce the risk of inadvertently destroying crops through incorrect identification; if the first time the weeding equipment passes a weed having a corrected plant height similar to the expected crop height it may not be identified as a weed. Nevertheless, at the subsequent passage of the weeding equipment, given the difference in growth rates of the weeds and the crops, the difference between the corrected plant height and the expected crop height will be greater thus facilitating correct identification of that particular weed.
• the weeding equipment may be partially or entirely solar powered; it may comprise one or more solar panels. It may use solar energy to supplement and/or replenish an electrical energy storage device, for example a battery or accumulator. Such arrangements may facilitate the autonomy of the weeding device.
• the weeding equipment may comprise a guidance system to direct its movement, for example, along a line of crops.
• the stereoscopic data of plants growing on soil may be acquired in the form of an image, for example using a camera as a sensor.
• the captured plant data points ie data points representing a position at which the presence of a plant has been detected
• the captured soil data points ie data points representing a position at which the presence of soil has been detected
• the data analysis and/or weed detection is preferably implemented using a computer, for example using computer software.
• structured light is projected, this may be coded structured light.
• non- coded structured light may be used and may allow faster data processing.
• a line scanner system may be used to acquire stereoscopic information, for example using common structured light. When using a line scanner, segmentation of each image may be used to find the scanner line but this may result in reliability problems due to occlusions and discontinuities and large differences of reflectance between soil and plants. Alternatively, temporal analysis on images sequences may be used to find the lines and may actually give better results than using coded structured light. Passive stereoscopy may also be used to acquire stereoscopic information.
• the weeds to be detected or differentiated from crops may be selected from the group consisting of: Sonchus asper L., Chenopodium sp., Cirsium sp., Merurialis M. perennis, Brassica sp. and Matricaria maritima.
• Fig 1 is a perspective view of an experimental apparatus for determining the position of weeds
• Fig 2 and Fig 3 which are schematic side views illustrating plant height
• Fig 4 which is a schematic representation of steps in the method of determining the position of weeds
• Fig 5 which is a schematic representation of a time multiplexed coded structured light system
• Fig 6a and 6b which are schematic cross sections of plants growing in soil; Fig 7 which is a representation of determination of expected plant height.
• the experimental apparatus of Fig 1 comprises a video projector 1 1 arranged to illuminate a portion of a band of soil 12 which contains both crops and weeds and a camera 13 arranged to capture an image, in this case a top-down image approximately 200 mm by 250 mm.
• the video projector 1 1 and camera 13 are mounted on a carriage 10 adapted to move along the band of soil 12. Additional lighting 14 and/or a reflector 15 may also be provided on the carriage 10.
• Shrouds (not shown) to occlude natural light and shield the scene to be analysed from external light were fitted to the carriage 10.
• the system uses a projected image and a camera to capture the image as reflected from the scene to be analysed; analysis can thus be based on the deformation between the projected image and the captured image.
• One aspect of the invention is based upon improving the accuracy of the estimation of plant height h when differentiating between crops and weeds. Whilst the distance between the top of a plant and an overlying camera 13 can be determined, for example by image analysis, this may not provide a particularly accurate indication of plant height h. As illustrated in Fig 2, one difficulty arises from the soil profile 22 which gives a variation in soil height such that the distances Z1 and Z2 between the top of a plant and the camera 13 is a poor approximation of the plant height h. Another problem, illustrated in Fig 3, arises from any inclination of the camera 13 with respect to the plane of the soil 23 which exacerbates this. Each of these factors may be significant when assessing plant height in the real conditions of an agricultural field.
• the embodiment used a coded structured light stereoscopic imaging system with a multispectral camera to allow registration of height information over multispectral images.
• the multispectral camera was based on a black and white camera (C-cam BCI 5 1.3 megapixels); the projector was a DLP video projector (OPTOMA EP719 with a 1024x768 resolution). Acquisition speed and mechanical vibrations concerns due to the filter wheel dictated the use of monochromatic patterns acquired without filter in front of the camera.
• the coded structured light technique had to take into account the specificities of the small scale agricultural scenes, namely occlusion and thin objects, internal reflections and scene high dynamic range. It was also necessary to obtain robust results and take into account the specificities of the video projector.
• Table 1 summarizes the choice of the codification and strategies used to overcome those problems. As fast acquisition was not a concern in this embodiment and a black and white camera was used (for the multispectral part of the acquisition device), we decided to use a time multiplexing approach with a binary codeword basis (black or white illumination).
• the Hamming distance between two codes is the number of corresponding bits that are different.
• the length of the code used was 22 bits, which allowed for the minimum Hamming distance requirement and gave good decoding results.
• the codes were decoded by correlation: the signal received by a single camera pixel over time was compared with all possible signals. As correlation also gives a measure of the reliability of the decoding, it was used to remove spurious measurements by applying a threshold.
• the projected images are comprised of black and white bands of large then finer width.
• the wider bands cause problems when the scene is prone to internal reflections (the illuminated part of one part of the scene will illuminate other parts of the scene).
• the code used here also happened to be pseudorandom (i.e. no apparent structure) which resulted in a more uniform illumination.
• the high dynamic range acquisition allowed us to have a strong signal to noise ratio for all pixels of the image.
• An equipment related problem encountered was the shallow depth of field of the projector: given the size of the scene and distance from projector it was not possible to have the projected pattern sharp on close and distant object of the scene.
• the choice of a per-pixel decoding scheme combined with the weakly correlated code was also motivated by that characteristic.
• the calibration of the camera-projector system was done using the Zhang technique from the Intel OpenCV library.
• the method used plant height as a discriminating parameter between crop and weed.
• the raw data acquired by the stereoscopic device was not plant height but the distance of the plants relative to the measurement device. This distance doesn't accurately represent plant height if the position of the device relative to the ground varies or if the ground is irregular.
• This parameter is the distance between plant pixels and the actual ground level under them obtained by fitting a surface and seen from a reconstructed point of view corresponding to a camera's optical axis perpendicular to the ridge plane.
• the crop/weed discrimination process is illustrated Figure 4.
• the camera 13 was used to acquire a stereoscopic image 41 of the plants growing on the soil.
• any spurious pixels were frequently present at the limit between plants and ground, which was the border of the regions that were of interest for the plant height determination. To avoid this problem, the borders of those regions were eroded by a round structuring element of diameter 3 pixels.
• the plant pixels 43 and soil pixels 45 (the latter with the interpolated pixels obtained from modelling since the ground under the plants was not visible from the camera, and not all points seen by the camera were illuminated by the projector) were then put back together to create a corrected image 46 for which the orientation of the fitted plane 44 was used to rotate the data in space so as to align the plane normal with a virtual camera ( ⁇ ) optical axis perpendicular to the calculated plane of the soil.
• the first parameter is for each plant pixel the distance between the plant pixel and the reconstructed soil underneath (corrected plant height).
• the second is the expected crop height which, in this case, was estimated from the number of days after sowing and empirical data previously determined giving the average height of the crops in question as a function of the number of days after sowing in similar growing conditions. il onas s i
• the overall classification accuracy without correction was 66%.
• the overall classification accuracy was 83%.
• the expected crop height may be determined automatically, for example by determining an average height of the plants or the crops from the captured images.
• the corrected plant height is used for such a determination.
• the carrots typically are sown in a band 5 cm wide with 10 to 15 carrots per 10 cm length of the band.
• the position of the band may be estimated from the captured images and the image divided into a zone in which there are only weeds, and a zone in which there are weeds and crops. Comparing data of plant height from these two zones may be used in estimating the height of the crops.
• Preferred stereoscopic imaging and analysis Stereoscopic imaging aims to record three-dimensional information.
• acquisition methods passive and active and either may be used in the context of the invention.
• Passive methods usually rely on several views of the same scene to recover depth information (e.g. binocular stereoscopy, similar to human depth perception).
• Active methods are characterized by the projection on the scene of some form of energy (commonly light) to help acquire depth information.
• Binocular stereoscopy is fairly common since it is simple to implement in hardware and is well suited to real-time acquisition. Robust acquisition of dense stereoscopic data by this technique is not an easy task.
• the imaging and analysis preferably uses structure coding light with: • A time multiplexing code;
• the projected coded light may have at least 18, preferably at least 20 and more preferably at least 22 patterns. There may be at least 6, preferably at least 7 and more preferably at least 8 differences between each projected code.
• At least two defined spectral bands may be used in acquiring the image, for example a spectral band centred on 450 nm and having a band width of 80 nm and a spectral band centred on 700 nm and having a band width of 50 nm.
• the spectral bands may be implemented by the use of filters, for example placed in front of the camera. Alternatively, the camera may be adapted to have a frequency response in the desired spectral bands.
• a third spectral band may also be used, for example centred on 550 nm and having a band width of 80 nm. The use of selected spectral bands may increase the accuracy of differentiating between plants and soil.
• Each light projection consisted of 768 fringes (corresponding to the resolution of the projector), each of these projections being called “a pattern”; Each luminous fringe is successively lit or not lit 22 times which corresponds to a 22 bit code; the codes are pseudo-random (without apparent structure) to avoid disturbances due to internal reflection of the scene and are weakly correlated between themselves;
• a pattern is thus characterised by 768 luminous fringes, each being lit or not lit as a function of one of the 22 values of the code; For each pattern, four exposures of different duration (0.6, 0.3, 0.07 and 0.01 seconds) are used so that each part of the scene is correctly exposed be it light or dark.
• the first image with the longest exposure time is used as the base.
• the overexposed part of the image is eliminated and replaced by the corresponding part of the image with a lower exposure time. This is repeated three times to until a correctly exposed image is obtained;
• the image is analysed pixel by pixel by correlation between the signal emitted and the signal received.
• Fig 6a illustrates crops 61 , 62 growing in soil 63 having an irregular soil profile 22.
• the variation in the soil height a ie the difference between the highest and lowest point on the soil profile in an area being considered
• the variation in soil height a is of the same order of magnitude as the average crop height b.
• the variation in soil height a may be 40 mm and the crop height b may be 50mm so that the ratio a/b is 0.8.
• Fig 6b is similar save that the area of soil analysed is larger and covers substantially the entire width of a mound of soil 64 in which crops 61 ,62 , for example carrots, are grown.
• the variation in the soil height a may be 15cm and the average crop height b may be 3 cm (early during the growing cycle) so that the ratio a/b is 0.2. Later in the growing cycle, the average crop height b may be 10 cm with the variation in soil height a 15 cm so that the ratio a/b is 0.67.
• the method may comprise determining the position and/or boundaries of the sown band. This may be used to reduce the area of a field where high precision weeding would be necessary as in the area outside the sown band, all plants may be destroyed without further characterisation on the assumption that they are weeds. Alternatively or additionally, this may be used in automatic determination of the expected crop height.
• Derivation of the expected crop height from captured data of the plants growing on the soil may be determined as follows: a) determining the boundaries of the sown band; b) determining the corrected plant height of plants within the boundaries of the sown band; c) determining the most frequently occurring corrected plant heights and assuming this to indicate the height of the crops (ie the average expected crop height on the basis that the crops are significantly more prevalent than weed in the sown band) and/or using this to determine a range of expected crop heights.
• line 71 represents the corrected heights of plants detected within the sown band against their probability density (which is representative of their frequency of occurrence) 21 days after sowing for a row of carrots.
• the peak of line 71 occurs at a corrected plant height of about 17 cm. This is shown as the average of the expected plant height 72.
• Line 74 represents the probability density function of carrots and line 73 the probability density function of weeds.
• the range of expected crop height for the carrots, as illustrated by line 72 is about 7 to 17 cm.
• the expected crop height is preferably used as a range so that any plant determined to have a corrected plant height outside this range is classified as a weed.
• a range may be determined from empirical data or may be derived from captured data.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Multimedia (AREA)
• Optics & Photonics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Soil Working Implements (AREA)

Priority Applications (1)

Applications Claiming Priority (3)

Publications (1)

Family

ID=40030373

Family Applications (1)

Country Status (2)

Families Citing this family (17)

Family Cites Families (1)

• 2009
  • 2009-06-18 EP EP09765883A patent/EP2327039A1/de not_active Withdrawn
  • 2009-06-18 WO PCT/EP2009/057588 patent/WO2009153304A1/en active Application Filing

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events