WO2023105112A1 - Weeding robot - Google Patents
Weeding robot Download PDFInfo
- Publication number
- WO2023105112A1 WO2023105112A1 PCT/FI2021/050866 FI2021050866W WO2023105112A1 WO 2023105112 A1 WO2023105112 A1 WO 2023105112A1 FI 2021050866 W FI2021050866 W FI 2021050866W WO 2023105112 A1 WO2023105112 A1 WO 2023105112A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- robot
- weeding
- plant
- data
- weeding robot
- Prior art date
Links
- 238000009333 weeding Methods 0.000 title claims abstract description 99
- 241000196324 Embryophyta Species 0.000 claims abstract description 266
- 238000011835 investigation Methods 0.000 claims abstract description 50
- 238000012545 processing Methods 0.000 claims abstract description 50
- 238000000034 method Methods 0.000 claims abstract description 33
- 238000004458 analytical method Methods 0.000 claims abstract description 23
- 230000005540 biological transmission Effects 0.000 claims abstract description 12
- 238000003384 imaging method Methods 0.000 description 9
- 230000015654 memory Effects 0.000 description 9
- 239000000463 material Substances 0.000 description 7
- 238000001228 spectrum Methods 0.000 description 6
- 241000239290 Araneae Species 0.000 description 5
- 239000003415 peat Substances 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 3
- 230000006378 damage Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 241000238017 Astacoidea Species 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 239000002689 soil Substances 0.000 description 2
- 239000004575 stone Substances 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 235000016626 Agrimonia eupatoria Nutrition 0.000 description 1
- 235000018185 Betula X alpestris Nutrition 0.000 description 1
- 235000018212 Betula X uliginosa Nutrition 0.000 description 1
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 241000196323 Marchantiophyta Species 0.000 description 1
- 241000218657 Picea Species 0.000 description 1
- 235000008331 Pinus X rigitaeda Nutrition 0.000 description 1
- 241000018646 Pinus brutia Species 0.000 description 1
- 235000011613 Pinus brutia Nutrition 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000005452 bending Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000005670 electromagnetic radiation Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 235000015097 nutrients Nutrition 0.000 description 1
- 238000012856 packing Methods 0.000 description 1
- 241000894007 species Species 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01M—CATCHING, TRAPPING OR SCARING OF ANIMALS; APPARATUS FOR THE DESTRUCTION OF NOXIOUS ANIMALS OR NOXIOUS PLANTS
- A01M21/00—Apparatus for the destruction of unwanted vegetation, e.g. weeds
- A01M21/02—Apparatus for mechanical destruction
Definitions
- the present invention relates to a weeding robot to be used for killing weeds, i.e. unwanted plants, in nurseries, as well as a method for killing weeds, and a computer program for performing the method.
- a nursery is an area for growing tree seedlings to be planted elsewhere. Nurseries produce healthy tree seedlings and forest planting materials of good quality for various areas, for example for forest cultivation, forests, and cutting areas. In a good seedling, both the stem and the root are of high quality. Therefore, careful monitoring and care during cultivation are important. Killing weeds, i.e. weeding, is an important part of the care. Manual weeding, however, is both laborious and time-consuming and thus requires a lot of labour and time, which increases costs in nurseries and thereby also the prices of tree seedlings and possibly other forest planting materials grown in the nursery.
- the invention relates to a weeding robot which is configured to move among tree seedlings growing in the cultivation area of a nursery, the weeding robot comprising a sensor for collecting data on a plant under investigation in the cultivation area, and data transmission means for transferring the data collected on the plant concerned to a data processing device, to be analyzed by a plant identification algorithm, as well as least one weed destroying means. Furthermore, as a result of analyzing image data, the data transmission means are configured to receive at least one control signal for controlling the operation of the robot.
- the weeding robot comprises moving means for moving among tree seedlings growing in the cultivation area of the nursery.
- the moving means comprise at least two pairs of legs fastened to the side surfaces or the bottom surface of the robot.
- the moving means comprise wheels provided on the bottom surface of the robot, for moving along moving surfaces provided in the cultivation area.
- the control signal is arranged to configure the robot to remove the plant under investigation by its weed destroying means, if the plant is identified as a weed by the plant identification algorithm.
- the control signal is arranged to configure a sensor to repeat collecting of data on the plant under investigation, if the data is determined to be of poor quality.
- the weed destroying means is a push rod for pushing the weed into the ground.
- the data processing device is a part of the weeding robot.
- the weeding robot further comprises a battery and means for receiving electric energy in a wireless manner.
- the weeding robot comprises a distance sensor for detecting a plant in the cultivation area.
- the sensor is a camera and the collected data is image data.
- the invention relates to a method for weeding.
- the method comprises capturing data on a plant under investigation in the tree seedling cultivation area of a nursery by a sensor of a weeding robot according to the first aspect, or an example thereof; transferring the data collected on the plant under investigation to a computer device, to be analyzed by a plant identification algorithm; and receiving, as the result of the analysis of the data, at least one control signal for controlling the operation of the robot.
- the method also comprises destroying the plant under investigation by the weed destroying means of the weeding robot, in response to the received control signal.
- the method further comprises repeating the collecting of data on the plant under investigation, and transferring the re-collected data to the data processing device, to be analyzed with a plant identification algorithm.
- the method further comprises moving the weeding robot in the area for cultivation of tree seedlings of the nursery by means of spider-like legs of the weeding robot.
- the method further comprises moving the weeding robot along moving surfaces provided in the area for cultivation of tree seedlings of the nursery by means of wheels on the bottom surface of the weeding robot.
- the invention relates to a computer software product which is stored in a computer-readable medium and is executable in a data processing device.
- the computer software product comprises instructions which make a weeding robot according to the first aspect, or an example thereof, perform the weeding method according to the second aspect, or an example thereof.
- Fig. 1a-c shows a weeding robot according to an embodiment of the invention
- Fig. 2a-b shows a weeding robot according to an embodiment of the invention in a cultivation area
- Fig. 3a-d shows a weeding robot according to an embodiment of the invention in operation
- Fig. 4 shows a flow chart of a weeding method for a weeding robot according to an embodiment of the invention.
- Nurseries are areas providing young forest tree seedlings for forest cultivation, to be planted in various areas, such as forests and cutting areas.
- Forest tree seedlings may be, for example, spruce, pine or birch seedlings
- the nursery may be a greenhouse or an area intended for
- nursery may comprise several cultivation areas.
- the cultivation areas in nurseries may be implemented, for example, by means of various nursery beds, areas or fields comprising, for example, various culture levels, areas or boxes, in which a plurality of tree seedlings are placed in their respective nursery trays, such as cells, balls or bags.
- Nursery trays, or ball trays, or ball compartment trays are matrix type structures having places for cultivation or growing for several seedlings in row and column directions. They have a high planting density but the single cells are small in volume.
- the nursery trays may be made of, for example, rigid plastic, peat, paper, cellular plastic, or any material suitable for the purpose.
- One cultivation area may often comprise several or even tens or hundreds of nursery trays, for example in matrix format; in other words, one cultivation area normally comprises several rows and columns of nursery trays.
- nurseries provide healthy tree seedlings and/or other forest planting material of good quality. Killing of weeds, i.e. unwanted plants, in the areas for cultivation of seedlings in nurseries is an important part of the care of the seedlings, because early weeding in the areas for cultivation of seedlings, already at the emergence of cotyledons, will prevent development of the weed. Weeding improves the growth environment of the tree seedlings, because weeds which deprive the seedlings of space and nutrients are removed from the rhizosphere of the seedlings. Weeding also prevents weeds from ending up in the final planting sites of the seedlings.
- weeding for example by pulling up manually, takes up a lot of workers’ time and effort, which increases nursery costs and prices of tree seedlings and other forest planting materials.
- the packing of cultivation trays tightly together in the cultivation area causes challenges of reaching the weeds; that is, weeds growing in the more central areas of the cultivation area cannot be destroyed/pulled out manually without moving the cultivation trays, which further increases the work needed for weeding.
- chemical weed control is not possible or not desirable, whereby weeding has to be performed mechanically, for example manually.
- a solution which reduces workers’ time and effort needed for weeding.
- One solution to this weed problem is a weeding robot according to the invention, i.e. a robot for destroying unwanted plants.
- the robot itself can be implemented in a variety of ways, as well as its movement among seedlings in cultivation areas, but in the solutions, a plant detection algorithm is used for analyzing data collected by a sensor or sensors of the weeding robot in cultivation areas and relating to a plant under investigation, and if the plant under investigation is identified as a weed, it is killed by the robot.
- the plant detection algorithm can identify plants by comparing data collected by sensors with sensor data in a database. For each respective sensor, a sufficient amount of reference data is stored in the database, to be used for identifying different plants or their features.
- the plant to be identified may thus be a weed or a seedling.
- the algorithm may be, for example, a machine learning based plant identification algorithm implemented by artificial intelligence (Al) and trained with a sufficient amount of data to identify different features of plants or different plants; or the algorithm may be another plant identification algorithm suitable for identifying plants, comparing collected sensor data with data in a database, for identifying a plant either as a weed or a seedling. Weeds to be destroyed include, for example, unwanted plants at the emergence of cotyledons, and sprouts of liverwort, which may differ from the tree seedlings in, for example, their appearance, moisture, size, or reflection spectrum.
- the sensor may be, for example, an image sensor, which in this context is referred to by the term camera.
- the camera may be, for example, an RGB (red green blue) camera, an sRGB camera, or a CCD camera (charge-coupled device), such as a digital camera or another ordinary photographic camera.
- the camera may be a hyper spectrum camera used for hyper spectrum imaging, comparing differences in the reflection spectra of electromagnetic radiation, i.e. hyper spectrum data, of weeds and seedlings, to distinguish weeds from seedlings by an algorithm. Spectral differences are formed by slight differences in the chemical compositions of different plant species.
- the sensor may also be an infra-red sensor or an infra-red camera measuring infrared radiation for identifying plant species and/or for distinguishing weeds from seedlings on the basis of differences in the infra-red radiation generated by the plant species.
- the sensor may be, for example, a laser scanner for distance measurement, forming a point cloud whose shape and/or height can be used to identify a plant as a weed or a seedling. Data collected by the laser scanner is thus a relatively dense point cloud which is a 3D representation of the object scanned, in this case a seedling or a weed.
- the weeding robot applies one or more of the above-presented sensors and plant identification methods as well as a database comprising data for each sensor to be used for identification of plants.
- the weeding robot is configured to capture images with its camera when it moves in an area or areas of cultivation of tree seedlings in a nursery. With its camera, the robot captures an image comprising a plant to be investigated.
- the robot transmits the image data captured by its camera, i.e. the data collected on the plant under investigation, to a data processing device for analysis by a plant identification algorithm.
- the robot comprises the data transmission means needed for the transmission of image data, for example a transmitter or a transceiver, for transferring the image data from the camera to the data processing device in a wireless manner or via a wired connection.
- the data processing device comprises at least one processor, at least one memory comprising a computer program code for one or more software units, and means, for example a receiver or a transceiver, for receiving image data from a camera in a wireless manner or via a wired connection.
- processors may be provided, for example a general-purpose processor, a graphic processor and/or a digital signal processor (DSP), and/or several different memories may be provided, for example a volatile memory for storing data and software, and a non-volatile memory, such as a hard disk for permanent storage of data and software.
- the data processing device may be any data processing device suitable for data analysis and plant identification, such as a data processing unit or a computer.
- the data processing device may be electronically connected to a sensor via signal lines, via which it may also transmit control signals, i.e. configuration messages, to the robot and the sensor.
- the data processing device may be a part of the robot, i.e. an internal data processor in the robot, or it may be an external data processing device, such as a server or a cloud server.
- the image data may be transmitted from the camera of the robot to the data processing device over a data transmission network, for example a wireless local area network (WLAN), or by GSM, CDMA or WCDMA technologies or future technologies, or other data network technologies.
- WLAN wireless local area network
- the data processing device is configured to apply a plant identification system which corresponds to a face recognition system and may be a plant identification algorithm trained by machine learning or another plant identification method which performs sensor data comparison and, for example when applying an image sensor, compares an image captured of a plant under investigation with images in a database and detects if the plant under investigation matches with a plant in the image database.
- a plant identification system which corresponds to a face recognition system and may be a plant identification algorithm trained by machine learning or another plant identification method which performs sensor data comparison and, for example when applying an image sensor, compares an image captured of a plant under investigation with images in a database and detects if the plant under investigation matches with a plant in the image database.
- An appropriate algorithm and a sufficiently large amount of images of plants in the image database increase the probability of successful plant identification.
- the database is loaded with data on point clouds formed by different plants, such as different weeds and/or seedlings.
- the plant identification is effected by training the algorithm to identify different features of different plants by supplying the algorithm with a sufficient amount of data. For this, a sufficient amount of image data or other data has to be available, to train the system to distinguish different plants from each other.
- the algorithms used for image recognition are based on comparing features computed from images. They compare sets of pixels with each other, and when the comparison provides a sufficient level of recognition, that is, a plant is recognized with a sufficient degree of probability to belong to a specific plant species, the plant has been identified. On the basis of the identification, the plant identification algorithm classifies the plants as, for example, weeds or tree seedlings.
- a plant it is also possible to classify a plant as not dentified, if the plant is not recognized from an image, in which case it is not destroyed, or not dentified if the image shows something else than a plant, such as sawdust, stone, or peat.
- the plant identification also makes it possible to distinguish plants, such as seedlings and weeds, from a growing medium.
- the material of the growing medium, such as peat, soil, or sawdust and stones contained in it, is classified as not identified, whereby it is not destroyed.
- the data processing device instructs the robot to destroy the plant. If, for example, 20% of weed sprouts having cotyledons can be destroyed by the weeding robot, this is of great significance in reducing manual weeding.
- the database may, according to a first example, comprise images of plants other than tree seedlings grown in the area where the robot is moving to capture images, that is, for example different weeds.
- the data processing device compares an image of a plant under investigation, captured by the camera of the robot, with images of plants in a database. If a match is found, that is, the plant under investigation matches with a weed in the database, at least with a required/desired degree of probability, the plant is identified and classified as a weed.
- the required/desired degree of probability may be defined for the data processing device in advance.
- the data processing device configures/in- structs the robot, for example by means of a control signal, i.e.
- Destroying weeds may involve, for example, pulling out the plant from the ground, pushing the plant into the ground, cutting the plant, or other mechanical destruction or removal of the plant.
- the robot comprises the means required for destroying the weeds.
- the robot may comprise, for example, a hand gripper or suction means for pulling out a weed, a push rod or the like for pushing the weed into growth peat or material, or scissors for cutting the weed.
- the image database may comprise images of one or more tree species grown in the area where the robot is moving to capture images.
- the data processing device compares an image captured of a plant under investigation with images of a seedling or seedlings in the database. If a match with the seedling is not found, that is, the data processing device does not identify the plant under investigation as a seedling, the plant under investigation is identified, i.e. classified, as a weed, and the data processing device configures/instructs the robot to destroy the plant concerned and shown in the image captured by the camera of the robot.
- the image database may comprise images of different plants, weeds as well as forest tree seedlings grown in the respective area.
- the data processing device analyzes image data by comparing an image of a plant under investigation, captured by the camera of the robot, with images of plants in the database. If a match is found, i.e.
- the data processing device classifies the plant either as a weed or as a forest tree seedling, on the basis of the result of the plant identification. In other words, if the plant under investigation is identified as a weed, it is classified as a weed, and if the plant under investigation is found to match with a seedling, it is classified as a seedling.
- the data processing device config- ures/instructs the robot, for example by means of a control signal, to destroy the weed concerned and shown in the image captured by the camera of the robot. The destroying of weeds corresponds to the way of destoying of weeds in the above presented examples, by the means presented above.
- the robot is configured to collect data, for example to use its camera to capture an image of the next plant and to transmit the image data of the next image for analysis, and the plant investigated and identified as a forest tree seedling is allowed to remain intact.
- the robot may continue to capture images, i.e. to collect data, by default if no instruction is received to destroy an identified plant.
- plant identification is not always successful, i.e. the plant is not always identified or the object to be identified is not even a plant.
- the plant/object to be identified is not classified as a weed or a forest tree seedling, but it may remain intact and the robot is configured to collect data on the next plant, or it is already configured to collect data on the next plant if no signal to destroy is received, for example within a given time from the transmission of the image data.
- the data processing device finds the data on the plant under investigation to be of too poor quality to run the plant identification algorithm.
- the data processing device may transmit a control signal to the camera of the robot to configure the camera to re-collect data, i.e. for example to capture at least one additional image of the same plant under investigation and to transmit the collected sensor data for analysis.
- the robot is configured to collect data, for example to apply its camera to capture an image of the next plant and to transmit data on the next plant, for example image data, for analysis, and the plant with the data of poor quality may remain intact.
- the weeding robot comprises means for moving among forest tree seedlings.
- the weeding robot may be configured to move on its legs on top of ball compartment units of nursery trays with regular shapes for seedlings, whereby no separate rails or other structures are needed for movement.
- the robot may utilize the structures of the nursery trays and the peat surface, i.e. soil surface, or other existing structures of the nursery, and move independently among seedlings.
- movement surfaces for the robot i.e. rails or plane surfaces, between, by the side of, or above the tree seedlings.
- the weeding robot may thus be provided with legs on e.g. its bottom surface and/or side surfaces i.e.
- flanks for example six legs, or three pairs of legs, like a spider.
- the legs are configured to move in a way similar to, for example, the legs of a spider or a crab, making it possible for the robot to move or walk among seedlings in the cultivation area, and keeping the robot in balance among seedlings even on an uneven stand/terrain.
- the robot may, in fact, resemble for example a spider or a crayfish, such as a crab, when moving among seedlings.
- the term “spider legs” comprise all similar leg types, such as crayfish legs, which comprise a sufficient number of pairs of legs for moving in the cultivation area.
- a sufficient number of pairs of legs is at least two pairs, but the number of pairs of legs may be greater, for example three or four pairs.
- the robot may have been, or may be, configured to always move and/or spin around a specific distance or for a specific time, determined in advance or by a control signal, after which it uses its sensor to collect data, for example to capture an image by a camera; or alternatively, the robot may be provided with one or more distance sensors for transmitting information to a data processing device, so that the robot can identify that an object to be imaged, such as a plant to be imaged, is in the vicinity, i.e. within the imaging area of the camera.
- the distance sensor may be, for example, a touch sensor, a proximity sensor, a distance sensor, or a computer vision system.
- the robot When moving among seedlings, either on its legs or wheels, the robot uses its camera to capture images of plants from the side, i.e. from the side of the plants.
- the robot may image the plants from above when moving above the seedlings.
- the robot After the sensor of the robot has detected an object, such as a plant, e.g. in the imaging area of the camera, or after a predetermined distance or time to the next data collecting object has been passed, the robot takes the next image, which may or may not contain the next plant under investigation, and transfers the image data of the image to the data processing device for analysis by the plant identification algorithm.
- the data processing device transmits a control signal to the robot, to be received by the data transmission means of the robot.
- the control signal controls the operation of the robot on the basis of the results of the plant identification analysis. If the plant under investigation is identified as a weed, the robot comprising at least one weed destroying means is configured to destroy the weed shown in the image.
- the robot is configured to collect data, that is, in the case of a camera, to capture an image of the next plant. If the next plant is not within the imaging area of the robot or the sensor does not detect a plant in the imaging area, the robot may move or spin a predetermined distance or time between capturing images or for capturing an image. If, on the other hand, the image of the plant under investigation is found to be of inadequate quality for running the plant identification algorithm, the robot may be configured by a control signal to capture a new image of the same plant, or to move/spin slightly and take an image of the same plant, or to continue moving and imaging among the seedlings.
- the weeding robot also comprises a battery and means for receiving electric energy in a wireless or wired manner.
- a charging point may be arranged in the cultivation area where the robot is moving.
- the robot may be configured to be charged at specific intervals, whereby it will independently move to the charging point at the end of the interval, or the robot may be configured to move to the charging point when the charging level of the robot has decreased below a threshold level. After being charged, the robot may automatically return to imaging mode.
- a zone of movement may be determined for the robot in advance, for example by dimension data on the area or by a copper wire to indicate the border of the cultivation area, i.e. the area in which the robot is intended to destroy weeds, to the robot.
- the robot utilizes GPS technology for positioning and comprises a GPS device by which it determines its location and is capable of moving and weeding only in the cultivation area defined for it.
- any other positioning method or means suitable for the purpose may be used in/for the robot for indicating the cultivation area.
- FIG. 1a shows a weeding robot 10a according to an embodiment of the invention.
- the weeding robot 10a has three pairs of legs 11 , so-called spider legs, for moving among tree seedlings growing in the cultivation area of a nursery; an image sensor, i.e. a camera 12, for imaging plants in the cultivation area; and data transmission means 13 for transferring image data to a data processing device 14, for analysis by a plant identification algorithm; as well as means for receiving, as a result of the analysis of the image data, at least one control signal for controlling the operation of the robot.
- the robot 10a comprises a weed destroying means 15.
- the destroying means 15 is a foldable plunger, i.e.
- an articulated rod which is configured to protrude towards the ground and to destroy a weed by pushing it into the ground, if the plant identification algorithm has identified the weed.
- the plunger has a relatively small diameter, for example a few millimeters, so that it does not make too large a hole or entrain sawdust, placed on top of the ground, simultaneously when pushing the weed into the ground. It is also possible that the robot 10a transmits the image data to an external data processing device for analysis, and a data processing device 14 integrated in the robot 10a is used for controlling the operation of the robot 10a.
- the data processing device 14 comprises at least one processor; at least one memory comprising a computer software code for one or more software units for running a plant identification algorithm; and means for receiving image data from the camera 12 of the robot 10a in a wireless manner or via a wired connection, for example a receiver or a transceiver; and means for transmitting control signal in a wireless manner or via a wired connection, for example a receiver or a transceiver.
- processors may be provided, for example a general-purpose processor and a graphic processor and a digital signal processor (DSP); and/or several different memories may be provided, for example a volatile memory and a non-volatile memory, such as a hard disk for permanent storage of data and software.
- Figure 1 b shows a weeding robot 10b according to an embodiment of the invention.
- the weeding robot 10b corresponds to the robot 10a of Fig. 1 a in other respects, but in this embodiment, its weed destroying means 15 is not an articulated rod but a non-articulated rod of plunger type. It is also possible that the non-articulated rod is not a plunger-type rod but a telescopic arm which is short in situations other than pushing a weed into the ground, i.e. destroying it, in order not to block the way when the robot is moving among seedlings and weeds, and is extended when pushing a weed into the ground.
- the weeding robot 10b also comprises a so-called distance sensor 16 by which the weeding robot 10b detects the plants in its vicinity, seedlings as well as weeds.
- the distance sensor 16 may be a touch sensor, a proximity sensor, a distance sensor, or a computer vision system.
- the sensor 16 may transmit the information detected by it to a data processing device 14 which analyzes, on the basis of the information from the sensor 16, whether there is a need for collecting data, for example, capturing images.
- Figure 1c shows a weeding robot 10c according to an embodiment of the invention.
- the weeding robot 10c corresponds to the robot 10b of Fig. 1 b in other respects, but its weed destroying means 15 is a hand gripper; moreover, it has a different appearance.
- the sensor 16 in the figure is a hyper spectrum camera.
- the weed destroying means may also be placed in another part of the robot than in its front, for example at the bottom of the robot, whereby the plunger can be used and a weed can be pushed into the ground by bending the legs of the robot, and the plunger does not have to be pushed separately.
- Figure 2a shows a weeding robot 20 according to an embodiment of the invention, in an area 22 for cultivation of forest tree seedlings 21 .
- the forest tree seedlings 21 are planted in nursery trays 24.
- the weeding robot 20 moves on its legs among plants, i.e. seedlings 21 and weeds 23, in the cultivation area 22, and captures images of the plants with its camera 27.
- the weeding robot 20 transfers the image data of the image captured by the camera 27 on a plant under investigation, to an external data processing device 25 for analysis by a plant identification algorithm.
- the data processing device 25 is a server, but it may also be any computer device suitable for the purpose.
- the data processing device 25 compares the image data of the plant under investigation with plant images in the database by means of an algorithm. If the data processing device 25 identifies and classifies the plant under investigation as a weed, the data processing device 25 will transmit a control signal to the robot 20, configuring the robot 20 to destroy the weed 23.
- Figure 2b shows a weeding robot 20 according to an embodiment of the invention in an area 22 of cultivation of forest tree seedlings 21 .
- Figure 2b differs from the embodiment of Fig. 2a in that the data processing device in Fig. 2b is a cloud server 26, and instead of a camera the weeding robot 28 comprises both a laser scanner sensor and a gamma sensor for collecting data which is transmitted to the cloud server 26 for analysis by a plant identification algorithm.
- Figures 3a-d show the operation of a weeding robot 30 according to an embodiment of the invention.
- the weeding robot 30 moves in an area
- the forest tree seedlings 31 are planted in nursery trays 34. In addition to the seedlings 31 , there are weeds
- the weeding robot 30 captures images of the plants with its camera, i.e. captures an image 35 of each plant 36 under investigation. An image 35 of a plant 36 under investigation is shown in Fig. 3b.
- the weeding robot 30 transfers the image data of the image 35 of the plant 36 under investigation, captured by the camera, to its integrated data processing device 37 for analysis by a plant identification algorithm.
- the data processing device 37 compares the plant of the image data under investigation with plants in an image database stored in its memory. In this embodiment, the data processing device 37 identifies and classifies the plant 36 under investigation as a weed, and the data processing device 37 configures the robot 30 by a control signal to destroy the weed 36.
- Figure 3c shows a situation in which the robot 30 has started destoying of the weed 36 under investigation by its destroying means 38, by pushing the weed into the ground, i.e. the growing medium 39.
- Figure 3d shows a situation in which the weed 30 under investigation has been totally pushed into the growing medium 39, i.e. weeded i.e. destroyed.
- Figure 4 shows a flow chart of a weeding method 40, i.e. a weed destruction method, of a weeding robot according to an embodiment of the invention.
- the sensor of the weeding robot is applied to collect data on a plant under investigation, growing in an area of cultivation of tree seedlings in a nursery.
- the sensor may be an image sensor.
- the collected data on the plant under investigation is transferred to a computer device for analysis by an algorithm.
- the data may be image data and the algorithm may be an image-based plant identification algorithm.
- a third step 43 as a result of the analysis of the data, at least one control signal is received for controlling the operation of the robot.
- the control signal may be, for example, a command to destroy a plant identified as a weed, or a command to continue the capturing of sensor data, such as image data.
- the weeding robot and the weed destruction or weeding method according to the invention also comprise other devices and/or means for transferring sensor data or for controlling the operation of the robot.
- the robot comprises means for computing and storing data on the amount of weeds destroyed and the ratio between the amount of weeds and the amount of tree seedlings.
- the robot comprises means for computing and storing data on the amount of weeds destroyed and the ratio between the amount of weeds and the amount of tree seedlings according to the area, whereby the robot is configured to move to capture images in area where previously more weeds have been found.
- the weeding robot according to the invention is configured to move among seedlings other than tree seedlings.
- the aim of the weeding method is to maintain these other seedlings and to destroy weeds around them.
Landscapes
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mechanical Engineering (AREA)
- Insects & Arthropods (AREA)
- Pest Control & Pesticides (AREA)
- Wood Science & Technology (AREA)
- Zoology (AREA)
- Environmental Sciences (AREA)
- Guiding Agricultural Machines (AREA)
Abstract
The invention relates to a weeding robot (30) which is configured to move among tree seedlings (31) growing in the cultivation area (32) of a nursery, and which weeding robot (30) comprises a sensor for collecting data on a plant (36) under investigation in the cultivation area (32), and data transmission means for transferring data collected on the plant (36) under investigation to a data processing device (37) for analysis by a plant identification algorithm, as well as least one weed destroying means (38). Furthermore, as a result of analyzing image data, the data transmission means are configured to receive at least one control signal for controlling the operation of the robot. The invention also relates to a weeding method and a data processing device (37) for controlling the weeding robot (30).
Description
WEEDING ROBOT
Field of the invention
The present invention relates to a weeding robot to be used for killing weeds, i.e. unwanted plants, in nurseries, as well as a method for killing weeds, and a computer program for performing the method.
Background of the invention
A nursery is an area for growing tree seedlings to be planted elsewhere. Nurseries produce healthy tree seedlings and forest planting materials of good quality for various areas, for example for forest cultivation, forests, and cutting areas. In a good seedling, both the stem and the root are of high quality. Therefore, careful monitoring and care during cultivation are important. Killing weeds, i.e. weeding, is an important part of the care. Manual weeding, however, is both laborious and time-consuming and thus requires a lot of labour and time, which increases costs in nurseries and thereby also the prices of tree seedlings and possibly other forest planting materials grown in the nursery.
Brief summary of the invention
It is an aim of the present invention to provide and to present a novel weeding robot, as well as a method for killing weeds with the weeding robot, and a computer program for performing the method. The invention is characterized in what will be presented in the independent claims, and the dependent claims relate to advantageous embodiments of the invention.
According to a first aspect, the invention relates to a weeding robot which is configured to move among tree seedlings growing in the cultivation area of a nursery, the weeding robot comprising a sensor for collecting data on a plant under investigation in the cultivation area, and data transmission means for transferring the data collected on the plant concerned to a data processing device, to be analyzed by a plant identification algorithm, as well as least one weed destroying means. Furthermore, as a result of analyzing image data, the
data transmission means are configured to receive at least one control signal for controlling the operation of the robot.
According to an example, the weeding robot comprises moving means for moving among tree seedlings growing in the cultivation area of the nursery. According to an example, the moving means comprise at least two pairs of legs fastened to the side surfaces or the bottom surface of the robot. According to an example, the moving means comprise wheels provided on the bottom surface of the robot, for moving along moving surfaces provided in the cultivation area. According to an example, the control signal is arranged to configure the robot to remove the plant under investigation by its weed destroying means, if the plant is identified as a weed by the plant identification algorithm. According to an example, the control signal is arranged to configure a sensor to repeat collecting of data on the plant under investigation, if the data is determined to be of poor quality. According to an example, the weed destroying means is a push rod for pushing the weed into the ground. According to an example, the data processing device is a part of the weeding robot. According to an example, the weeding robot further comprises a battery and means for receiving electric energy in a wireless manner. According to an example, the weeding robot comprises a distance sensor for detecting a plant in the cultivation area. According to an example, the sensor is a camera and the collected data is image data.
According to a second aspect, the invention relates to a method for weeding. The method comprises capturing data on a plant under investigation in the tree seedling cultivation area of a nursery by a sensor of a weeding robot according to the first aspect, or an example thereof; transferring the data collected on the plant under investigation to a computer device, to be analyzed by a plant identification algorithm; and receiving, as the result of the analysis of the data, at least one control signal for controlling the operation of the robot.
According to an example, the method also comprises destroying the plant under investigation by the weed destroying means of the weeding robot, in response to the received control signal. According to an example, the method further comprises repeating the collecting of data on the plant under investigation, and transferring the re-collected data to the data processing device, to be analyzed with a plant identification algorithm. According to an example, the
method further comprises moving the weeding robot in the area for cultivation of tree seedlings of the nursery by means of spider-like legs of the weeding robot. According to an example, the method further comprises moving the weeding robot along moving surfaces provided in the area for cultivation of tree seedlings of the nursery by means of wheels on the bottom surface of the weeding robot.
According to a third aspect, the invention relates to a computer software product which is stored in a computer-readable medium and is executable in a data processing device. The computer software product comprises instructions which make a weeding robot according to the first aspect, or an example thereof, perform the weeding method according to the second aspect, or an example thereof.
Description of the drawings
In the following, the present invention will be described in more detail with reference to the appended drawings, in which
Fig. 1a-c shows a weeding robot according to an embodiment of the invention,
Fig. 2a-b shows a weeding robot according to an embodiment of the invention in a cultivation area,
Fig. 3a-d shows a weeding robot according to an embodiment of the invention in operation, and
Fig. 4 shows a flow chart of a weeding method for a weeding robot according to an embodiment of the invention.
Detailed description of the invention
Nurseries, particularly forest nurseries, are areas providing young forest tree seedlings for forest cultivation, to be planted in various areas, such as forests and cutting areas. Forest tree seedlings may be, for example, spruce, pine or birch seedlings The nursery may be a greenhouse or an area intended for
SUBSTITUTE SHEET (RULE 26)
cultivation of seedlings outdoors. One nursery may comprise several cultivation areas. The cultivation areas in nurseries may be implemented, for example, by means of various nursery beds, areas or fields comprising, for example, various culture levels, areas or boxes, in which a plurality of tree seedlings are placed in their respective nursery trays, such as cells, balls or bags. Nursery trays, or ball trays, or ball compartment trays, are matrix type structures having places for cultivation or growing for several seedlings in row and column directions. They have a high planting density but the single cells are small in volume. The nursery trays may be made of, for example, rigid plastic, peat, paper, cellular plastic, or any material suitable for the purpose. One cultivation area may often comprise several or even tens or hundreds of nursery trays, for example in matrix format; in other words, one cultivation area normally comprises several rows and columns of nursery trays.
Thanks to careful monitoring and cultivation of seedlings at the cultivation stage, nurseries provide healthy tree seedlings and/or other forest planting material of good quality. Killing of weeds, i.e. unwanted plants, in the areas for cultivation of seedlings in nurseries is an important part of the care of the seedlings, because early weeding in the areas for cultivation of seedlings, already at the emergence of cotyledons, will prevent development of the weed. Weeding improves the growth environment of the tree seedlings, because weeds which deprive the seedlings of space and nutrients are removed from the rhizosphere of the seedlings. Weeding also prevents weeds from ending up in the final planting sites of the seedlings. However, weeding, for example by pulling up manually, takes up a lot of workers’ time and effort, which increases nursery costs and prices of tree seedlings and other forest planting materials. Furthermore, for example the packing of cultivation trays tightly together in the cultivation area causes challenges of reaching the weeds; that is, weeds growing in the more central areas of the cultivation area cannot be destroyed/pulled out manually without moving the cultivation trays, which further increases the work needed for weeding. In forest vegetation, chemical weed control is not possible or not desirable, whereby weeding has to be performed mechanically, for example manually.
To achieve better cost efficiency, a solution is needed which reduces workers’ time and effort needed for weeding. One solution to this weed problem is a weeding robot according to the invention, i.e. a robot for destroying unwanted
plants. The robot itself can be implemented in a variety of ways, as well as its movement among seedlings in cultivation areas, but in the solutions, a plant detection algorithm is used for analyzing data collected by a sensor or sensors of the weeding robot in cultivation areas and relating to a plant under investigation, and if the plant under investigation is identified as a weed, it is killed by the robot.
The plant detection algorithm can identify plants by comparing data collected by sensors with sensor data in a database. For each respective sensor, a sufficient amount of reference data is stored in the database, to be used for identifying different plants or their features. The plant to be identified may thus be a weed or a seedling. The algorithm may be, for example, a machine learning based plant identification algorithm implemented by artificial intelligence (Al) and trained with a sufficient amount of data to identify different features of plants or different plants; or the algorithm may be another plant identification algorithm suitable for identifying plants, comparing collected sensor data with data in a database, for identifying a plant either as a weed or a seedling. Weeds to be destroyed include, for example, unwanted plants at the emergence of cotyledons, and sprouts of liverwort, which may differ from the tree seedlings in, for example, their appearance, moisture, size, or reflection spectrum.
The sensor may be, for example, an image sensor, which in this context is referred to by the term camera. The camera may be, for example, an RGB (red green blue) camera, an sRGB camera, or a CCD camera (charge-coupled device), such as a digital camera or another ordinary photographic camera. Alternatively, the camera may be a hyper spectrum camera used for hyper spectrum imaging, comparing differences in the reflection spectra of electromagnetic radiation, i.e. hyper spectrum data, of weeds and seedlings, to distinguish weeds from seedlings by an algorithm. Spectral differences are formed by slight differences in the chemical compositions of different plant species. The sensor may also be an infra-red sensor or an infra-red camera measuring infrared radiation for identifying plant species and/or for distinguishing weeds from seedlings on the basis of differences in the infra-red radiation generated by the plant species.
In addition to those mentioned above, the sensor may be, for example, a laser scanner for distance measurement, forming a point cloud whose shape and/or height can be used to identify a plant as a weed or a seedling. Data collected by the laser scanner is thus a relatively dense point cloud which is a 3D representation of the object scanned, in this case a seedling or a weed. For identifying plants, it is also possible to use any other existing identification method which is suitable to be applied by a sensor of the robot and for analysis by a plant detection algorithm. It is also possible that to increase the probability of identification, the weeding robot applies one or more of the above-presented sensors and plant identification methods as well as a database comprising data for each sensor to be used for identification of plants.
When the sensor used is a photographic camera, the weeding robot is configured to capture images with its camera when it moves in an area or areas of cultivation of tree seedlings in a nursery. With its camera, the robot captures an image comprising a plant to be investigated. The robot transmits the image data captured by its camera, i.e. the data collected on the plant under investigation, to a data processing device for analysis by a plant identification algorithm. The robot comprises the data transmission means needed for the transmission of image data, for example a transmitter or a transceiver, for transferring the image data from the camera to the data processing device in a wireless manner or via a wired connection. The data processing device comprises at least one processor, at least one memory comprising a computer program code for one or more software units, and means, for example a receiver or a transceiver, for receiving image data from a camera in a wireless manner or via a wired connection. Several processors may be provided, for example a general-purpose processor, a graphic processor and/or a digital signal processor (DSP), and/or several different memories may be provided, for example a volatile memory for storing data and software, and a non-volatile memory, such as a hard disk for permanent storage of data and software. The data processing device may be any data processing device suitable for data analysis and plant identification, such as a data processing unit or a computer. The data processing device may be electronically connected to a sensor via signal lines, via which it may also transmit control signals, i.e. configuration messages, to the robot and the sensor. The data processing device may be a part of the robot, i.e. an internal data processor in the robot, or it may be an external data processing device, such as a server or a cloud server. The image data may be
transmitted from the camera of the robot to the data processing device over a data transmission network, for example a wireless local area network (WLAN), or by GSM, CDMA or WCDMA technologies or future technologies, or other data network technologies.
The data processing device is configured to apply a plant identification system which corresponds to a face recognition system and may be a plant identification algorithm trained by machine learning or another plant identification method which performs sensor data comparison and, for example when applying an image sensor, compares an image captured of a plant under investigation with images in a database and detects if the plant under investigation matches with a plant in the image database. An appropriate algorithm and a sufficiently large amount of images of plants in the image database increase the probability of successful plant identification. In case of, for example, a sensor other than an image sensor, for example a laser scanner, the database is loaded with data on point clouds formed by different plants, such as different weeds and/or seedlings. Consequently, the plant identification is effected by training the algorithm to identify different features of different plants by supplying the algorithm with a sufficient amount of data. For this, a sufficient amount of image data or other data has to be available, to train the system to distinguish different plants from each other. The algorithms used for image recognition are based on comparing features computed from images. They compare sets of pixels with each other, and when the comparison provides a sufficient level of recognition, that is, a plant is recognized with a sufficient degree of probability to belong to a specific plant species, the plant has been identified. On the basis of the identification, the plant identification algorithm classifies the plants as, for example, weeds or tree seedlings. It is also possible to classify a plant as not dentified, if the plant is not recognized from an image, in which case it is not destroyed, or not dentified if the image shows something else than a plant, such as sawdust, stone, or peat. The plant identification also makes it possible to distinguish plants, such as seedlings and weeds, from a growing medium. The material of the growing medium, such as peat, soil, or sawdust and stones contained in it, is classified as not identified, whereby it is not destroyed.
In case of a weed, that is, when a plant is identified and classified as a weed, the data processing device instructs the robot to destroy the plant. If, for
example, 20% of weed sprouts having cotyledons can be destroyed by the weeding robot, this is of great significance in reducing manual weeding.
When an image sensor is used, the database may, according to a first example, comprise images of plants other than tree seedlings grown in the area where the robot is moving to capture images, that is, for example different weeds. The data processing device compares an image of a plant under investigation, captured by the camera of the robot, with images of plants in a database. If a match is found, that is, the plant under investigation matches with a weed in the database, at least with a required/desired degree of probability, the plant is identified and classified as a weed. The required/desired degree of probability may be defined for the data processing device in advance. After the classification as a weed, the data processing device configures/in- structs the robot, for example by means of a control signal, i.e. a configuration signal, to destroy the weed in the image and under investigation. Destroying weeds may involve, for example, pulling out the plant from the ground, pushing the plant into the ground, cutting the plant, or other mechanical destruction or removal of the plant. The robot comprises the means required for destroying the weeds. The robot may comprise, for example, a hand gripper or suction means for pulling out a weed, a push rod or the like for pushing the weed into growth peat or material, or scissors for cutting the weed.
According to a second example, the image database may comprise images of one or more tree species grown in the area where the robot is moving to capture images. The data processing device compares an image captured of a plant under investigation with images of a seedling or seedlings in the database. If a match with the seedling is not found, that is, the data processing device does not identify the plant under investigation as a seedling, the plant under investigation is identified, i.e. classified, as a weed, and the data processing device configures/instructs the robot to destroy the plant concerned and shown in the image captured by the camera of the robot. Weeding involves the same steps of pulling out the plant from the ground, pushing the plant into the ground or cutting the plant, as in the first example, and the robot has the same means required for destroying the weed, for example a hand gripper, suction means, a push rod or the like, or scissors.
According to a third example, the image database may comprise images of different plants, weeds as well as forest tree seedlings grown in the respective area. In this third example, the data processing device analyzes image data by comparing an image of a plant under investigation, captured by the camera of the robot, with images of plants in the database. If a match is found, i.e. the plant is identified, the data processing device classifies the plant either as a weed or as a forest tree seedling, on the basis of the result of the plant identification. In other words, if the plant under investigation is identified as a weed, it is classified as a weed, and if the plant under investigation is found to match with a seedling, it is classified as a seedling. In case of a weed, that is, if the plant is identified and classified as a weed, the data processing device config- ures/instructs the robot, for example by means of a control signal, to destroy the weed concerned and shown in the image captured by the camera of the robot. The destroying of weeds corresponds to the way of destoying of weeds in the above presented examples, by the means presented above.
If the plant under investigation is identified and classified as a forest tree seedling by the plant identification algorithm, i.e. for example by comparing images, the robot is configured to collect data, for example to use its camera to capture an image of the next plant and to transmit the image data of the next image for analysis, and the plant investigated and identified as a forest tree seedling is allowed to remain intact. Alternatively, the robot may continue to capture images, i.e. to collect data, by default if no instruction is received to destroy an identified plant. However, it should be noted that plant identification is not always successful, i.e. the plant is not always identified or the object to be identified is not even a plant. In such a case, the plant/object to be identified is not classified as a weed or a forest tree seedling, but it may remain intact and the robot is configured to collect data on the next plant, or it is already configured to collect data on the next plant if no signal to destroy is received, for example within a given time from the transmission of the image data. Furthermore, it is possible that during data analysis, the data processing device finds the data on the plant under investigation to be of too poor quality to run the plant identification algorithm. In such a case, the data processing device may transmit a control signal to the camera of the robot to configure the camera to re-collect data, i.e. for example to capture at least one additional image of the same plant under investigation and to transmit the collected sensor data for analysis. Or, alternatively, the robot is configured to collect data, for example to apply its
camera to capture an image of the next plant and to transmit data on the next plant, for example image data, for analysis, and the plant with the data of poor quality may remain intact.
For successful imaging of plants in the cultivation area, the weeding robot comprises means for moving among forest tree seedlings. The weeding robot may be configured to move on its legs on top of ball compartment units of nursery trays with regular shapes for seedlings, whereby no separate rails or other structures are needed for movement. When moving on its legs, the robot may utilize the structures of the nursery trays and the peat surface, i.e. soil surface, or other existing structures of the nursery, and move independently among seedlings. However, it is also possible to provide movement surfaces for the robot, i.e. rails or plane surfaces, between, by the side of, or above the tree seedlings. For moving, the weeding robot may thus be provided with legs on e.g. its bottom surface and/or side surfaces i.e. flanks, for example six legs, or three pairs of legs, like a spider. The legs are configured to move in a way similar to, for example, the legs of a spider or a crab, making it possible for the robot to move or walk among seedlings in the cultivation area, and keeping the robot in balance among seedlings even on an uneven stand/terrain. The robot may, in fact, resemble for example a spider or a crayfish, such as a crab, when moving among seedlings. In this context, the term “spider legs” comprise all similar leg types, such as crayfish legs, which comprise a sufficient number of pairs of legs for moving in the cultivation area. A sufficient number of pairs of legs is at least two pairs, but the number of pairs of legs may be greater, for example three or four pairs. The robot may have been, or may be, configured to always move and/or spin around a specific distance or for a specific time, determined in advance or by a control signal, after which it uses its sensor to collect data, for example to capture an image by a camera; or alternatively, the robot may be provided with one or more distance sensors for transmitting information to a data processing device, so that the robot can identify that an object to be imaged, such as a plant to be imaged, is in the vicinity, i.e. within the imaging area of the camera. The distance sensor may be, for example, a touch sensor, a proximity sensor, a distance sensor, or a computer vision system.
When moving among seedlings, either on its legs or wheels, the robot uses its camera to capture images of plants from the side, i.e. from the side of the
plants. Alternatively, as already mentioned above, if rails, plane surfaces or the like are provided above, i.e. on top of, the area of cultivation, along which the robot moves with wheels provided at its bottom, the robot may image the plants from above when moving above the seedlings.
After the sensor of the robot has detected an object, such as a plant, e.g. in the imaging area of the camera, or after a predetermined distance or time to the next data collecting object has been passed, the robot takes the next image, which may or may not contain the next plant under investigation, and transfers the image data of the image to the data processing device for analysis by the plant identification algorithm. In response to the analysis, the data processing device transmits a control signal to the robot, to be received by the data transmission means of the robot. The control signal controls the operation of the robot on the basis of the results of the plant identification analysis. If the plant under investigation is identified as a weed, the robot comprising at least one weed destroying means is configured to destroy the weed shown in the image. If, on the other hand, the plant under investigation is not identified as a weed, or it is identified as a seedling, or identification is not successful, the robot is configured to collect data, that is, in the case of a camera, to capture an image of the next plant. If the next plant is not within the imaging area of the robot or the sensor does not detect a plant in the imaging area, the robot may move or spin a predetermined distance or time between capturing images or for capturing an image. If, on the other hand, the image of the plant under investigation is found to be of inadequate quality for running the plant identification algorithm, the robot may be configured by a control signal to capture a new image of the same plant, or to move/spin slightly and take an image of the same plant, or to continue moving and imaging among the seedlings.
The weeding robot also comprises a battery and means for receiving electric energy in a wireless or wired manner. For wireless charging, a charging point may be arranged in the cultivation area where the robot is moving. The robot may be configured to be charged at specific intervals, whereby it will independently move to the charging point at the end of the interval, or the robot may be configured to move to the charging point when the charging level of the robot has decreased below a threshold level. After being charged, the robot may automatically return to imaging mode.
A zone of movement may be determined for the robot in advance, for example by dimension data on the area or by a copper wire to indicate the border of the cultivation area, i.e. the area in which the robot is intended to destroy weeds, to the robot. It is also possible that the robot utilizes GPS technology for positioning and comprises a GPS device by which it determines its location and is capable of moving and weeding only in the cultivation area defined for it. In addition, any other positioning method or means suitable for the purpose may be used in/for the robot for indicating the cultivation area.
Figure 1a shows a weeding robot 10a according to an embodiment of the invention. The weeding robot 10a has three pairs of legs 11 , so-called spider legs, for moving among tree seedlings growing in the cultivation area of a nursery; an image sensor, i.e. a camera 12, for imaging plants in the cultivation area; and data transmission means 13 for transferring image data to a data processing device 14, for analysis by a plant identification algorithm; as well as means for receiving, as a result of the analysis of the image data, at least one control signal for controlling the operation of the robot. In addition to these, the robot 10a comprises a weed destroying means 15. In this example, the destroying means 15 is a foldable plunger, i.e. an articulated rod, which is configured to protrude towards the ground and to destroy a weed by pushing it into the ground, if the plant identification algorithm has identified the weed. The plunger has a relatively small diameter, for example a few millimeters, so that it does not make too large a hole or entrain sawdust, placed on top of the ground, simultaneously when pushing the weed into the ground. It is also possible that the robot 10a transmits the image data to an external data processing device for analysis, and a data processing device 14 integrated in the robot 10a is used for controlling the operation of the robot 10a. The data processing device 14 comprises at least one processor; at least one memory comprising a computer software code for one or more software units for running a plant identification algorithm; and means for receiving image data from the camera 12 of the robot 10a in a wireless manner or via a wired connection, for example a receiver or a transceiver; and means for transmitting control signal in a wireless manner or via a wired connection, for example a receiver or a transceiver. Several processors may be provided, for example a general-purpose processor and a graphic processor and a digital signal processor (DSP); and/or several different memories may be provided, for example a volatile memory and
a non-volatile memory, such as a hard disk for permanent storage of data and software.
Figure 1 b shows a weeding robot 10b according to an embodiment of the invention. The weeding robot 10b corresponds to the robot 10a of Fig. 1 a in other respects, but in this embodiment, its weed destroying means 15 is not an articulated rod but a non-articulated rod of plunger type. It is also possible that the non-articulated rod is not a plunger-type rod but a telescopic arm which is short in situations other than pushing a weed into the ground, i.e. destroying it, in order not to block the way when the robot is moving among seedlings and weeds, and is extended when pushing a weed into the ground. The weeding robot 10b also comprises a so-called distance sensor 16 by which the weeding robot 10b detects the plants in its vicinity, seedlings as well as weeds. The distance sensor 16 may be a touch sensor, a proximity sensor, a distance sensor, or a computer vision system. The sensor 16 may transmit the information detected by it to a data processing device 14 which analyzes, on the basis of the information from the sensor 16, whether there is a need for collecting data, for example, capturing images.
Figure 1c shows a weeding robot 10c according to an embodiment of the invention. The weeding robot 10c corresponds to the robot 10b of Fig. 1 b in other respects, but its weed destroying means 15 is a hand gripper; moreover, it has a different appearance. The sensor 16 in the figure is a hyper spectrum camera.
It should be noted that the weed destroying means may also be placed in another part of the robot than in its front, for example at the bottom of the robot, whereby the plunger can be used and a weed can be pushed into the ground by bending the legs of the robot, and the plunger does not have to be pushed separately.
Figure 2a shows a weeding robot 20 according to an embodiment of the invention, in an area 22 for cultivation of forest tree seedlings 21 . The forest tree seedlings 21 are planted in nursery trays 24. The weeding robot 20 moves on its legs among plants, i.e. seedlings 21 and weeds 23, in the cultivation area 22, and captures images of the plants with its camera 27. In this example, the weeding robot 20 transfers the image data of the image captured by the
camera 27 on a plant under investigation, to an external data processing device 25 for analysis by a plant identification algorithm. In this example, the data processing device 25 is a server, but it may also be any computer device suitable for the purpose. The data processing device 25 compares the image data of the plant under investigation with plant images in the database by means of an algorithm. If the data processing device 25 identifies and classifies the plant under investigation as a weed, the data processing device 25 will transmit a control signal to the robot 20, configuring the robot 20 to destroy the weed 23.
Figure 2b shows a weeding robot 20 according to an embodiment of the invention in an area 22 of cultivation of forest tree seedlings 21 . Figure 2b differs from the embodiment of Fig. 2a in that the data processing device in Fig. 2b is a cloud server 26, and instead of a camera the weeding robot 28 comprises both a laser scanner sensor and a gamma sensor for collecting data which is transmitted to the cloud server 26 for analysis by a plant identification algorithm.
Figures 3a-d show the operation of a weeding robot 30 according to an embodiment of the invention. In Fig. 3a, the weeding robot 30 moves in an area
32 for cultivating forest tree seedlings 31. The forest tree seedlings 31 are planted in nursery trays 34. In addition to the seedlings 31 , there are weeds
33 in the cultivation area 32. The weeding robot 30 captures images of the plants with its camera, i.e. captures an image 35 of each plant 36 under investigation. An image 35 of a plant 36 under investigation is shown in Fig. 3b. In this example, the weeding robot 30 transfers the image data of the image 35 of the plant 36 under investigation, captured by the camera, to its integrated data processing device 37 for analysis by a plant identification algorithm. The data processing device 37 compares the plant of the image data under investigation with plants in an image database stored in its memory. In this embodiment, the data processing device 37 identifies and classifies the plant 36 under investigation as a weed, and the data processing device 37 configures the robot 30 by a control signal to destroy the weed 36. Figure 3c shows a situation in which the robot 30 has started destoying of the weed 36 under investigation by its destroying means 38, by pushing the weed into the ground, i.e. the growing medium 39. Figure 3d shows a situation in which the weed 30 under
investigation has been totally pushed into the growing medium 39, i.e. weeded i.e. destroyed.
Figure 4 shows a flow chart of a weeding method 40, i.e. a weed destruction method, of a weeding robot according to an embodiment of the invention. In a first step 41 , the sensor of the weeding robot is applied to collect data on a plant under investigation, growing in an area of cultivation of tree seedlings in a nursery. The sensor may be an image sensor. In a second step 42, the collected data on the plant under investigation is transferred to a computer device for analysis by an algorithm. In the case of an image sensor, the data may be image data and the algorithm may be an image-based plant identification algorithm. In a third step 43, as a result of the analysis of the data, at least one control signal is received for controlling the operation of the robot. The control signal may be, for example, a command to destroy a plant identified as a weed, or a command to continue the capturing of sensor data, such as image data.
It is thus possible that the weeding robot and the weed destruction or weeding method according to the invention also comprise other devices and/or means for transferring sensor data or for controlling the operation of the robot. Furthermore, it is possible that the robot comprises means for computing and storing data on the amount of weeds destroyed and the ratio between the amount of weeds and the amount of tree seedlings. In addition to these, it is possible that the robot comprises means for computing and storing data on the amount of weeds destroyed and the ratio between the amount of weeds and the amount of tree seedlings according to the area, whereby the robot is configured to move to capture images in area where previously more weeds have been found.
Furthermore, it is possible that the weeding robot according to the invention is configured to move among seedlings other than tree seedlings. In this case, the aim of the weeding method is to maintain these other seedlings and to destroy weeds around them.
It will be obvious that the present invention is not limited solely to the abovepresented embodiments but it can be modified within the scope of the appended claims.
Claims
1. A weeding robot which is configured to move among tree seedlings growing in a cultivation area of a nursery, and which weeding robot comprises a sensor for collecting data on a plant under investigation in the cultivation area, and data transmission means for transferring data collected on the plant under investigation to a data processing device for analysis by a plant identification algorithm, characterized in that the robot further comprises at least one weed destroying means, and that, as a result of the analysis, the data transmission means are further configured to receive at least one control signal for controlling the operation of the robot.
2. The weeding robot according to claim 1 , the weeding robot comprising moving means for moving among tree seedlings growing in the cultivation area of the nursery.
3. The weeding robot according to claim 2, wherein the moving means are at least two pairs of legs attached to the side surfaces or the bottom surface of the robot.
4. The weeding robot according to claim 2, wherein the moving means are wheels provided on the bottom surface of the robot for moving along moving surfaces provided in the cultivation area.
5. The weeding robot according to any of the preceding claims, wherein the control signal is arranged to configure the robot to remove the plant under investigation by its weed destroying means, if the plant is identified as a weed by the plant identification algorithm.
6. The weeding robot according to any of the preceding claims, wherein the control signal is arranged to configure a sensor to re-collect data on the plant under investigation, if the data is determined to be of inadequate quality.
7. The weeding robot according to any of the preceding claims, wherein the weed destroying means is a push rod for pushing the weed into the ground.
8. The weeding robot according to any of the preceding claims, wherein the data processing device is a part of the weeding robot.
9. The weeding robot according to any of the preceding claims, wherein the weeding robot further comprises a battery and means for receiving electric energy in a wireless manner.
10. The weeding robot according to any of the preceding claims, comprising a distance sensor for detecting a plant in the cultivation area.
11 . The weeding robot according to any of the preceding claims, wherein the sensor is a camera and the collected data is image data.
12. A method for weeding, wherein the method comprises capturing data on a plant under investigation in the tree seedling cultivation area of a nursery by a sensor of a weeding robot according to any of the claims 1 to 11 ; transferring the data collected on the plant under investigation to a computer device for analysis by a plant identification algorithm; characterized in that the method further comprises receiving, as the result of the analysis of the data, at least one control signal for controlling the operation of the robot.
13. The method according to claim 12, the method further comprising removing the plant under investigation by the weed destroying means of the weeding robot, on the basis of the received control signal.
14. The method according to claim 13, the method further comprising re-collecting data on the plant under investigation by means of the sensor of the weeding robot, and transferring the re-collected data to the computer device for analysis by the plant identification algorithm.
18
15. The method according to any of the claims 12 to 14, the method further comprising moving the weeding robot in the area for cultivation of tree seedlings of the nursery by means of spider-like legs of the weeding robot.
16. The method according to any of the claims 12 to 14, the method further comprising moving the weeding robot along moving surfaces provided in the area for cultivation of tree seedlings of the nursery by means of wheels on the bottom surface of the weeding robot.
17. A computer software product which is stored in a computer-readable me- dium and is executable in a data processing device, the computer software product comprising commands which make a weeding robot according to any of the claims 1 to 11 perform a weeding method according to any of the claims 12 to 16.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/FI2021/050866 WO2023105112A1 (en) | 2021-12-10 | 2021-12-10 | Weeding robot |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/FI2021/050866 WO2023105112A1 (en) | 2021-12-10 | 2021-12-10 | Weeding robot |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023105112A1 true WO2023105112A1 (en) | 2023-06-15 |
Family
ID=86729694
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/FI2021/050866 WO2023105112A1 (en) | 2021-12-10 | 2021-12-10 | Weeding robot |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2023105112A1 (en) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2019133575A1 (en) * | 2017-12-29 | 2019-07-04 | Dcentralized Systems, Llc | Autonomous mobile platform with harvesting system and pest and weed suppression systems |
WO2019144231A1 (en) * | 2018-01-25 | 2019-08-01 | Eleos Robotics Inc. | Autonomous unmanned ground vehicle and handheld device for pest control |
US10548306B2 (en) * | 2015-05-29 | 2020-02-04 | Robert Bosch Gmbh | Weed control device |
WO2021067520A1 (en) * | 2019-10-01 | 2021-04-08 | X Development Llc | Robotic agricultural remediation |
EP3811748A1 (en) * | 2019-10-24 | 2021-04-28 | Ekobot Ab | A weeding machine and a method for carrying out weeding using the weeding machine |
EP3815510A1 (en) * | 2019-10-31 | 2021-05-05 | Dahlia Robotics GmbH | Plant removal apparatus and method |
-
2021
- 2021-12-10 WO PCT/FI2021/050866 patent/WO2023105112A1/en unknown
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10548306B2 (en) * | 2015-05-29 | 2020-02-04 | Robert Bosch Gmbh | Weed control device |
WO2019133575A1 (en) * | 2017-12-29 | 2019-07-04 | Dcentralized Systems, Llc | Autonomous mobile platform with harvesting system and pest and weed suppression systems |
WO2019144231A1 (en) * | 2018-01-25 | 2019-08-01 | Eleos Robotics Inc. | Autonomous unmanned ground vehicle and handheld device for pest control |
WO2021067520A1 (en) * | 2019-10-01 | 2021-04-08 | X Development Llc | Robotic agricultural remediation |
EP3811748A1 (en) * | 2019-10-24 | 2021-04-28 | Ekobot Ab | A weeding machine and a method for carrying out weeding using the weeding machine |
EP3815510A1 (en) * | 2019-10-31 | 2021-05-05 | Dahlia Robotics GmbH | Plant removal apparatus and method |
Non-Patent Citations (1)
Title |
---|
WALZ JÖRG-DIETER: "Mobile weed killer for tree nurseries", FRAUNHOFER, 1 December 2021 (2021-12-01), XP093073437, Retrieved from the Internet <URL:https://www.fraunhofer.de/en/press/research-news/2021/december-2021/mobile-weed-killer-for-tree-nurseries.html> [retrieved on 20230814] * |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11445665B2 (en) | Plant treatment based on morphological and physiological measurements | |
Ampatzidis et al. | iPathology: robotic applications and management of plants and plant diseases | |
US10568316B2 (en) | Apparatus and methods for in-field data collection and sampling | |
Steward et al. | The use of agricultural robots in weed management and control | |
de Luna et al. | Tomato growth stage monitoring for smart farm using deep transfer learning with machine learning-based maturity grading | |
Kurtulmuş et al. | Detecting corn tassels using computer vision and support vector machines | |
CN114818909B (en) | Weed detection method and device based on crop growth characteristics | |
KR102297913B1 (en) | Plant growth monitoring system using hyperspectral reflected light and fluorescence scattering, and method thereof | |
Mishra et al. | Weed Species Identification in Different Crops Using Precision Weed Management: A Review. | |
Adhikari et al. | 3D reconstruction of apple trees for mechanical pruning | |
US11927551B2 (en) | Systems and methods for using backscatter imaging in precision agriculture | |
Hennessy et al. | Evaluation of cameras and image distance for CNN-based weed detection in wild blueberry | |
CN115661647A (en) | Timed inspection method for tree planting growth conditions | |
US20220207852A1 (en) | Generating a ground plane for obstruction detection | |
Gregorio et al. | Sensing crop geometry and structure | |
Jiang et al. | Automatic Localization of Soybean Seedlings Based on Crop Signaling and Multi-View Imaging | |
WO2023105112A1 (en) | Weeding robot | |
Duman et al. | Design of a smart vertical farming system using image processing | |
Moghaddam et al. | Developing a selective thinning algorithm in sugar beet fields using machine vision system | |
CN107656287B (en) | A kind of Boundary Extraction device and method of the crudefiber crop row based on laser radar | |
Sharma | Recognition of Anthracnose Injuries on Apple Surfaces using YOLOV 3-Dense | |
Poleshchenko et al. | Development of a System for Automated Control of Planting Density, Leaf Area Index and Crop Development Phases by UAV Photos | |
Yang | Maize and Sorghum Plant Detection at Early Growth Stages Using Proximity Laser and Time-Offlight Sensors | |
CN118285234A (en) | Mutant plant material picking system and method | |
Matache et al. | TENDENCIES REGARDING USAGE OF INTELLIGENT ALGHORITHMS OF IMAGE RECOGNITION WITHIN AGRICULTURAL CROP MAINTENANCE WORKS |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21967046 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |