WO2020142822A1

WO2020142822A1 - Device provided with a viewing and identification system installed on agricultural equipment

Info

Publication number: WO2020142822A1
Application number: PCT/BR2020/050001
Authority: WO
Inventors: Eduardo MARCKMANN; Gabriel DE MORAIS BORGES; Diógenes DOS SANTOS MACHADO; Ismael VITOR SCHEFFLER; Jéferson CARDOSO DO ROSÁRIO
Original assignee: Eirene Projetos E Consultoria Ltda
Priority date: 2019-01-10
Filing date: 2020-01-02
Publication date: 2020-07-16
Also published as: BR102019000530A2; BR102019000530B1

Abstract

The invention pertains to the technological field of agricultural implements and relates to a device provided with a viewing system installed in a sprayer vehicle in order to select the plants to be sprayed, that consists of an electronic assembly made up of cameras (1.3), LEDs (1.5) and (1.6) and at least one processing unit with control of the herbicide-application valves (5). A method is provided, which uses a multispectral camera (1.3) and blue (1.5) and infrared (1.6) LEDs installed in the spraying boom, in order to identify the presence and type of plant, pest or disease, controlling the spraying valves (5) independently. The method includes a fully mobile structure that can be installed in different land-based agricultural vehicles, involving less investment and offering a simpler pest control process. The booms can be changed so that they are able to select the plants that require spraying, thus preventing waste.

Description

DEVICE WITH A VISION AND IDENTIFICATION SYSTEM INSTALLED IN AGRICULTURAL EQUIPMENT

Technological sector of the invention

[1] The present invention relates to a vision system installed in a spray vehicle to make the selection of the plants to be sprayed. An electronic set consisting of cameras, LEDs and a processing center with control of herbicide application valves, which can be installed in agricultural spray vehicles already on the market.

State of the art of the invention

[2] Currently, the focus on technological advances of the most diverse automation companies is being focused on the automation of the agricultural area. In a few decades, these advances have changed the agricultural landscape of the world, many of which are related to cultivation and harvesting. However, even with this advance, the availability of automated options in the area of care and treatment of plants and soil for planting is still a failure.

[3] In order to have a healthier plantation, it is necessary to remove plants that can harm planting, plants that are commonly known as weeds. To remove these plants, herbicides are used, that is, a chemical treatment. This chemical treatment, which is usually done by spraying the compost, performs a procedure known as desiccation, being a practice for the elimination of all existing vegetation before sowing the crops.

[4] Aiming at this situation, many companies have focused on automating the spraying of plants, with different methods of selecting and applying pesticides, herbicides and the like. The instrument most used in large rural properties for the application of agricultural inputs is known as a self-propelled sprayer, which consists of a vehicle with a tank for the storage of the input, more specifically a spray tank, and bars of varying sizes from 15 to 48 meters that can be located on the front, center or rear of the vehicle. The bars in question have spray nozzles to distribute the input in the desired area and, generally, the spacing between the nozzles is 35cm or 50cm.

[5] However, self-propelled sprayers currently available to consumers apply spraying without selecting the plants that really need the application of the chemical. With this, the spraying operation of these known and commercially available equipment is applied to soil or plants that would not require treatment and / or application of a herbicide, wasting the input, generating economic losses and losses and / or impacts to the environment. .

[6] In order to identify which plants should be sprayed or treated, many inventions use multispectral cameras that, in addition to capturing images, capture the electromagnetic waves reflected by the plants, which are then used to process the image. With this, through the visualization of the planting through cameras it is possible to identify or select the plants to be treated. Other inventions use digital cameras accompanied by sensors to capture electromagnetic waves.

[7] In this context, looking for a solution to this problem, many companies resorted to vision systems to identify plants that need treatment, with this, several types of models and methods were presented to the market. The documents CN104034710, CN103091296 and CN203178188 describe the construction of a platform that, through cameras, sensors, filters and LEDs, identify the health of a plant that has been placed at its base. With that, the documents solve the problem of identifying which plants need treatment, but do not apply the treatment. Furthermore, the proposed solution does not identify the plants in the field, but on a platform.

[8] To solve the problem of the previous documents, the document CN103636582 provides a method of selecting plants, using sensors that identify them by the reflected light of the plants in a greenhouse. From this document, it becomes unnecessary to move the plant to a platform for analysis, but it still does not solve the problem in an open field, but only in a greenhouse, limiting the use of the proposed invention.

[9] In order to apply the treatment in the open field, the document CN106417234 uses sensors installed on a mobile base to identify the plant and apply pesticide. Despite solving the problem of previous documents, the invention of this document only applies and identifies plants in a certain area, limiting itself to the movement of the base. Furthermore, such an invention does not identify which plant, it only differentiates the plant from a human body, so it can still apply the spray to plants that do not need it, which causes waste and does not solve the problem of the state of the art.

[10] The document CN206056578 describes the construction of a drone comprising a multispectral camera for agricultural use. But, the document does not describe for what purpose the drone captures the images. Thus, the document does not solve the problem of automatic application, but provides for the identification of plants through the use of cameras that capture, through filters, different wavelengths for a central to be able to process the images.

[1 1] To solve the problems of the documents described above, the document CN106564599 provides for the use of a multispectral camera installed in a drone that captures images of plants located in an area previously flown by the same drone, the images are sent to a central installed in a fixed base that processes the images and defines the flow of the spray input applied by a second drone. Despite identifying and applying the pesticide, the invention of the document in question is expensive and still requires a fixed base to perform the processing. Furthermore, to perform the analysis, the invention requires the drone to calculate the flight area and plan the best route, after that first step, it is installed, manually, a multispectral camera, after processing the images a second drone is used for pesticide application.

[12] With this, despite the state of the art foresees the use of LEDs, cameras and image processing to control the application of pesticides, herbicides and the like, a solution where simple and mobile spray control is not yet known is known. . In addition, a solution that can be applied to land vehicles or spray bars from a simple installation on equipment and devices already established on the market has not yet been presented on the market.

Summary of the Invention

[13] The invention provides a device that makes use of an innovative method for identifying plants, using a multispectral camera, accompanied by blue and infrared LEDs installed in agricultural equipment to identify the presence or not of plants in the field.

[14] Therefore, the present invention proposes, in general, a compact and modular device that performs the capture and processing of images for the control of application of pesticides, herbicides and the like. In other words, from the installation of the device subject to protection in this report, it is possible that the spray control is carried out in a simple and mobile way. Therefore, a spray bar, to identify the presence or absence of plants and to control the spray valves independently. In addition, the method features a completely mobile construction, which can be installed in different land-based agricultural vehicles, presenting a lower cost and a simpler process. With this, the automatic spray bars that are already known in the current market, can be changed to be able to select the plants that need spraying, avoiding waste.

[15] Therefore, the present invention has the functionality of crop analysis to detect certain species of plants and / or weeds, pests or diseases. Through the images acquired by the system camera and from the computational capacity of the processing plant, the present invention is able to identify not only the plant and the soil, but also to classify what is being identified to perform more specific sprayings and provide real-time information on current planting conditions.

[16] Finally, it is known that to remove weeds resistant to the most common herbicides, it is necessary to use more specific products that are often more expensive and more toxic, thus generating financial and environmental impacts. In this case, the identification of the species provided by the present invention is important for the spraying to be located, thus avoiding waste and environmental impact due to unnecessary chemical deposition. In relation to pests and diseases, the present invention allows the producer to have dominion over the monitoring of planting on a routine basis, taking into consideration the species, the level of infestation and the region in which a certain plant or disease is present.

Description of the figures

[17] Figure 1 represents the front perspective view of the device, according to the present invention.

[18] Figure 2 represents the rear perspective view of the device, according to the present invention.

[19] Figure 3 represents the exploded perspective view of the sensor, according to the present invention.

[20] Figure 4 represents, for example, an image captured by the camera, according to the present invention.

[21] Figure 5 represents, for example, an image processed by the system, according to the present invention.

[22] Figure 6 represents, for example, a binary image, according to the present invention.

Detailed description of the fiquras [23] In general, the present invention consists of a vision and identification device and system installed in agricultural equipment, preferably in the spray bars of agricultural equipment. From the installation of the equipment proposed in this specification, more specifically in agricultural spraying vehicles, the system is able to detect plants and spray only where necessary. Therefore, through a camera and computer vision algorithms, the proposed solution differentiates plants and soil and only triggers spraying when it finds a plant.

[24] The present invention basically comprises at least one sensor (1) fixed to the dry bar (3) of the spraying vehicle by a plate (4), as shown in figures 1 and 2. Thus, the sensor assembly (1) is it gives the support (2) that is fixed to the dry bar (3) of the spraying vehicle by at least one plate (4). The sensor (1) connects to at least one valve (5) per connector (6). The valve (5) is connected to at least one nozzle holder (10), where a nozzle (11) is installed. The nozzle holder (10) is fixed to the wet bar (7) of the spraying vehicle by “U” plates (8) which are fixed to the support (2) of the sensor device (1) by screws (9). With this, the sensor device (1) can control the spraying via valve (5).

[25] The valve (5) is powered via the connector (6) by the sensor device (1), which is powered via the power cable (13) and power connector (14), which comprises a CAN bus, through which it is carried out communication with a man-machine interface located in a central farm equipment cabin. With this mechanical configuration, the installation of the sensor device (1) can be adaptable to different land vehicles that are common in the agricultural sector, since the sensor (1) is preferably fixed directly on the dry bar (3) of the spraying vehicles.

[26] The assembly of the sensor device (1) of the present invention is further detailed in FIG. 3. The sensor device (1) consists of a box (1.1) to which a PCB board (1.2) is attached, where at least one multispectral camera (1.3) with at least one filter (1.4), preferably a blue filter, arranged between the multispectral camera (1.3) and the box (1.1) being fixed by pressure. Below the multispectral camera (1.3) are installed on the PCB board (1.2) blue LEDs (1.5) and infrared LEDs (1.6), at least one display (1.7) is installed between the LEDs (1.5 and 1.6) and the box (1.1) . At the bottom of the box (1.1) passers (1.8) are installed to pass the cables that go to the connector (6) and a power cable, while at the rear a cover (1.9) is mounted to close the box (1.1) and thus protect the sensor components (1) from dust and other impurities and / or weather.

[27] The multispectral camera (1.3) captures an image, as shown in figure 4, and the electromagnetic waves reflected by the plants. Plants absorb the wavelengths relative to the blue color, 450 to 495 nm, which are emitted by the blue LEDs (1.5) and the wavelength relative to the infrared, which is emitted by the infrared LEDs (1.6), is also reflected. The electromagnetic waves are captured by the multispectral camera (1.3) through the filter (1.4), which consists of a film of translucent blue material, which allows the multispectral camera to capture only wavelengths in the 450 to 495 nm range and the infrared range. The information captured by the multispectral camera (1.3) is sent to a microprocessor located on the PCB board (1.2) where the information is processed.

[28] The electromagnetic waves are then processed by the microprocessor located on the PCB board (1.2) and the system generates a vegetative index that can vary from 0 to 1. With this index, the image (FIG. 4) is processed, identifying what it is a plant and what is soil by the vegetative index. After the image is processed (FIG. 4), a second image (FIG. 5) is generated, where the separation by pixel of what is soil and what is plant is already done. To activate the spraying system, a binary image is generated, as shown in figure 6 from the second image (FIG. 5), which considers only pixels that are greater than 0.2, making the image appear make it black (pixels below 0.2, that is, soil) and white (pixels above 0.2, that is, plant). This binary image (FIG. 6) is used to calculate the percentage of plants present in the image, and, if the calculated percentage is greater than the minimum percentage previously defined by the user via the human-machine interface, the sensor (1), via connector (6), activates the valve (5) applying the input. If it is not higher than the percentage defined by the user, the valve (5) will not be activated, avoiding waste of the input.

[29] The invention features a control panel that is located in the vehicle's cabin and communicates with the sensor device (1) through a CAN bus installed in the connector (14) to receive and send information. The control unit also acquires the vehicle speed so that the sensor (1) can adjust the spray response time according to the speed. The plant comprises a commercial human-machine interface adapted to be able to send and receive information. This plant has the capacity to connect with more than one sensor device (1) and, thus, it allows that all valves of the spraying vehicle have at least one sensor device (1) installed and all of them communicating with the plant installed in the cabin.

[30] From this constructivity, the sensor device (1) can detect pests, weeds and certain types of plants. For this, the sensor device (1) comprises an image bank classified according to the characteristics of each pest, plant and weed to train neural networks located in the sensor's microprocessor (1). With this, it is possible to make a comparison with the images captured by the multispectral camera (1.3), being able to return the type of plant, the ideal herbicide and the quantity to be applied. The captured images are processed in the microprocessor of the PCB board (1.2) in order to extract the necessary information for identification, such as format and texture, via the previously described process. After making this comparison, the sensor (1) the system performs or not the application of herbicide according to characteristics found in previously processed images. For this, the user can configure, in the cabin, the type of plant or pest to be detected, so that the type of herbicide to be applied is defined.

[31] Thus, more specifically, the process of detecting a species of plant, pest or disease, according to the present invention, is done through a set of algorithms based on neural networks located in the processing unit. These neural networks were developed and trained according to the specifications of a database of previously classified images. The images acquired by the camera are, therefore, processed to extract information of format, texture and vegetative index and then this information is inserted in the neural network which, in turn, will return with the most appropriate answer. The user will be able to specify the type of operation he will perform through the control center. For example, if the user wants to specify the application of a herbicide only when he finds a certain species of plant, just configure that species to be sprayed and, thus, the system through the algorithm activates the neural network corresponding to what the user defined. Pests and diseases can also be defined and configured in this way. Thus, chemical pesticides such as herbicides, fungicides, insecticides will be applied by the system in an optimized way in planting.

[32] In addition, to avoid waste and reduce dispersion in the environment, the flow of herbicide is controlled by two other factors acquired via image processing. The first factor to be considered is the biomass index, by which the green mass that must receive the treatment previously defined is calculated, being obtained via counting of white pixels at the moment of processing the image obtained through a multispectral camera (1.3). In this way, it is possible to control the chemical dose that will be applied, since low biomass requires less input compared to high biomass detected. Also taken into account is the nitrogen, obtained through the relation between the calculated vegetative index and the concentration of leaf nitrogen. This relationship establishes that the higher the vegetative index, the higher the nitrogen concentration. On the other hand, the lower the vegetative index, the lower the nitrogen concentration, suggesting a deficiency of this component and indicating the need for a higher dose of this input. With this, the sensor (1) can define precisely and precisely, how much input will be released by the valve (5) and applied to the plant in question.

[33] Finally, it should be noted that this application dose control is called “variable rate application”. In the present invention, the dose variation is done through pulse modulation, which starts from the processing unit to the control valve (5).

[34] It is important to note that the figures and descriptions made do not have the ability to limit the forms of execution of the inventive concept now proposed, but rather to illustrate and make understandable the conceptual innovations revealed in this solution. Thus, the descriptions and images must be interpreted in an illustrative and non-limiting way, and there may be other equivalent or similar ways of implementing the inventive concept now revealed and that do not escape the protection spectrum outlined in the proposed solution.

[35] This specification describes an improvement applied to the vision system for agricultural application, with novelty, inventive step, descriptive sufficiency, industrial application and, consequently, covered with all the essential requirements for granting the privilege claimed. .

Claims

1. DEVICE WITH A VISION SYSTEM AND

IDENTIFICATION INSTALLED IN AGRICULTURAL EQUIPMENT comprising a box (1.1) with lid (1.9), microprocessor located on a plate characterized by comprising at least one PCB plate (1 .2) consisting of at least one multispectral camera (1. 3) with filter (1 .4) and LEDs (1 .4) (1 .5); wherein said sensor device (1) is connected to at least one valve (5) per connector (6).

2. DEVICE WITH A VISION SYSTEM AND

IDENTIFICATION INSTALLED IN AGRICULTURAL EQUIPMENT according to claim 1, and further characterized by said valve (5) being energized via connector (6) by the sensor device (1) and being connected to at least one nozzle holder (10), where a spray nozzle is installed (11).

3. DEVICE WITH A VISION SYSTEM AND

IDENTIFICATION INSTALLED IN AGRICULTURAL EQUIPMENT according to claims 1 and 2, and further characterized by said nozzle holder (10) being fixed to the wet bar (7) of the spraying vehicle through the support (2) of the sensor device (1) by fastening elements (9) ).

4. DEVICE WITH A VISION SYSTEM AND

IDENTIFICATION INSTALLED IN AGRICULTURAL EQUIPMENT according to claim 1, and further characterized by comprising at least one blue filter (1.4) arranged between the multispectral camera (1 .3) and the box (1 .1), being fixed by pressure.

5. DEVICE WITH A VISION SYSTEM AND

IDENTIFICATION INSTALLED IN AGRICULTURAL EQUIPMENT according to claim 1, and further characterized by comprising below the multispectral camera (1 .3) and installed on the PCB board (1 .2), blue LEDs (1 .5) and Infrared LEDs (1 .6).

6. DEVICE FITTED WITH A VISION AND IDENTIFICATION SYSTEM INSTALLED IN AGRICULTURAL EQUIPMENT as claimed in claim 1, and further characterized by the fact that in the lower part of the box (1.1) passers (1.8) are installed for the passage of cables that go to the connector ( 6) and a power cable, while a lid (1.9) for closing the box (1.1) is mounted on the rear.

7. DEVICE WITH A VISION SYSTEM AND

IDENTIFICATION INSTALLED IN AGRICULTURAL EQUIPMENT according to claim 1, and further characterized by said sensor device (1) being energized via power cable (13) and power connector (14).

8. DEVICE WITH A VISION SYSTEM AND

IDENTIFICATION INSTALLED IN AGRICULTURAL EQUIPMENT according to claim 1, and further characterized by the fixation of said sensor device (1) occurs in the support (2) that is fixed to the dry bar (3) of the spraying vehicle by at least one plate (4).

9. DEVICE WITH A VISION SYSTEM AND

IDENTIFICATION INSTALLED IN AGRICULTURAL EQUIPMENT, according to claims 1 and 2, characterized by a sensor (1) communicating, via the power connector (14), with a central unit comprising a commercial human-machine interface adapted in the spray vehicle cabin.

10. DEVICE WITH A VISION SYSTEM AND

IDENTIFICATION INSTALLED IN AGRICULTURAL EQUIPMENT, according to claim 1, and further characterized by said multispectral camera (1 .3) capturing an image and the electromagnetic waves reflected by the plants, by the waves that are emitted by the infrared LEDs (1 .6) and blue LEDs (1.5), through the filter (1.4), which consists of at least one film of translucent blue material.

11. DEVICE WITH A VISION SYSTEM AND

IDENTIFICATION INSTALLED IN AGRICULTURAL EQUIPMENT, according to claims 1 and 10, characterized by the multispectral camera (1 .3) capture wavelengths in the 450 to 495 nm range and the infrared range.

12. DEVICE WITH A VISION SYSTEM AND

IDENTIFICATION INSTALLED IN AGRICULTURAL EQUIPMENT, according to claims 1 and 10, characterized by the electromagnetic waves being processed by the microprocessor located on the PCB board (1 .2) and the system generating a vegetative index that can vary from 0 to 1.

13. DEVICE WITH A VISION SYSTEM AND

IDENTIFICATION INSTALLED IN AGRICULTURAL EQUIPMENT, according to claims 1 and 2, characterized by performing the image processing with the electromagnetic waves next to the microprocessor of the PCB board (1 .2) controlling the valve (5) of the sprayer; that the application of input to the plants is controlled via the processing of the image captured by the multispectral camera (1 .3) and processed on the PCB board (1 .2).

14. DEVICE WITH A VISION SYSTEM AND

IDENTIFICATION INSTALLED IN AGRICULTURAL EQUIPMENT, according to claims 1 and 2, characterized by comprising a database with images classified according to the characteristics of each pest, plant and weeds to train neural networks located in the microprocessor.

15. DEVICE WITH A VISION SYSTEM AND

IDENTIFICATION INSTALLED IN AGRICULTURAL EQUIPMENT, according to claims 1 and 2, characterized in that the herbicide flow is controlled by at least one biomass index and at least one nitrogen index acquired via image processing obtained by the multispectral camera (1 .3 ); in which the biomass index is calculated by the green mass of the image, being obtained via counting of white pixels at the moment of image processing; and the nitrogen index is obtained by the relationship between the calculated vegetative index and the concentration of leaf nitrogen.