BR102022013930A2 - METHOD AND SYSTEM FOR MONITORING SUGAR CANE HARVEST QUALITY - Google Patents

Abstract

method and system for monitoring sugarcane harvest quality. The present invention refers to a method and system for monitoring the quality of sugarcane harvesting, preferably used in harvesting machines, but not limited to this type of application. said method and system monitors in real time the material collected from images and machine learning techniques, which identifies parameters related to the quality of the monitored material, such as identifying sugarcane ratoons in a sample, making such information available for decision making for harvester calibration and future replanting.

Description

Field of Invention

[0001] The present invention refers to a method and system for monitoring the quality of sugarcane harvesting, preferably used in harvesting machines, but not limited to this type of application.

[0002] Said method and system monitors in real time the material collected from images and machine learning techniques, identifying parameters related to the quality of the monitored material and making such information available for decision making.

Fundamentals of Invention

[0003] Sugarcane harvesting has been evolving driven by the need to reduce environmental impact and increase productivity. The traditional system of manual harvesting of whole sugar cane has been replaced by the system of mechanized harvesting of chopped sugar cane, where the great advantage is that there is no need to previously burn the sugar cane field.

[0004] The main restrictions of the mechanized harvesting system are related to the need to efficiently harvest chaff (straw remains after harvest) and the difficulty of working on uneven and sloping terrain, making it difficult to maintain quality. at an attractive cost.

[0005] Despite the increased operational improvement of sugarcane harvesters and the many automation technologies presented during recent years, there is still no system/technology capable of measuring the quality of the material being harvested depending on the area in real time.

[0006] Sugarcane producers depend entirely on agricultural and industrial technicians to determine the quality of the harvested material and the yield of a given space. All these processes and analyzes require time and experience.

[0007] An important factor considered to determine the performance of combines in the field during harvesting is the amount of losses that the machines leave along the way. This factor is calculated by making an estimate in relation to a certain area, where for each defined area the chaff, the material that was no longer harvested and that which was harvested by mistake is separated - called ratoons (sugar cane base). with the roots). This material is separated into classes and weighed, thus calculating the average loss of that space.

[0008] For the quality of the material actually harvested, the assessment is made through the appearance of the sugar cane flowing through the harvester belt. The presence of brown or dark billets (pieces of sugar cane), poorly cut (appearing macerated), or the occurrence of straw and leaves, roots, weeds and ratoons are visual indicators that the harvester is not performing well. performance. It is not possible to effectively quantify these parameters in the field.

[0009] After cutting the sugar cane at the base, chopping it into pieces and cleaning off as much dirt as possible, these billets go into transport boxes (fields) towed by tractors or trucks. These vehicles have an inclined mechanism to transfer the harvested material to transport trucks that take the material to the plants.

[0010] In plants, the truck load is weighed and the value and origin are recorded manually or digitally. During weighing, samples of the collected material are collected and taken to a laboratory for evaluation. Among the parameters evaluated are the sucrose content, which is the kilogram of sugar per ton of sugar cane, and the percentage of external vegetable and mineral material, which are the most relevant for suppliers and harvesting employees. The mills only pay for the sugar potential of the material harvested and shipped, all these parameters take several hours to obtain, therefore only a certain amount of samples from the trucks are collected and processed every day.

[0011] There are inventions that propose monitoring the material that passes through the sugarcane harvester conveyor belt. Patent document US2015/124054 discloses a monitoring system based on the use of a stereoscopic (3D) camera to determine the volume of material that is passing through the conveyor, identifying the percentage of desired and unwanted material, generating through GPS and speed correction a yield map. Furthermore, based on this desired material flow, the method proposes to act directly on calibrating the height of the cutting blades until it reaches an ideal height. Such a solution has a high implementation cost and does not present the desired efficiency, since it requires great accuracy to identify the flow of unwanted material harvested and correlation with the cutting height of the harvester blades, requiring refined control and processing capacity.

[0012] Another example is that revealed in patent document BRPI0502658, which describes the use of load sensors to determine the weight of harvested materials and defines a productivity map for the harvest region. This solution is low cost, but it does not determine the quality of the material being harvested and does not help with real-time operations.

[0013] In this way, the present invention aims to surpass previous solutions both in terms of effectiveness and efficiency in relation to costs.

Objectives of the Invention

[0014] Thus, the main objective of the present invention is to reveal a method and system for monitoring the quality of sugarcane harvesting, preferably used in harvesting machines, but not limited to this type of application.

[0015] Additionally, it is the objective of the present invention to provide a method and system that monitors material collected from images in real time, identifying parameters related to the quality of the monitored material and making such information available for decision making.

[0016] Furthermore, the present invention aims to reveal a method for quantifying parameters in real time related to the harvesting of chopped sugar cane, aiming to mitigate the possibility of losing harvested material due to the amount of impurities in the samples, in addition to indicate such information for future replanting in areas that remain empty.

[0017] Furthermore, it is the objective of the present invention to present a system for monitoring the quality of sugarcane harvesting, preferably used in chopped sugarcane harvesting machines, which provides measurement quality and high reliability to identify harvested material using machine learning methods, indicating in real time impurities that are being acquired due to misdirection of the harvester machine. Summary of the Invention

[0018] All of the above-mentioned objectives are achieved through the method for monitoring sugarcane harvest quality which comprises the steps of: obtaining at least one image through an image acquisition unit from a higher angle close to or equal to the normal of the plane of a transport unit, process the image obtained by at least one processing unit, identify in at least one image an area evaluated by means of machine learning using a convolutional neural network, localize in the evaluated area brass knuckles through machine learning using a convolutional neural network, quantify the brass knuckles found in the assessed area and make the information available.

[0019] In accordance with the fundamental premises of the invention in question, the method for monitoring sugarcane harvest quality further comprises the fact that in the step of identifying in at least one image an area evaluated through learning machine using a convolutional neural network, the regression technique is used for the convolutional neural network, with Darknet19 architecture.

[0020] Additionally, a method for monitoring sugarcane harvest quality is provided which comprises the fact that the evaluated area corresponds to the step of the transport unit.

[0021] Furthermore, in the present invention a method is proposed which comprises the fact that in the step of locating brass knuckles in the area evaluated by means of machine learning using a convolutional neural network, the classification technique for the convolutional neural network is used, with Darknet19 architecture.

[0022] Furthermore, according to the present invention, the method for monitoring sugarcane harvest quality further comprises the fact that there is an additional step of generating a map of ratoons removed based on the information made available and in location data via a location unit.

[0023] Additionally, in the present invention, the method for monitoring sugarcane harvest quality comprises the fact that in the step of locating ratoons in the area evaluated by means of machine learning using a convolutional neural network, additionally it is located in the area assessed billets through machine learning using a convolutional neural network.

[0024] Also, according to the present invention, a system for monitoring sugarcane harvest quality is presented, comprising: at least one image acquisition unit positioned perpendicular to a transport unit, at least a processing unit and at least one communication interface.

[0025] Additionally, the system for monitoring sugarcane harvest quality further comprises the fact that the image acquisition unit comprises a two-dimensional photographic camera.

[0026] Furthermore, the system for monitoring the quality of sugarcane harvesting also comprises a lighting unit.

[0027] Finally, the present invention includes a system for monitoring the quality of sugarcane harvesting further comprising a location unit.

Brief Description of Figures

[0028] The preferred embodiment of the invention in question is described in detail based on the figures listed, which:

[0029] Figure 1 illustrates an image obtained from a transport unit.

[0030] Figure 2 illustrates the segmentation of the area of interest from an image of a transport unit.

[0031] Figure 3 illustrates the identification of sugarcane ratoons in an image of a transport unit.

[0032] Figure 4 illustrates a sugarcane harvest quality mapping.

[0033] Figure 5 illustrates the identification of sugar cane billets in an image of a transport unit.

[0034] Figure 6 illustrates a system for monitoring the quality of sugar cane harvesting.

[0035] Figure 7 illustrates an example of application of a system for monitoring the quality of sugarcane harvesting in a harvester.

Detailed Description of the Invention

[0036] All of the above-mentioned objectives are achieved through the method for monitoring sugarcane harvest quality comprised of the steps of: obtaining at least one image 1a through an image acquisition unit 1 from an angle upper close to or equal to the normal of the plane of a transport unit 2, process the image 1a obtained by at least one processing unit 3, identify in at least one image 1a an evaluated area 1b by means of machine learning using a neural network convolutional, locate 1c brass knuckles in the evaluated area 1b through machine learning using a convolutional neural network, quantify the 1c knuckledusters found in the evaluated area 1b and make the information available.

[0037] Thus, by obtaining at least one image 1a by means of an image acquisition unit 1 from a superior angle close to or equal to the normal of the plane of a transport unit 2 and processing the image 1a obtained by at least one unit of processing 3, image 1a uniformly provides what is being transported by the transport unit 2, that is, in a way that it is possible to visualize all the matter being transported, as illustrated in figure 1, and preferably with homogeneous lighting, whether in a controlled manner with artificial light in a closed environment or in an uncontrolled manner in a well-lit environment. The transport unit 2 is preferably a step elevator on a chopped sugar cane harvester, but it can also be any type of conveyor belt or means of transporting chopped sugar cane. The image acquisition unit 1 comprises a “simple” photographic camera (that takes photos in two dimensions) that makes the images available digitally in a “raw” way (or better known as “RAW”), but also any device that is capable of to capture images, such as video cameras, camcorders, etc. The processing unit 3 is responsible for processing the acquired image and converted into digital signals, normally composed of at least one processor and at least one memory unit capable of executing previously programmed routines, and may be directly associated with the image acquisition unit 1, be embedded in it, either through physical communication protocols (serial, CAN, ethernet, etc.) or even through “wireless” means (wifi, bluetooth, LoRa, etc.), thus being able to alternatively carry out the processing in a local or external network (commonly called “cloud” processing).

[0038] By identifying in at least one image 1a an evaluated area 1b through machine learning using a convolutional neural network, the method isolates the area of interest and segmentation for quality assessment. Machine learning methods (or “machine learning”) are a form of artificial intelligence that uses past data to predict future data, where a convolutional neural network is a “deep” machine learning algorithm (or “Deep Learning” ) that captures an input image and assigns importance such as weights and biases to various aspects and objects in the image, thus being able to differentiate one from the other. Its learning consists of pointing out the points you want to identify in a database of images, so the larger the database, the greater the accuracy of artificial intelligence in identifying.

[0039] In a chopped sugar cane harvester, area 1b of an image 1a corresponds to a step of an elevator by transport unit 2, which may preferably be in a transport unit 2 the step of a elevator, or any type of segmentation on a conveyor belt or means of transporting chopped sugar cane, which is capable of segmenting the flow of what is being harvested and identifying units correlating with quality parameters. Figure 2 illustrates the evaluated area 1b in an image 1a corresponding to the step of an elevator that contains chopped sugar cane in a transport unit 2. Thus, the method identifies the movement of the transport unit 2 without the need for sensors additional positioning or speed, simplifying the installation and application of the system, in addition to reducing costs.

[0040] From the assessed area 1b, the method proposes to locate brass knuckles 1c through machine learning using a convolutional neural network and quantify the brass knuckles 1c found in the assessed area 1b. As illustrated in figure 3, using the machine learning technique using a convolutional neural network, it is identified among the chopped sugar cane which was harvested together with the root, called “ratoons”. When the cutting blade is incorrectly adjusted, the harvester ends up removing the sugarcane root which, in addition to bringing dirt and dirt into the collected material, ends up leaving an empty space for the new “regrowth” cycle. requiring new planting at the site. Thus, by quantifying the ratoons 1c found in the evaluated area 1b, the method makes it possible to measure the quality of the harvesting operation in real time, since it is possible to estimate the amount of dirt that is going into the sample and also the amount that will be needed for the replanting.

[0041] By making the information available, the method allows the machine's operation to be adjusted in real time in order to prevent more ratoons from being collected and thus reduce the contagion of the sample. Real-time information also makes it possible to identify where the ratoons were removed so that locations can be identified for replanting.

[0042] Additionally, the method for monitoring sugarcane harvest quality comprises the fact that in the step of identifying in at least one image 1a an evaluated area 1b through machine learning using a convolutional neural network, The regression technique is used for the convolutional neural network, with “Darknet19” architecture. Regression techniques are based on basic principles of physics that help predict the future based on current data, as well as help find the correlation between two variables to define the cause and effect relationship. The “Darknet19” architecture is a convolutional neural network for “deep” machine learning (“deep learning”) composed of 19 convolution layers (linear digital processing operations) and 5 “pooling” layers (data grouping). It is an architecture built especially to deal with object detection problems and presents excellent image classification results, reaching 8.8% error in ILSVRC (“ImageNet Large Scale Visual Recognition Challenge” classification). Alternatively, other “deep” machine learning methods that have similar accuracy can also be applied to the method.

[0043] Furthermore, the method for monitoring the quality of sugarcane harvesting comprises the fact that the evaluated area 1b corresponds to the step of the transport unit 2, preferably an elevator or any type of segmentation on a conveyor belt. transport or means of transporting chopped sugar cane, which is capable of segmenting the flow of what is being harvested and identifying units correlating with quality parameters, whether due to good harvested material, poorly harvested material or impurities.

[0044] Furthermore, the method for monitoring the quality of sugarcane harvesting also comprises the fact that in the step of locating 1b ratoons 1c in the evaluated area through machine learning using a convolutional neural network, the technique classification system for the convolutional neural network, with Darknet19 architecture, thus identifying the amount of impurities that are going into the sample with great accuracy. Alternatively, other “deep” machine learning methods that have similar accuracy can also be applied to the method.

[0045] The method for monitoring sugarcane harvest quality further comprises the fact that there is an additional step of generating a map of ratoons removed based on the information made available and location data through a unit location 6. As illustrated in figure 4, the map of removed ratoons identifies on the land where the harvest was carried out where new sugar cane should be planted, based on information on the number of identified ratoons correlated with location data obtained by a location unit 6, which can preferably be a GPS (global position system) identification unit, but alternatively the unit can use technologies from other techniques such as triangulation via cell phone signal (3G, 4G, etc.), use of maps correlated with the movement of accelerometers or speed, or any other type that can identify within a field the positioning of ratoons harvested by mistake. This optimizes the land, as it helps technicians replant the removed sugar cane, reducing the need for more precise visual inspection to identify locations.

[0046] Furthermore, the method for monitoring the quality of the sugarcane harvest is understood by the fact that in the step of locating 1b ratoons 1c in the evaluated area through machine learning using a convolutional neural network, it is additionally located in the evaluated area 1b, “billets” 1d through machine learning using a convolutional neural network. The “billets” are chopped sugar cane that was harvested correctly, that is, they are being harvested free of dirt and unwanted parts, such as ratoon roots. By identifying and counting the “billets”, it is possible to correlate the effectiveness and quality of that batch with harvesting information, whether by area harvested or by truck harvested.

[0047] Additionally, a system for monitoring the quality of sugarcane harvesting is proposed, which comprises the fact of executing the method for monitoring the quality of sugarcane harvesting and comprising: at least one sugarcane acquisition unit images 1 positioned at a superior angle close to or equal to the normal of the plane of a transport unit 2, at least one processing unit 3 and at least one communication interface 4. The image acquisition unit 1 comprises a photographic camera “ simple” (that takes photos in two dimensions) that makes images available digitally in a “raw” (“RAW”) way, but also any device that is capable of capturing images, such as “video cameras”, camcorders, etc. The transport unit 2 is preferably a step elevator on a chopped sugar cane harvester, but it can also be any type of conveyor belt or means of transporting chopped sugar cane. The processing unit 3 is responsible for processing the image acquired and converted into digital signals, normally composed of at least one processor and at least one memory unit capable of executing previously programmed routines, and may be directly associated with the image acquisition unit 1 , whether embedded in it, or through communication protocols via physical means (serial, CAN, ethernet, etc.) or even through “wireless” means (wifi, bluetooth, LoRa, etc.), thus being able to alternatively carry out processing in a local or external network (commonly called “cloud” processing).

[0048] Communication interface 4 acts as the system's information provider, making the processed information available through a human machine interface (HMI), which informs the operator of the number of ratoons being harvested in real time (or alternatively billets), thus being able to correct the height at which the harvest is being carried out. Furthermore, the communication interface 4 can act as a “gateway” device (interface with other devices), making this information available to another interface device with the operator, such as a tablet or smartphone, and can communicate in a wired manner (via CAN protocols). , Serial, Isobus, etc.) or wireless (bluetooth, wifi, LoRa, etc.).

[0049] Furthermore, the system for monitoring sugarcane harvest quality comprises the fact that the image acquisition unit 1 comprises a two-dimensional photographic camera (common 2D acquisition cameras), The unit image acquisition system 1 comprises a “simple” photographic camera that makes images available digitally in a “raw” way, but also any device that is capable of capturing images, such as “video cameras”, camcorders, etc.

[0050] Furthermore, the system for monitoring sugarcane harvest quality also comprises a lighting unit 5. In this way, the lighting unit 5 allows the image acquisition unit 1 to be positioned at any point of a superior angle of a transport unit 2, even if it is in a closed place where there is no external light. Furthermore, the lighting unit 5 provides the operation of a chopped sugar cane harvester at night. The lighting unit 5 can be any type of light source, such as LED rings, “spots”, etc. In a preferred execution environment, a closed lighting area monitoring transport unit 2, controlled homogeneous lighting provides better system acuity, ensuring that the method performs detection without interference due to light instability. In a harvester, the upper part of the elevator can be used, with the lower part and sides blocking the light, with the upper part closed with an apparatus forming a box (made of stainless steel or any other material) that provides a controlled lighting environment. , receiving the installation of the lighting unit 5 and the image acquisition unit 1 internally - blocking all unwanted lighting and having controlled homogeneous lighting. Furthermore, other areas of the harvester can be adapted for the same purpose.

[0051] Additionally, the system for monitoring sugarcane harvest quality also comprises a location unit 6. Preferably the location unit 6 acts as a GPS (global position system) unit that identifies the geolocation of the system and the sugarcane harvester assembly, but alternatively the unit can use technologies from other techniques such as triangulation via cell phone signal (3G, 4G, etc.), use of maps correlated with accelerometer movement or speed, or any another type of device that can identify within a field the positioning of ratoons harvested due to poor calibration.

[0052] Figure 6 illustrates a complete sugarcane harvest quality monitoring system, with an image acquisition unit 1 associated with a processing unit 3 and a communication unit 4 - still associated with the unit lighting unit 5 and location unit 6 - arranged at an angle superior to the transport unit 2, through which the chopped sugar cane that you want to monitor the quality of is passing. Figure 7 illustrates the system arranged in a chopped sugar cane harvester with an image acquisition unit 1 associated with a processing unit 3 and a communication unit 4, monitoring the passage of the harvest in a transport unit. two.

[0053] It is important to emphasize that the above description has the sole purpose of describing in an exemplary way the particular embodiment of the invention in question. Therefore, it becomes clear that modifications, variations and constructive combinations of elements that perform the same function in substantially the same way to achieve the same results, remain within the scope of protection delimited by the attached claims.

Claims (10)

1. Method for monitoring sugarcane harvest quality CHARACTERIZED by the fact that it comprises the steps of: i. obtaining at least one image (1a) by means of an image acquisition unit (1) from a top angle close to or equal to the normal of a transport unit (2); ii. processing the image (1a) obtained by at least one processing unit (3); iii. identify in at least one image (1a) an evaluated area (1b) by means of machine learning using a convolutional neural network; iv. locate brass knuckles (1c) in the evaluated area (1b) through machine learning using a convolutional neural network; v. quantify the ratoons (1c) found in the assessed area (1b); saw. make the information available. 2. Method for monitoring sugarcane harvest quality, according to claim 1, CHARACTERIZED by the fact that in step iii. The regression technique is used for the convolutional neural network, with Darknet19 architecture. 3. Method for monitoring sugarcane harvest quality, according to claims 1 and 2, CHARACTERIZED by the fact that the evaluated area (1b) corresponds to the step of the transport unit (2). 4. Method for monitoring sugarcane harvest quality, according to claim 1, CHARACTERIZED by the fact that in step iv. The classification technique for the convolutional neural network is used, with Darknet19 architecture. 5. Method for monitoring sugarcane harvest quality, according to claim 1, CHARACTERIZED by the fact that there is also an additional step of generating a map of ratoons removed based on the information made available in step vi. and in location data via a location unit (6). 6. Method for monitoring sugarcane harvest quality, according to claim 1, CHARACTERIZED by the fact that in step iv. additionally it is located in the evaluated area (1b) billets (1d) through machine learning using a convolutional neural network. 7. System for monitoring sugarcane harvest quality, CHARACTERIZED by the fact that it executes the method for monitoring sugarcane harvest quality and comprises: at least one image acquisition unit (1) positioned at an angle greater than a transport unit (2); at least one processing unit (3); at least one communication interface (4). 8. System for monitoring sugarcane harvest quality, according to claim 7, CHARACTERIZED by the fact that the image acquisition unit (1) comprises a two-dimensional photographic camera. 9. System for monitoring sugarcane harvest quality, according to claim 6, CHARACTERIZED by the fact that it also comprises a lighting unit (5). 10. System for monitoring sugarcane harvest quality, according to claim 6, CHARACTERIZED by the fact that it also comprises a location unit (6).