WO2020110027A1 - An improved method for controlling a soil working means based on image processing and related system - Google Patents

An improved method for controlling a soil working means based on image processing and related system Download PDF

Info

Publication number
WO2020110027A1
WO2020110027A1 PCT/IB2019/060228 IB2019060228W WO2020110027A1 WO 2020110027 A1 WO2020110027 A1 WO 2020110027A1 IB 2019060228 W IB2019060228 W IB 2019060228W WO 2020110027 A1 WO2020110027 A1 WO 2020110027A1
Authority
WO
WIPO (PCT)
Prior art keywords
soil
working means
parameter
processing unit
soil working
Prior art date
Application number
PCT/IB2019/060228
Other languages
French (fr)
Inventor
Silvio REVELLI
Original Assignee
Volta Robots S.R.L.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Volta Robots S.R.L. filed Critical Volta Robots S.R.L.
Priority to EP19831890.9A priority Critical patent/EP3886564A1/en
Publication of WO2020110027A1 publication Critical patent/WO2020110027A1/en

Links

Classifications

    • AHUMAN NECESSITIES
    • A01AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01FPROCESSING OF HARVESTED PRODUCE; HAY OR STRAW PRESSES; DEVICES FOR STORING AGRICULTURAL OR HORTICULTURAL PRODUCE
    • A01F15/00Baling presses for straw, hay or the like
    • A01F15/08Details
    • A01F15/0825Regulating or controlling density or shape of the bale
    • A01F15/0833Regulating or controlling density or shape of the bale for round balers
    • AHUMAN NECESSITIES
    • A01AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01BSOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL
    • A01B69/00Steering of agricultural machines or implements; Guiding agricultural machines or implements on a desired track
    • A01B69/001Steering by means of optical assistance, e.g. television cameras
    • AHUMAN NECESSITIES
    • A01AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01BSOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL
    • A01B49/00Combined machines
    • A01B49/04Combinations of soil-working tools with non-soil-working tools, e.g. planting tools

Definitions

  • the present invention relates in general to the field of controlling soil working machines, such as for example, lawnmower machines or lawnmower robots configured to operate autonomously.
  • the invention in particular relates to a method for controlling a soil working means based on image processing.
  • the object of the invention is also a system which includes a soil working means which implements the aforesaid method.
  • Lawnmower equipment or lawnmower robots configured to operate autonomously, that is without being driven by an operator, are available on the market today for residential uses.
  • Control methods which allow the movement of an autonomous yard lawnmower robot to be controlled, the operating scope thereof to be confined within the related work area and it to be directed towards the areas of the yard to be cut.
  • the various control forms of the lawnmower robot in particular provide processing images taken using neural networks. Such processing operations may have various levels of complexity and therefore, require different levels of computational resources. Indeed, the processing of simpler images is executed more quickly while the more complex processing is executed during a longer period of time, on-board computational capacity of the soil working means being equal.
  • the synthetic descriptor includes a first parameter representative of a reliability of the description of the soil by the synthetic descriptor or a second parameter representative of the completeness of information contained in the synthetic descriptor or a third parameter representative of the reciprocal of the computing time used by said processing step to obtain the synthetic descriptor;
  • the method comprises the step, by the electronic processing unit, of generating a control signal of said locomotion member to reduce the speed of the soil working means.
  • the method comprises a step of processing, by the electronic processing unit, the at least one digital image acquired through a second processing process with which a second computational complexity being greater than said first computational complexity, is associated.
  • the method of the invention advantageously provides reducing the forward speed of the working means to give the latter "more time to think". Secondarily to such a reduction of forward speed, the method of the invention also provides using a processing process having a greater computational complexity, that is a cognitive process which is more adequate for processing the aforesaid images.
  • the invention relates to a method for controlling a soil working means based on image processing executed with a neural network, in particular of the convolutional type.
  • the method comprises a step of using a FIFO (Fist In First Out) type buffer for determining a value of parameter associated with two or more classifications to be compared with a respective threshold.
  • a FIFO ist In First Out
  • the object of the present invention is also a system which implements the aforesaid method according to claim 8.
  • FIG. 1 shows a flow diagram of a more general embodiment of the control method of the invention which, in response to an insufficient level of reliability or of information content or rapidity of obtaining a synthetic descriptor of the soil provided by the image processing, responds by reducing a forward speed of the soil working means;
  • figure 2 shows a block diagram of a soil working system based on image processing which implements the method in figure 1;
  • figure 3 shows an alternative embodiment of a processing step carried out by the method in figure 1;
  • figure 4 shows an example of a buffer of synthetic descriptors of the soil related to images acquired in sequence .
  • a method for controlling a soil working means is indicated with numeral 100.
  • the invention in particular relates to a method for controlling a soil working means based on image processing. Later, such a working means control method 100 is also simply indicated as control method.
  • a system comprising the aforesaid soil working means is indicated as a whole with numeral 1000.
  • Such a soil working means is for example, chosen from the group consisting of: lawnmower machine or lawnmower robot, reaping machine, plow, grain cutting harvesting machine, vegetable cutting-harvesting machine, hay cutting-harvesting machine, snow plough machine and any similar machine.
  • the working means of system 1000 includes a suitable locomotion member 201 for moving the working means at a predetermined speed V, and a working member 202.
  • the above-mentioned locomotion member 201 includes, by way of example, a propulsion unit fed by a respective fuel, a steering unit, a stability control and safety control unit and a power supply unit (e.g. battery) . Moreover, such a locomotion member 201 may also include suitable input/output communication interfaces for receiving control signals and returning signals indicative of the movement.
  • the working member 202 includes a suitable working tool, which can be moved through a respective engine, and further input/output communication interfaces for receiving control signals and returning signals indicative of the working carried out.
  • system 1000 comprises digital image acquisition means 203 installed on the working means for acquiring at least one digital image of the soil.
  • image acquisition means materialize for example, in one or more cameras 203.
  • Such cameras 203 may acquire a more or less broad portion of the surrounding soil according to the number thereof, the position thereof and to the geometry of the lens which characterizes the field of view thereof.
  • Such cameras 203 and similar image acquisition means may receive images in scales of greys, or preferentially in the visible spectrum with color coding (e.g. RGB) .
  • Such image acquisition means may be configured to operate in the infrared and ultraviolet spectrum or complete the optical information with a channel dedicated to depth (e.g. RGB-D) .
  • System 1000 further comprises an electronic processing unit 204 connected to the digital image acquisition means 203 and to the above-mentioned locomotion member 201 and working member 202 of the soil working means .
  • Such an electronic processing unit 204 comprises at least one processor 205 and a storage block 206, 207 associated with the processor for storing instructions.
  • a storage block 206, 207 in particular is connected to processor 205 through a data communication line or bus 211 (e.g. PCI) and consists of a volatile type service memory 206 (e.g. of SDRAM type) and of a non- volatile type system memory 207 (e.g. SSD type) .
  • a volatile type service memory 206 e.g. of SDRAM type
  • non- volatile type system memory 207 e.g. SSD type
  • an electronic processing unit 204 comprises input/output interface means 208 connected to the at least one processor 205 and to the storage block 206, 207 through the communication bus 211 to allow an operator close to system 1000 to directly interact with the processing unit itself.
  • input/output interface means 208 comprise for example, a touch-control display operating as data input/output interface and information display.
  • the electronic processing unit 204 is associated with a wired or wireless data communication interface 209 configured to connect such a processing unit 204 to a data communication network 210, e.g. the Internet, to allow an operator to remotely interact with the processing unit 204 or to receive updates.
  • a data communication network 210 e.g. the Internet
  • the electronic processing unit 204 of system 1000 is prepared to execute the codes of an application program which implements method 100 of the present invention.
  • processor 205 is configured to upload, in the storage block 206, 207, and execute the codes of the application program which implements method 100 of the present invention.
  • Method 100 comprises symbolic start STR and end ED steps .
  • control method 100 comprises a first step 101 of acquiring at least one digital image of the soil by means of the digital image acquisition means 203, that is the cameras, installed on the working means .
  • Method 100 then comprises a processing step 102, by an electronic processing unit 204, of the at least one digital image acquired.
  • a processing step is carried out through a first processing process with which a first computational complexity is associated.
  • the method 100 comprises a step 103 of obtaining, by the electronic processing unit, at least one synthetic descriptor DS of the soil based on such a processing .
  • Such a synthetic descriptor DS includes a first parameter PI representative of a reliability of the description of the soil by the synthetic descriptor DS or a second parameter P2 representative of the completeness of information contained in the synthetic descriptor DS or a third parameter P3 representative of the reciprocal of the computing time used by said processing step to obtain the synthetic descriptor.
  • the synthetic descriptor DS may comprise the first parameter PI alone, or the second parameter P2 alone, or the third parameter P3 alone, or the first PI and the second P2 parameters alone, or the second parameter P2 and the third parameters P3 alone, or the first parameter PI and the third P3 parameters alone, or all three parameters.
  • Method 100 also comprises comparing 104 at least one of said above-mentioned first PI, second P2 and third P3 parameters representative of the synthetic descriptor DS with a respective preset threshold value kl, k2 , k3.
  • method 100 comprises the step 105 of generating, by the electronic processing unit 204, a control signal of said locomotion member 201 to reduce speed V of the soil working means.
  • the aforesaid processing step 102 comprises the further steps of:
  • the aforesaid first or second processing process comprise a neural network conveniently trained to obtain the synthetic descriptor of the soil.
  • Such a training of the neural network is carried out with machine-learning techniques, i.e. the modeling of the soil is not expressly coded by a human being, rather is automatically learned by the machine.
  • the advantage is that the system which implements the method has increased cognitive abilities and an increased level of generalization. This makes the system which implements the method of the invention sound under the multiple conditions of soils which have a significant variance aspect in relation to the factors of atmosphere, lighting, state of the soil and the presence of shadows and/or reflections.
  • such a neural network of image processing process comprises at least one convolutional layer and the aforesaid at least one convolution operation is carried out by means of a "feed-forward" execution of the neural network.
  • a convolutional neural network i.e. which implements a series of successive convolutions, has the advantage of internally building a hierarchical representation which may not necessarily be interpretable - of the features of the image; this allows circumstantial features to have a high level of abstraction and soundness.
  • each task is a sequence of instructions which may be sent to the locomotion member 201 and are affected by a plurality of inputs, among which the synthetic descriptor DS of the soil obtained by means of image processing .
  • the first parameter PI representative of the confidence or reliability of the synthetic descriptor DS of the soil or the second parameter P2 representative of the specificity required by the synthetic descriptor of the soil, that is the completeness of information of the descriptor, or the third parameter mentioned above, is initially set for each task.
  • the level of confidence of the synthetic descriptor DS in particular expresses a level of reliability of the classifier and varies from 0 to 1.
  • the specificity of the synthetic descriptor DS of the soil indicates the level of detail with which it is capable of detecting the soil itself according to a given criterion.
  • a highly generic descriptor DS of the soil may for example, describe only if a soil may be worked or has obstacles .
  • a more specific descriptor may get into detail about the individual obstacle, recognizing it separately among a plurality of classes of obstacles.
  • a specific descriptor DS may indicate the related direction of the previous working line of the soil. Or it may complete the information on the obstacle by also providing the approximate distance thereof.
  • soil image processing processes may be available in the memory of the soil working means, each with its own computational time and capable of providing synthetic descriptors of the soil with adequate information completeness or level of confidence.
  • the case at hand considers a first processing process having a first computational complexity and a second processing process having a second computational complexity being greater than the first complexity.
  • the control method 100 assigns a speed V of the soil working means which is adequate to the type of image processing and returns a synthetic descriptor DS of the soil, it also assigned to the specific task.
  • a speed V is the speed of the movements related to the specific task.
  • Such a speed is for example the forward, reverse, rotation speed on one axle; it may also be the speed of a combined motion of the above speeds; a particular case of speed is the null speed.
  • the method 100 provides a reduction of the absolute speed V of the soil working means.
  • the about- mention processing operations of method 100 of the invention may also be performed by a further electronic processing unit, different from the processing unit 204 provided on-board of the soil working means and positioned in a position which is distal from the latter, for example by a remote server with respect to the working means.
  • a further processing unit comprises a respective processor and a respective storage block associated with the processor.
  • such a further processing unit or remote server is configured to communicate with the soil working means through the wired or wireless data communication interface 209 which connects such a processing unit 204 to the data communication network 210, e.g. the Internet.
  • the aforesaid step 103 of obtaining the synthetic descriptor DS also comprises a step of saving each synthetic descriptor DS of the soil based on the processing in a buffer 400 having N elements.
  • buffer 400 of the FIFO type, comprises four elements.
  • Buffer 400 consists of four slots numbered #1, #2, #3,
  • Each element of buffer 400 includes a first, a second and a third field for saving the first PI, the second P2 and the third P3 parameters, respectively.
  • each new descriptor DS with the related parameters (PI, P2, P3) is inserted into slot #1 (first column) .
  • the preceding descriptor (with the related parameters) is moved into slot #2.
  • the descriptor of the latter in turn is moved into slot #3.
  • the descriptor first present in slot # 3 in turn is moved into slot #4.
  • the descriptor in the latter is removed from the buffer and eliminated.
  • the method comprises a step of selecting values of one of such first PI or second P2 or third P3 parameter for all N elements of buffer 400.
  • a step is provided for generating, from the N values of the parameter selected, a modified parameter representative of the N values of such a parameter .
  • the step of generating the modified parameter from the N values of the parameter selected is chosen from the group consisting of:
  • the aforesaid comparison step 104 of method 100 comprises a step of comparing said modified parameter with the respective threshold value associated with said first PI or second P2 or third P3 parameter.
  • the general method 100 of the invention advantageously allows taking advantage of such processing operations only when needed and with respect to a decrease in speed V of the soil working means.
  • the working means may therefore proceed more quickly.
  • a first image processing process available in the memory of the lawnmower robot is capable of returning, as synthetic descriptor of the soil, a binary type classification: a "GO" class, i.e. completely grassy soil and therefore crossable, and a "NOGO” class, i.e. not completely grassy soil or soil with at least one obstacle.
  • the robot advances with a given speed V and uses the first processing process to decide whether or not to continue.
  • the individual classifications of the soil returned are used by the processing unit 204 to decide whether or not to continue the motion or change direction.
  • Each descriptor DS arrives with an associated level of confidence. If the classifications are "GO” with a high level of confidence, then for example, the control logic controls the continuation of the motion. If the classifications are "NOGO" with a high level of confidence, then the lawnmower control logic controls the change in speed direction.
  • the classifications are uncertain, i.e. with a lower level of confidence than the specific threshold for the forward task (independently of "GO” and "NOGO"), then the specific method in figure 1 is applied, which provides a reduction of the speed, in particular the forward speed, of the lawnmower robot.
  • the classifications or synthetic descriptors DS may be saved in a buffer 400 with FIFO (First In First Out) logic which contains the outputs of the last N classifications.
  • FIFO First In First Out
  • a first image processing process available in the memory of the lawnmower robot which is computationally less burdensome and therefore is quicker in the execution, is capable of returning a binary type classification as synthetic descriptor of the soil: a "GO" class, i.e. completely grassy soil and therefore crossable, and a "NOGO” class, i.e. not completely grassy soil or soil with at least one obstacle.
  • a second image processing process available in the memory of the lawnmower robot which is computationally more burdensome and therefore is less quick in the execution, is capable of returning a binary type classification as synthetic descriptor of the soil: a "GO" class, i.e. completely grassy soil and therefore crossable, and a "NOGO” class, i.e. not completely grassy soil or soil with at least one obstacle.
  • the lawnmower robot advances with a given speed and uses the first processing process to decide whether or not to continue.
  • the individual classifications of the soil returned by the process are used by the processing unit to decide whether or not to continue the motion or change direction. Each classification arrives with an associated level of confidence.
  • the control logic controls the continuation of the motion.
  • the lawnmower control logic controls the change in speed direction.
  • the classifications are uncertain, i.e. with a lower level of confidence than the specific threshold for the forward task (independently of "GO” and "NOGO"), then the method in figure 3 is applied, which provides a reduction of the forward speed V of the lawnmower robot and the simultaneous passing to the second image processing process, which as mentioned, is slower but more accurate than the first process.
  • the level of confidence of the synthetic descriptor DS of the soil given by the second process is higher and the processing unit 204 can undertake the most suitable control actions with increased accuracy.
  • a first image processing process available in the memory of the lawnmower robot which is computationally less burdensome and therefore is quicker in the execution, is capable of returning a binary type classification as synthetic descriptor of the soil: a "GO" class, i.e. completely grassy soil and therefore crossable, and a "NOGO” class, i.e. not completely grassy soil or soil with at least one obstacle.
  • a second image processing process available in the memory of the lawnmower robot which is computationally more burdensome and therefore is less quick in the execution, is capable of distinguishing a particular feature of the soil, thus providing a synthetic descriptor of the soil which is more complete from an information viewpoint.
  • the lawnmower robot advances with a given speed and uses the first processing process to decide whether or not to continue.
  • the individual classifications of the soil returned by the process are used by processing unit to decide whether or not to continue the motion or undertake another action. Each classification arrives with an associated level of reliability.
  • the control logic controls the continuation of the motion.
  • the lawnmower control logic does not have enough information to decide which control to impart on the motion members; thus the method in figure 3 is applied, which provides a reduction of the forward speed V of the lawnmower robot and the simultaneous passing to the second image processing process, which as mentioned, is slower but more accurate than the first process .
  • the second processor provides an estimate of the distance between the obstacle identified and a front portion of the robot. It is apparent that the first process that obtains only the "GO” or “NOGO” crossability information is less burdensome from the computational viewpoint of the process which also estimates the distance from the obstacle. If for example, the processing processes are based on convolutional neural networks, it is known that the "detection" task uses more neurons than the "classification” task. Thus, so long as the first process detects a "GO" crossable soil, the soil working means may proceed at high speed V. When the first process detects a "NOGO", it may be useful to estimate the distance between the detected obstacle and accordingly the method in figure 3 is applied, i.e. the image is processed by means of said second computationally more burdensome process. Thus, the soil working means reduces speed V to allow the execution of the second process.
  • the second process is configured to classify the type of obstacle, distinguishing for example, if it is a false alarm or a real obstacle. For example during the operating step, a lawnmower robot could encounter one or more leaves, which are not a true obstacle to the cutting operations. It is apparent that the first process that obtains only the "GO” or "NOGO” crossability information is computationally less burdensome than the process which also estimates the type of obstacle by distinguishing if it is a leaf or something else. If for example, the processing processes are based on convolutional neural networks, it is known that the "several-class classification" task uses more neurons than the "two- class classification" task.
  • the soil working means may proceed at high speed V.
  • the first process detects a "NOGO”
  • it may be suitable to estimate the type of obstacle detected and accordingly the method in figure 3 is applied, i.e. the image is processed by means of said second computationally more burdensome process.
  • the soil working means reduces speed V to allow the execution of the second process.
  • a possible cause which results in the reduction of the forward speed V lies in the third parameter P3 (i.e. the inverse of the computation time) below a given threshold.
  • P3 i.e. the inverse of the computation time
  • a processing process employs more time than expected (and therefore processes less frames per second) usually lies in a concomitant process requested from the CPU.
  • Such a concomitant process may be unexpected, or voluntarily started by the control system. It may for example, be a more complex extended image processing to obtain other navigation data than the classification of the soil to be crossed.
  • a particular example of such a process which could occupy the CPU is the image processing which is capable of providing the characterization of the grass cutting line in the image; for example, it may characterize the straight line which on the image divides the portion of tall grass to be cut from the portion of already cut short grass, with two offset and inclination parameters.
  • control method 100 of a soil working means of the present invention and the related system 1000 have several advantages and achieve the predetermined objects.
  • control method 100 proposed allows the soil working operations to be speeded up to the greatest extent possible while simultaneously keeping high the accuracy and specificity of the control actions.
  • the solution proposed in particular uses burdensome classifications from a computational viewpoint when they are needed and light classifications when they are needed, thus being highly versatile with respect to the known solutions.
  • the slowdown is not due to a sensorial limit (e.g. the camera does not correctly acquire the image) , rather always to a computational limit in terms of non-adequate confidence, information completeness or rapidity of the synthetic descriptor DS of the soil.
  • the present invention consists in identifying the overcoming of a cognitive limit according to the task performed and speed V, and accordingly reduces speed V.
  • a soil working means like the one the object of the invention may decide to stop "to think” at any time.
  • the system is applied without limitation to lawnmower robots, but more generally to yard service robots, and even more generally, to soil working means.

Abstract

The invention relates to improvements to a method (100) for controlling a soil working means based on image processing. The soil working means comprises a locomotion member (201) for moving the working means at a predetermined speed (V). The method comprises the steps of: acquiring (101) at least one digital image of the soil by means of digital image acquisition means (203) installed on the working means; processing (102) the at least one digital image acquired through a first processing process with which a first computational complexity is associated; obtaining (103) at least one syntheticdescriptor (DS) of the soil based on such a processing; the synthetic descriptor includes a first parameter (P1) or a second parameter (P2) or a third parameter (P3); comparing (104) at least one of the first, second and third parameters with a respective preset threshold value (k1, k2, k3); if the at least one of the first, second and third parameters is lower than the respective preset threshold value, the method comprises the step (105) of generating a control signal of said locomotion member to reduce the speed of the soil working means.

Description

DESCRIPTION
AN IMPROVED METHOD FOR CONTROLLING A SOIL WORKING MEANS BASED ON IMAGE PROCESSING AND RELATED SYSTEM
BACKGROUND ART OF THE INVENTION
Field of application
The present invention relates in general to the field of controlling soil working machines, such as for example, lawnmower machines or lawnmower robots configured to operate autonomously. The invention in particular relates to a method for controlling a soil working means based on image processing. The object of the invention is also a system which includes a soil working means which implements the aforesaid method.
Prior art
Lawnmower equipment or lawnmower robots configured to operate autonomously, that is without being driven by an operator, are available on the market today for residential uses.
Control methods are known which allow the movement of an autonomous yard lawnmower robot to be controlled, the operating scope thereof to be confined within the related work area and it to be directed towards the areas of the yard to be cut. The various control forms of the lawnmower robot in particular provide processing images taken using neural networks. Such processing operations may have various levels of complexity and therefore, require different levels of computational resources. Indeed, the processing of simpler images is executed more quickly while the more complex processing is executed during a longer period of time, on-board computational capacity of the soil working means being equal.
On the other hand, a more complex processing of the images potentially is more accurate or provides the control system with greater information content concerning the state of the soil to be worked. Such greater information content allows specific control actions to be undertaken for each specific working circumstance .
However, the computational resources available on board a yard robot - especially a small one - are currently limited by a cost factor of the robot itself, which is to be as contained as possible, and by a factor of electric energy consumed by such resources since the batteries feeding the lawnmower robot have reduced autonomy .
In consideration of such limitations, the need today is particularly felt to create methods for controlling lawnmower robots which allow the soil working operations to be speeded to the greatest extent possible while simultaneously keeping high the accuracy and specificity of the control actions.
SUMMARY OF THE INVENTION
It is the object of the present invention to devise and make available a method and a related system for controlling a soil working means, which allows a simultaneous quick and accurate performance of the work, thus at least partially overcoming the above drawbacks in relation to known control methods.
Such an object is achieved by a method for controlling a soil working means according to claim 1.
According to a main embodiment of the method for controlling a soil working means based on image processing, the following is provided:
- acquiring at least one digital image of the soil by means of digital image acquisition means installed on the working means;
processing, by an electronic processing unit, the at least one digital image acquired through a first processing process with which a first computational complexity is associated;
obtaining, from the electronic processing unit, at least one synthetic descriptor of the soil based on such a processing operation, in which the synthetic descriptor includes a first parameter representative of a reliability of the description of the soil by the synthetic descriptor or a second parameter representative of the completeness of information contained in the synthetic descriptor or a third parameter representative of the reciprocal of the computing time used by said processing step to obtain the synthetic descriptor;
comparing at least one of such first, second and third parameters representative of the synthetic descriptor with a respective preset threshold value;
if the at least one of said first, second and third parameters is lower than the respective preset threshold value, the method comprises the step, by the electronic processing unit, of generating a control signal of said locomotion member to reduce the speed of the soil working means.
According to a further embodiment of the control method of the invention, the method comprises a step of processing, by the electronic processing unit, the at least one digital image acquired through a second processing process with which a second computational complexity being greater than said first computational complexity, is associated.
In other words, if the soil working means implements, by means of image processing, an excessively slow cognitive process with respect to the forward speed thereof, or an inaccurate or inadequate process with respect to such a speed, the method of the invention advantageously provides reducing the forward speed of the working means to give the latter "more time to think". Secondarily to such a reduction of forward speed, the method of the invention also provides using a processing process having a greater computational complexity, that is a cognitive process which is more adequate for processing the aforesaid images.
Thus, at least two separate image processing operations are used, one after the other.
In a further example embodiment, the invention relates to a method for controlling a soil working means based on image processing executed with a neural network, in particular of the convolutional type.
In a further example embodiment, the method comprises a step of using a FIFO (Fist In First Out) type buffer for determining a value of parameter associated with two or more classifications to be compared with a respective threshold.
The object of the present invention is also a system which implements the aforesaid method according to claim 8.
Preferred embodiments of such a control method are described in the dependent claims.
BRIEF DESCRIPTION OF THE DRAWINGS
Further features and advantages of the method for controlling the soil working means based on image processing and of the related system according to the invention will become apparent from the following description of preferred embodiments thereof, given only by way of non-limiting, indicative example, with reference to the accompanying drawings, in which:
- figure 1 shows a flow diagram of a more general embodiment of the control method of the invention which, in response to an insufficient level of reliability or of information content or rapidity of obtaining a synthetic descriptor of the soil provided by the image processing, responds by reducing a forward speed of the soil working means;
figure 2 shows a block diagram of a soil working system based on image processing which implements the method in figure 1;
- figure 3 shows an alternative embodiment of a processing step carried out by the method in figure 1;
figure 4 shows an example of a buffer of synthetic descriptors of the soil related to images acquired in sequence .
Equal or similar elements are indicated with the same numerals in the aforesaid drawings.
DETAILED DESCRIPTION
In reference to figure 1, a method for controlling a soil working means according to the present invention is indicated with numeral 100. The invention in particular relates to a method for controlling a soil working means based on image processing. Later, such a working means control method 100 is also simply indicated as control method.
In reference to figure 2, a system comprising the aforesaid soil working means is indicated as a whole with numeral 1000.
Such a soil working means is for example, chosen from the group consisting of: lawnmower machine or lawnmower robot, reaping machine, plow, grain cutting harvesting machine, vegetable cutting-harvesting machine, hay cutting-harvesting machine, snow plough machine and any similar machine.
Although reference is made later in the description to a soil working means which materializes in a lawnmower machine or lawnmower robot, the principles of the invention are applicable to other types of above- mentioned soil working means without substantial modifications .
The working means of system 1000 includes a suitable locomotion member 201 for moving the working means at a predetermined speed V, and a working member 202.
The above-mentioned locomotion member 201 includes, by way of example, a propulsion unit fed by a respective fuel, a steering unit, a stability control and safety control unit and a power supply unit (e.g. battery) . Moreover, such a locomotion member 201 may also include suitable input/output communication interfaces for receiving control signals and returning signals indicative of the movement.
The working member 202 includes a suitable working tool, which can be moved through a respective engine, and further input/output communication interfaces for receiving control signals and returning signals indicative of the working carried out.
Moreover, system 1000 comprises digital image acquisition means 203 installed on the working means for acquiring at least one digital image of the soil. Such image acquisition means materialize for example, in one or more cameras 203.
Such cameras 203 may acquire a more or less broad portion of the surrounding soil according to the number thereof, the position thereof and to the geometry of the lens which characterizes the field of view thereof. Such cameras 203 and similar image acquisition means may receive images in scales of greys, or preferentially in the visible spectrum with color coding (e.g. RGB) . Such image acquisition means may be configured to operate in the infrared and ultraviolet spectrum or complete the optical information with a channel dedicated to depth (e.g. RGB-D) .
System 1000 further comprises an electronic processing unit 204 connected to the digital image acquisition means 203 and to the above-mentioned locomotion member 201 and working member 202 of the soil working means .
Such an electronic processing unit 204 comprises at least one processor 205 and a storage block 206, 207 associated with the processor for storing instructions. Such a storage block 206, 207 in particular is connected to processor 205 through a data communication line or bus 211 (e.g. PCI) and consists of a volatile type service memory 206 (e.g. of SDRAM type) and of a non- volatile type system memory 207 (e.g. SSD type) .
In greater detail, such an electronic processing unit 204 comprises input/output interface means 208 connected to the at least one processor 205 and to the storage block 206, 207 through the communication bus 211 to allow an operator close to system 1000 to directly interact with the processing unit itself. Such input/output interface means 208 comprise for example, a touch-control display operating as data input/output interface and information display.
Moreover, the electronic processing unit 204 is associated with a wired or wireless data communication interface 209 configured to connect such a processing unit 204 to a data communication network 210, e.g. the Internet, to allow an operator to remotely interact with the processing unit 204 or to receive updates.
In reference to figure 1, described in greater detail below are the operating steps of method 100 for controlling a soil working means based on image processing, which is implemented through system 1000.
In an example embodiment, the electronic processing unit 204 of system 1000 is prepared to execute the codes of an application program which implements method 100 of the present invention.
In a particular example embodiment, processor 205 is configured to upload, in the storage block 206, 207, and execute the codes of the application program which implements method 100 of the present invention.
Method 100 comprises symbolic start STR and end ED steps .
In the more general example embodiment, the control method 100 (shown in figure 1) comprises a first step 101 of acquiring at least one digital image of the soil by means of the digital image acquisition means 203, that is the cameras, installed on the working means .
Method 100 then comprises a processing step 102, by an electronic processing unit 204, of the at least one digital image acquired. Such a processing step is carried out through a first processing process with which a first computational complexity is associated.
Moreover, the method 100 comprises a step 103 of obtaining, by the electronic processing unit, at least one synthetic descriptor DS of the soil based on such a processing .
Such a synthetic descriptor DS includes a first parameter PI representative of a reliability of the description of the soil by the synthetic descriptor DS or a second parameter P2 representative of the completeness of information contained in the synthetic descriptor DS or a third parameter P3 representative of the reciprocal of the computing time used by said processing step to obtain the synthetic descriptor.
In other words, the synthetic descriptor DS may comprise the first parameter PI alone, or the second parameter P2 alone, or the third parameter P3 alone, or the first PI and the second P2 parameters alone, or the second parameter P2 and the third parameters P3 alone, or the first parameter PI and the third P3 parameters alone, or all three parameters.
Method 100 also comprises comparing 104 at least one of said above-mentioned first PI, second P2 and third P3 parameters representative of the synthetic descriptor DS with a respective preset threshold value kl, k2 , k3.
If the at least one of such first PI and second P2 parameters is lower than the respective preset threshold value, method 100 comprises the step 105 of generating, by the electronic processing unit 204, a control signal of said locomotion member 201 to reduce speed V of the soil working means.
With reference to figure 3, in a further example embodiment, if the at least one of such first PI, second P2 and third P3 parameters is lower than the respective preset threshold value kl, k2, k3, the aforesaid processing step 102 comprises the further steps of:
- replacing 1021 the first processing process with a second processing process with which a second computational complexity being greater than such a first computational complexity, is associated;
- processing 1022, by the electronic processing unit 204, the at least one digital image acquired through the second processing process.
In an example embodiment of the invention, the aforesaid first or second processing process comprise a neural network conveniently trained to obtain the synthetic descriptor of the soil. Such a training of the neural network is carried out with machine-learning techniques, i.e. the modeling of the soil is not expressly coded by a human being, rather is automatically learned by the machine. The advantage is that the system which implements the method has increased cognitive abilities and an increased level of generalization. This makes the system which implements the method of the invention sound under the multiple conditions of soils which have a significant variance aspect in relation to the factors of atmosphere, lighting, state of the soil and the presence of shadows and/or reflections.
In a particular example embodiment, such a neural network of image processing process comprises at least one convolutional layer and the aforesaid at least one convolution operation is carried out by means of a "feed-forward" execution of the neural network. A convolutional neural network, i.e. which implements a series of successive convolutions, has the advantage of internally building a hierarchical representation which may not necessarily be interpretable - of the features of the image; this allows circumstantial features to have a high level of abstraction and soundness.
It is worth noting that the soil working means may be in various processing steps, which may be defined as tasks. Each task is a sequence of instructions which may be sent to the locomotion member 201 and are affected by a plurality of inputs, among which the synthetic descriptor DS of the soil obtained by means of image processing .
The first parameter PI representative of the confidence or reliability of the synthetic descriptor DS of the soil or the second parameter P2 representative of the specificity required by the synthetic descriptor of the soil, that is the completeness of information of the descriptor, or the third parameter mentioned above, is initially set for each task.
It is worth noting that the level of confidence and the specificity characterize each synthetic descriptor DS of the soil obtained following a given image processing.
The level of confidence of the synthetic descriptor DS in particular expresses a level of reliability of the classifier and varies from 0 to 1.
If the level of confidence is close to zero, it means that the synthetic descriptor DS of the soil is not very reliable .
Vice versa, if the level of confidence is 1, it means that the synthetic descriptor DS is very reliable.
The specificity of the synthetic descriptor DS of the soil indicates the level of detail with which it is capable of detecting the soil itself according to a given criterion.
A highly generic descriptor DS of the soil may for example, describe only if a soil may be worked or has obstacles .
Vice versa, a more specific descriptor may get into detail about the individual obstacle, recognizing it separately among a plurality of classes of obstacles. Or a specific descriptor DS may indicate the related direction of the previous working line of the soil. Or it may complete the information on the obstacle by also providing the approximate distance thereof.
Several soil image processing processes may be available in the memory of the soil working means, each with its own computational time and capable of providing synthetic descriptors of the soil with adequate information completeness or level of confidence. The case at hand considers a first processing process having a first computational complexity and a second processing process having a second computational complexity being greater than the first complexity.
In the more general example embodiment, with reference to figure 1, the control method 100 assigns a speed V of the soil working means which is adequate to the type of image processing and returns a synthetic descriptor DS of the soil, it also assigned to the specific task. Such a speed V is the speed of the movements related to the specific task. Such a speed is for example the forward, reverse, rotation speed on one axle; it may also be the speed of a combined motion of the above speeds; a particular case of speed is the null speed.
If such a level of confidence or specificity required is not sufficient for the task, advantageously the method 100 provides a reduction of the absolute speed V of the soil working means.
In an alternative example embodiment, the about- mention processing operations of method 100 of the invention may also be performed by a further electronic processing unit, different from the processing unit 204 provided on-board of the soil working means and positioned in a position which is distal from the latter, for example by a remote server with respect to the working means. Such a further processing unit comprises a respective processor and a respective storage block associated with the processor. Here, such a further processing unit or remote server is configured to communicate with the soil working means through the wired or wireless data communication interface 209 which connects such a processing unit 204 to the data communication network 210, e.g. the Internet.
A particular embodiment of the method 100 of the invention is described in reference to figure 4.
In such an example, the aforesaid step 103 of obtaining the synthetic descriptor DS also comprises a step of saving each synthetic descriptor DS of the soil based on the processing in a buffer 400 having N elements. In the example in figure 4 in particular, buffer 400, of the FIFO type, comprises four elements. Buffer 400 consists of four slots numbered #1, #2, #3,
#4.
Each element of buffer 400 includes a first, a second and a third field for saving the first PI, the second P2 and the third P3 parameters, respectively. In the example, each new descriptor DS with the related parameters (PI, P2, P3) is inserted into slot #1 (first column) . The preceding descriptor (with the related parameters) is moved into slot #2. The descriptor of the latter in turn is moved into slot #3. The descriptor first present in slot # 3 in turn is moved into slot #4. The descriptor in the latter is removed from the buffer and eliminated.
The method comprises a step of selecting values of one of such first PI or second P2 or third P3 parameter for all N elements of buffer 400.
Moreover, a step is provided for generating, from the N values of the parameter selected, a modified parameter representative of the N values of such a parameter .
In greater detail, the step of generating the modified parameter from the N values of the parameter selected is chosen from the group consisting of:
- choosing the minimum value of the parameter,
- choosing the maximum value of the parameter,
- choosing the average value.
Moreover, the aforesaid comparison step 104 of method 100 comprises a step of comparing said modified parameter with the respective threshold value associated with said first PI or second P2 or third P3 parameter.
As indicated above, more confident or more specific processing methods which return specific descriptors of the soil generally are computationally more burdensome.
The general method 100 of the invention advantageously allows taking advantage of such processing operations only when needed and with respect to a decrease in speed V of the soil working means.
Contrarily, when they are not required, such more specific or more reliable processing operations are not invoked, thus allowing the system to use the synthetic descriptors of the soil coming from lighter and therefore quicker processing operations.
By processing more FPS (frames per second), the working means may therefore proceed more quickly.
By way of a non-limiting example of the use of the method of the present invention, certain cases of use applied to cutting grass by means of an autonomous lawnmower robot which implements two or more image processing operations to obtain synthetic descriptors of the soil, are described.
EXAMPLE 1 : INFORMATION CONTENT NOT RELIABLE
For example, a first image processing process available in the memory of the lawnmower robot is capable of returning, as synthetic descriptor of the soil, a binary type classification: a "GO" class, i.e. completely grassy soil and therefore crossable, and a "NOGO" class, i.e. not completely grassy soil or soil with at least one obstacle.
During the forward task, the robot advances with a given speed V and uses the first processing process to decide whether or not to continue. The individual classifications of the soil returned are used by the processing unit 204 to decide whether or not to continue the motion or change direction.
Each descriptor DS arrives with an associated level of confidence. If the classifications are "GO" with a high level of confidence, then for example, the control logic controls the continuation of the motion. If the classifications are "NOGO" with a high level of confidence, then the lawnmower control logic controls the change in speed direction.
If instead, the classifications are uncertain, i.e. with a lower level of confidence than the specific threshold for the forward task (independently of "GO" and "NOGO"), then the specific method in figure 1 is applied, which provides a reduction of the speed, in particular the forward speed, of the lawnmower robot. According to particular aspect of the present invention described in reference to figure 4, the classifications or synthetic descriptors DS may be saved in a buffer 400 with FIFO (First In First Out) logic which contains the outputs of the last N classifications. A reduction of the forward speed V of the lawnmower robot results in classifying images of portions of soil which are closer to one another, and therefore increasing the overall confidence.
EXAMPLE 2: INFORMATION CONTENT NOT RELIABLE
For example, a first image processing process available in the memory of the lawnmower robot, which is computationally less burdensome and therefore is quicker in the execution, is capable of returning a binary type classification as synthetic descriptor of the soil: a "GO" class, i.e. completely grassy soil and therefore crossable, and a "NOGO" class, i.e. not completely grassy soil or soil with at least one obstacle.
A second image processing process available in the memory of the lawnmower robot, which is computationally more burdensome and therefore is less quick in the execution, is capable of returning a binary type classification as synthetic descriptor of the soil: a "GO" class, i.e. completely grassy soil and therefore crossable, and a "NOGO" class, i.e. not completely grassy soil or soil with at least one obstacle.
During the forward task, the lawnmower robot advances with a given speed and uses the first processing process to decide whether or not to continue. The individual classifications of the soil returned by the process are used by the processing unit to decide whether or not to continue the motion or change direction. Each classification arrives with an associated level of confidence.
If the classifications are "GO" with a high level of confidence, then for example, the control logic controls the continuation of the motion.
If the classifications are "NOGO" with a high level of confidence, then the lawnmower control logic controls the change in speed direction.
If instead, the classifications are uncertain, i.e. with a lower level of confidence than the specific threshold for the forward task (independently of "GO" and "NOGO"), then the method in figure 3 is applied, which provides a reduction of the forward speed V of the lawnmower robot and the simultaneous passing to the second image processing process, which as mentioned, is slower but more accurate than the first process.
Thereby, the level of confidence of the synthetic descriptor DS of the soil given by the second process is higher and the processing unit 204 can undertake the most suitable control actions with increased accuracy.
EXAMPLE 3 : INFORMATION CONTENT NOT SUFFICIENT
For example, a first image processing process available in the memory of the lawnmower robot, which is computationally less burdensome and therefore is quicker in the execution, is capable of returning a binary type classification as synthetic descriptor of the soil: a "GO" class, i.e. completely grassy soil and therefore crossable, and a "NOGO" class, i.e. not completely grassy soil or soil with at least one obstacle.
A second image processing process available in the memory of the lawnmower robot, which is computationally more burdensome and therefore is less quick in the execution, is capable of distinguishing a particular feature of the soil, thus providing a synthetic descriptor of the soil which is more complete from an information viewpoint.
During the forward task, the lawnmower robot advances with a given speed and uses the first processing process to decide whether or not to continue. The individual classifications of the soil returned by the process are used by processing unit to decide whether or not to continue the motion or undertake another action. Each classification arrives with an associated level of reliability.
If the classifications are "GO" with a high level of confidence, then for example, the control logic controls the continuation of the motion.
If the classifications are "NOGO" with a high level of confidence, then the lawnmower control logic does not have enough information to decide which control to impart on the motion members; thus the method in figure 3 is applied, which provides a reduction of the forward speed V of the lawnmower robot and the simultaneous passing to the second image processing process, which as mentioned, is slower but more accurate than the first process .
Thereby, the quantity of information of the synthetic descriptor DS of the soil increases, given by the second processing process, and unit 204 can undertake the most suitable control actions with increased specificity.
In a first particular example, the second processor provides an estimate of the distance between the obstacle identified and a front portion of the robot. It is apparent that the first process that obtains only the "GO" or "NOGO" crossability information is less burdensome from the computational viewpoint of the process which also estimates the distance from the obstacle. If for example, the processing processes are based on convolutional neural networks, it is known that the "detection" task uses more neurons than the "classification" task. Thus, so long as the first process detects a "GO" crossable soil, the soil working means may proceed at high speed V. When the first process detects a "NOGO", it may be useful to estimate the distance between the detected obstacle and accordingly the method in figure 3 is applied, i.e. the image is processed by means of said second computationally more burdensome process. Thus, the soil working means reduces speed V to allow the execution of the second process.
In a second particular example, the second process is configured to classify the type of obstacle, distinguishing for example, if it is a false alarm or a real obstacle. For example during the operating step, a lawnmower robot could encounter one or more leaves, which are not a true obstacle to the cutting operations. It is apparent that the first process that obtains only the "GO" or "NOGO" crossability information is computationally less burdensome than the process which also estimates the type of obstacle by distinguishing if it is a leaf or something else. If for example, the processing processes are based on convolutional neural networks, it is known that the "several-class classification" task uses more neurons than the "two- class classification" task.
Thus, so long as the first process detects a "GO" crossable soil, the soil working means may proceed at high speed V. When the first process detects a "NOGO", it may be suitable to estimate the type of obstacle detected and accordingly the method in figure 3 is applied, i.e. the image is processed by means of said second computationally more burdensome process. Thus, the soil working means reduces speed V to allow the execution of the second process.
EXAMPLE 4 : INCREASED COMPUTATION TIME
According to method 100 of the invention, a possible cause which results in the reduction of the forward speed V lies in the third parameter P3 (i.e. the inverse of the computation time) below a given threshold. The reason whereby a processing process employs more time than expected (and therefore processes less frames per second) usually lies in a concomitant process requested from the CPU. Such a concomitant process may be unexpected, or voluntarily started by the control system. It may for example, be a more complex extended image processing to obtain other navigation data than the classification of the soil to be crossed.
A particular example of such a process which could occupy the CPU is the image processing which is capable of providing the characterization of the grass cutting line in the image; for example, it may characterize the straight line which on the image divides the portion of tall grass to be cut from the portion of already cut short grass, with two offset and inclination parameters.
As noted above, the control method 100 of a soil working means of the present invention and the related system 1000 have several advantages and achieve the predetermined objects.
In particular, the control method 100 proposed allows the soil working operations to be speeded up to the greatest extent possible while simultaneously keeping high the accuracy and specificity of the control actions. The solution proposed in particular uses burdensome classifications from a computational viewpoint when they are needed and light classifications when they are needed, thus being highly versatile with respect to the known solutions.
However, it is worth noting that the slowdown is not due to a sensorial limit (e.g. the camera does not correctly acquire the image) , rather always to a computational limit in terms of non-adequate confidence, information completeness or rapidity of the synthetic descriptor DS of the soil. In other words, the present invention consists in identifying the overcoming of a cognitive limit according to the task performed and speed V, and accordingly reduces speed V.
Finally, unlike a road vehicle, a soil working means like the one the object of the invention may decide to stop "to think" at any time. For this reason, the system is applied without limitation to lawnmower robots, but more generally to yard service robots, and even more generally, to soil working means.
Those skilled in the art may make several changes and adaptations to the embodiments of the method and system of the invention, and may replace elements with others which are functionally equivalent in order to meet contingent needs, without departing from the scope of the following claims. Each of the features described as belonging to a possible embodiment may be achieved irrespective of the other embodiments described.
k k k k k k k

Claims

1. A method (100) for controlling a soil working means based on image processing, said soil working means comprising a locomotion member (201) for moving the working means at a predetermined speed (V) , wherein the method comprises the steps of:
acquiring (101) at least one digital image of the soil by means of digital image acquisition means (203) installed on said working means;
- processing (102), by an electronic processing unit
(204), the at least one digital image acquired through a first processing process with which a first computational complexity is associated;
obtaining (103), from the electronic processing unit (204), at least one synthetic descriptor (DS) of the soil based on said processing, said synthetic descriptor (DS) including a first parameter (PI) representative of a reliability of the description of the soil by the synthetic descriptor (DS) or a second parameter (P2) representative of the completeness of information contained in the synthetic descriptor (DS) or a third parameter (P3) representative of the reciprocal of the computing time used by said processing step to obtain the synthetic descriptor;
comparing (104) at least one of said first (PI), second (P2) and third ( P3 ) parameters representative of the synthetic descriptor with a respective preset threshold value (kl, k2, k3);
if the at least one of said first (PI), second (P2) and third ( P3 ) parameters is lower than the respective preset threshold value, the method comprises the step (105) of generating, by the electronic processing unit (204), a control signal of said locomotion member (201) to reduce the speed (V) of the soil working means.
2 . A method (100) for controlling a soil working means according to claim 1, wherein if the at least one of said first (PI), second (P2) and third (P3) parameters is lower than the respective preset threshold value (kl, k2, k3), said processing step (102) comprises the further steps of:
- replacing (1021) said first processing process with a second processing process with which a second computational complexity being greater than said first computational complexity, is associated;
- processing (1022), by the electronic processing unit (204), the at least one digital image acquired through said second processing process.
3 . A method (100) for controlling a soil working means according to claim 1 or 2, wherein said first or second processing process comprises a neural network.
4 . A method (100) for controlling a soil working means according to claim 3, wherein said neural network comprises at least one convolutional layer and said at least one convolution operation is carried out by means of a "feed-forward" execution of the neural network.
5 . A method (100) for controlling a soil working means according to claim 1, wherein said obtaining step (103) further comprises the steps of:
- saving each synthetic descriptor (DS) of the soil obtained based on said processing in a buffer (400) having N elements, each element of the buffer (400) including a first, a second and a third field for saving said first (PI), second (P2) and third (P3) parameters, respectively;
- selecting values of one of said first (PI) or second (P2) or third (P3) parameter for all the N elements of the buffer;
- generating a modified parameter representative of the N values of said parameter from the N values of the parameter selected,
and wherein said comparison step (104) comprises a step of comparing said modified parameter with the respective threshold value associated with said first (PI) or second (P2) or third (P3) parameter.
6. A method (100) for controlling a soil working means according to claim 5, wherein said step of generating the modified parameter from the N values of the parameter selected is chosen from the group consisting of:
- choosing the minimum value of the parameter,
- choosing the maximum value of the parameter,
- choosing the average value.
7 . A method (100) for controlling a soil working means according to any one of the preceding claims, wherein said speed (V) of the working means is chosen from the group consisting of:
forward speed,
reverse speed,
rotation speed on one axle,
null speed,
speed of a combined motion of the above speeds.
8. A system (1000) comprising:
a soil working means which includes a locomotion member (201 ) ;
digital image acquisition means (203) installed on said working means for acquiring at least one digital image of the soil;
an electronic processing unit (204) connected to said digital image acquisition means and to said locomotion member (201) of the soil working means, said electronic processing unit (204) comprising at least one processor (205) and a storage block (206, 207) associated with the processor for storing instructions, said processor and said storage block being configured to perform the steps of the method according to any one of claims 1 to 8.
9 . A system (1000) according to claim 8, wherein said soil working means is chosen from the group consisting of :
- lawnmower machine or lawnmower robot,
- reaping machine,
- plow,
- grain cutting-harvesting machine,
- vegetable cutting-harvesting machine,
- hay cutting-harvesting machine,
- snow plough machine or robot,
- any machine, the passage of which, over a soil, alters the visual perception of the soil.
10 . A system (1000), comprising:
a soil working means which includes a locomotion member (201 ) ;
digital image acquisition means (203) installed on said working means for acquiring at least one digital image of the soil; an electronic processing unit (204) connected to said digital image acquisition means and to said locomotion member (201) of the soil working means;
- a further electronic processing unit, separate from said electronic processing unit (204), and positioned in a position which is distal from the soil working means, said further electronic processing unit being configured to communicate with the soil working means through a data communication interface (209) which connects the processing unit (204) to a data communication network (210) ,
said further electronic processing unit comprising at least one respective processor and a respective storage block associated with the processor to store instructions, said processor and said storage block being configured to perform the steps of the method according to any one of claims 1 to 8.
PCT/IB2019/060228 2018-11-30 2019-11-27 An improved method for controlling a soil working means based on image processing and related system WO2020110027A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP19831890.9A EP3886564A1 (en) 2018-11-30 2019-11-27 An improved method for controlling a soil working means based on image processing and related system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
IT102018000010737A IT201800010737A1 (en) 2018-11-30 2018-11-30 METHOD OF CONTROL OF A SOIL PROCESSING MEANS ON THE BASIS OF IMPROVED IMAGE PROCESSING AND RELATED SYSTEM
IT102018000010737 2018-11-30

Publications (1)

Publication Number Publication Date
WO2020110027A1 true WO2020110027A1 (en) 2020-06-04

Family

ID=65767185

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2019/060228 WO2020110027A1 (en) 2018-11-30 2019-11-27 An improved method for controlling a soil working means based on image processing and related system

Country Status (3)

Country Link
EP (1) EP3886564A1 (en)
IT (1) IT201800010737A1 (en)
WO (1) WO2020110027A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011002512A1 (en) * 2009-07-01 2011-01-06 Mtd Products Inc Visual segmentation of lawn grass
EP2620050A1 (en) * 2012-01-25 2013-07-31 Honda Research Institute Europe GmbH System, method and apparatus for unsupervised adaptation of the perception of an autonomous mower
EP2689650A1 (en) * 2012-07-27 2014-01-29 Honda Research Institute Europe GmbH Trainable autonomous lawn mower
EP3123858A1 (en) * 2015-07-29 2017-02-01 CLAAS Omaha Inc. Agricultural system
WO2018123631A1 (en) * 2016-12-28 2018-07-05 Honda Motor Co.,Ltd. Control device, work machine and program

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011002512A1 (en) * 2009-07-01 2011-01-06 Mtd Products Inc Visual segmentation of lawn grass
EP2620050A1 (en) * 2012-01-25 2013-07-31 Honda Research Institute Europe GmbH System, method and apparatus for unsupervised adaptation of the perception of an autonomous mower
EP2689650A1 (en) * 2012-07-27 2014-01-29 Honda Research Institute Europe GmbH Trainable autonomous lawn mower
EP3123858A1 (en) * 2015-07-29 2017-02-01 CLAAS Omaha Inc. Agricultural system
WO2018123631A1 (en) * 2016-12-28 2018-07-05 Honda Motor Co.,Ltd. Control device, work machine and program

Also Published As

Publication number Publication date
IT201800010737A1 (en) 2020-05-30
EP3886564A1 (en) 2021-10-06

Similar Documents

Publication Publication Date Title
US10102429B2 (en) Systems and methods for capturing images and annotating the captured images with information
US11402848B2 (en) Collision-avoidance system for autonomous-capable vehicles
CN111434554B (en) Controlling an autonomous vehicle based on passenger and context aware driving style profiles
Ollis et al. Vision-based perception for an automated harvester
Subramanian et al. Sensor fusion using fuzzy logic enhanced kalman filter for autonomous vehicle guidance in citrus groves
GB2551001A (en) Vision-Based Rain Detection Using Deep Learning
US11768080B2 (en) Optimal route searching device and operation method thereof
CN110723072B (en) Driving assistance method and device, computer equipment and storage medium
KR102525191B1 (en) System and method for generating and controlling driving paths in autonomous vehicles
US20190130223A1 (en) System and method for system-aware classifiers
CN114265409A (en) Track information processing method and device and ground equipment
Guan et al. Vinet: Visual and inertial-based terrain classification and adaptive navigation over unknown terrain
CN113228131A (en) Method and system for providing ambient data
Sun et al. Learning from examples in unstructured, outdoor environments
WO2020110027A1 (en) An improved method for controlling a soil working means based on image processing and related system
CN115416692A (en) Automatic driving method and device and electronic equipment
US11433901B2 (en) Selection of a main object for an assistance function or automated driving function of a driver assistance system or driving system of a motor vehicle
KR102238263B1 (en) Autonomous tractor having crop hight sensing algorithm
WO2020210493A1 (en) Autonomous machine navigation in lowlight conditions
Elbagoury et al. A hybrid liar/radar-based deep learning and vehicle recognition engine for autonomous vehicle Precrash control
CN111930117B (en) Steering-based lateral control method, device, equipment and storage medium
US20240043016A1 (en) Computer-Implemented Method for Estimating a Vehicle Position
US20220292376A1 (en) Methods for Compressing a Neural Network
US11603119B2 (en) Method and apparatus for out-of-distribution detection
Rosenblum Neurons that know how to drive

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19831890

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019831890

Country of ref document: EP

Effective date: 20210630