EP4716930A1 - Determining a value for a swath - Google Patents

Abstract

A mechanism for predicting a value responsive to a dimension of a swath produced by a harvesting machine. Monocular image data of the swath is obtained and processing using one or more image processing algorithms to predict the value.

Description

TITLE

DETERMINING A VALUE FOR A SWATH

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] Not applicable.

FIELD

[0002] Embodiments of the present disclosure generally relate to the field of harvesting machinery.

BACKGROUND

[0003] With ever-increasing population numbers and ongoing interest in more environmentally friendly farming practices, there is an increasing desire to reduce waste when harvesting crop and improve the efficiency of harvesting machinery, such as combine harvesters, windrowers, mowers, and so on.

[0004] One outcome of interest in combine harvesting is to provide high quality swath when harvesting a cereal product. In the field of combine harvesting, swath is the label given to the collection of material other than grain (MOG) that is output/ejected from the combine harvester after the cereal product has been removed. Typically, this MOG comprises at least the stalks of the harvested crop, i.e., straw. More particularly, a combine harvester is able to separate grain from a harvested crop (which is then typically stored in a grain bin) and output or eject the remaining material of the harvested crop (particularly the stalks) from the combine harvester. Another label for the remaining material is tailings. The collection of ejected material or tailings is commonly labelled swath. Thus, the swath can primarily comprise unbaled straw or unbaled stalks of a cereal crop.

[0005] In other forms of harvesting, the swath is the crop material left behind, e.g., on the ground, after the harvester machine has interacted with the crop. This can similarly include cut or felled straw, grass or the like.

[0006] It is desirable for material in this swath to be of high quality, e.g., such that a large percentage of the stalks making up the swath are unbroken. This facilitates ease of later baling or collecting of the swath, and is advantageous for providing animal bedding and the like.

BRIEF SUMMARY

[0007] The invention is defined by the claims.

[0008] According to examples in accordance with an aspect of the invention, there is provided A computer-implemented method of determining a value responsive to one or more dimensions of a swath produced by a harvesting machine.

[0009] The computer-implemented method comprises: obtaining first monocular image data containing a representation of a portion of the swath produced by the harvesting machine, the first monocular image data being captured using a single camera mounted on the harvesting machine; and processing the first monocular image data using one or more image processing algorithms to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data.

[0010] There is proposed a method for processing first monocular image data of a swath to predict one or more dimensions (e.g., height or volume) of the swath produced. One or more image processing algorithms process the first monocular image data in order to determine or derive the value responsive to or of the dimension(s) of the swath produced by the harvesting machine.

[0011] Determining the dimension(s) of a swath is advantageous in selection of certain parameters for elements of a harvesting machine. In particular, a reduced height or volume of the swath (e.g., compared to an expected or historic height or volume) may indicate that there is significant damage to the material making up the swath during processing by the harvesting machine. One or more parameters of the harvesting machine can therefore be controlled responsive to the detected dimension(s)

[0012] Any herein described method for determining a value responsive to one or more dimensions of a swath can be repurposed to determine a value responsive to one or more dimensions of any other form of material ejected or output from an agricultural machine/vehicle, such as material output by a spreader or the like. In such approaches, the term "swath" may be replaced by the term "output material" or "ejected material". Similarly, the term "harvesting machine" may be replaced by the term "agricultural machine". [0013] The method may further comprise obtaining second monocular image data containing a representation of stubble and/or ground surface remaining after harvesting of crop by the harvesting machine and/or a cutting bar of the harvesting machine, wherein processing the first monocular image data comprises processing the first monocular image data and the second monocular image data to predict the value responsive to one or more dimensions of the swath.

[0014] In some examples, the first monocular image data comprises a first sequence of monocular images, each monocular image containing a representation of the portion of the swath produced by the harvesting machine.

[0015] The computer-implemented method may further comprise obtaining first movement information identifying a movement of the harvesting machine between capturing adjacent monocular images in the first sequence of monocular images. The step of processing the first monocular image data may comprise: processing the first sequence of monocular images, and no other image data, together with the first movement information using a first monocular simultaneous localization and mapping, SLAM, algorithm to produce a plurality of first points, each identifying a predicted location of a portion of a top of the swath with respect to a co-ordinate system; and processing the plurality of first points to predict the value responsive to the one or more dimensions of the swath represented in the first monocular image data.

[0016] This approach makes use of a SLAM algorithm to effectively track or monitor the top of a swath as depicted in sequential monocular images. It has been recognized that a monocular SLAM technique or algorithm is able to produce a plurality of points in a 3D space or 3D co-ordinate system that represent (or are predicted to represent) the top of a swath represented within the (SLAM -processed) sequence of monocular images.

[0017] In some embodiments, the step of processing the plurality of first points comprises: fitting a contour to the first points to produce a first contour that represents the top of the swath; and processing the first contour to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data.

[0018] In some examples, the step of processing the plurality of first points comprises fitting a contour to the second points to produce a second contour that represents the stubble and/or ground surface with respect to the co-ordinate system; and the step of processing the first contour comprises processing the first contour and the second contour to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data.

[0019] The step of processing the first contour and the second contour may comprise: identifying a first portion of the first contour; identifying a second portion of the second contour, the first portion of the first contour being located proximate to the second portion of the second contour within the co-ordinate system; and determining a vertical distance or average vertical distance within the co-ordinate system between the first portion of the first contour and the second portion of the second contour to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data.

[0020] In some examples, the first portion of the first contour comprises more than one point of the first contour; and the second portion of the second contour comprises more than one point of the second contour.

[0021] In some examples, the first sequence of monocular images and the second sequence of monocular images are the same, such that each monocular image contains a representation of a portion of the swath produced by the harvesting machine and a portion of a top of the stubble and/or ground surface after harvesting of crop by the harvesting machine.

[0022] The distance information may be derived from: a speed of the harvesting machine and a time elapsed between capturing each monocular image; and/or navigation information identifying, for each monocular image, a position at which the monocular image was captured.

[0023] The step of processing the monocular image data may comprise processing the first monocular image data, and no other image data, using a machine-learning method to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data.

[0024] The value may be a measure of one of the one or more dimensions, the one or more dimensions optionally comprising a height of the swath and/or a volume of the swath. [0025] There is also proposed a computer-implemented method of controlling one or more parameters of a combine harvester. The computer-implemented method comprises: determining the value responsive to one or more dimensions of a swath produced by the combine harvester by performing any herein proposed method; and adjusting, responsive to the determined value responsive to one or more dimensions of the swath, one or more of: a rotation speed of a threshing cylinder of the combine harvester; a distance between the threshing cylinder and a concave of the combine harvester; and/or a number of rotations performed by a harvested crop during threshing by the combine harvester before being ejected from the combine harvester.

[0026] There is also proposed a computer program product comprising computer program code means which, when executed on a computing device having a processing system, cause the processing system to perform all of the steps of any herein proposed method.

[0027] There is also proposed a processing system configured to determine a value responsive to one or more dimensions of a swath produced by a harvesting machine, the processing system being configured to: obtain first monocular image data containing a representation of a portion of the swath produced by the harvesting machine, the first monocular image data being captured using a single camera mounted on the harvesting machine; and process the first monocular image data using one or more image processing algorithms to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data.

[0028] The processing system may be adapted to perform the functions of any herein described method, and vice versa.

[0029] There is also proposed an image processing system comprising the processing system and the single camera for capturing the first monocular image data. There is also proposed a harvesting machine comprising the image processing system.

[0030] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

[0031] One or more embodiments of the disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0032] FIG. 1 illustrates a harvesting machine in which embodiments can be employed;

[0033] FIG. 2 illustrates a portion of a combine harvester;

[0034] FIG. 3 illustrates a header of a harvesting machine;

[0035] FIG. 4 is a flowchart illustrating a proposed method; [0036] FIG. 5 is a flowchart illustrating an embodiment of the proposed method;

[0037] FIG. 6 is a flowchart illustrating another embodiment of the proposed method;

[0038] FIG. 7 illustrates a technique for predicting a measure of swath height;

[0039] FIG. 8 illustrates a SLAM algorithm;

[0040] FIG. 9 illustrates another proposed method;

[0041] FIG. 10 illustrates a proposed system; and

[0042] FIG. 11 illustrates a proposed processing system.

DETAILED DESCRIPTION

[0043] The invention will be described with reference to the figures.

[0044] It should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the apparatus, systems and methods, are intended for purposes of illustration only and are not intended to limit the scope of the invention. These and other features, aspects, and advantages of the apparatus, systems and methods of the present invention will become better understood from the following description, appended claims, and accompanying drawings. It should be understood that the figures are merely schematic and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the figures to indicate the same or similar parts.

[0045] The invention provides a mechanism for predicting a value responsive to a dimension of a swath produced by a harvesting machine. Monocular image data of the swath is obtained and processing using one or more image processing algorithms to predict the value.

[0046] FIG. 1 conceptually illustrates a harvesting machine 10 in the form of a combine harvester, for improved contextual understanding.

[0047] More particularly, FIG. 1 shows a known combine harvester 10 in which embodiments may be integrated. The combine harvester includes a threshing unit 20 for detaching grains of cereal from the ears of cereal, and a separating unit 30 which is connected downstream of the threshing unit 20. The grains after separation by the separating device 30 pass to a grain cleaning apparatus 40. Non-grain material may be ejected from the combine harvester through one or more output apertures 75 of the combine harvester. This non-grain material may, for instance, comprise at least the stalks of the harvested cereal crop. [0048] The combine harvester has a front elevator housing 12 at the front of the machine for attachment of a crop cutting head 15 (commonly known as the header). The header 15, when attached, serves to cut and collect the crop material as it progresses across the field. The header 15 comprises a cutter bar 16 that engages with a base of the crop to cut the crop. The header 15 also comprises a reel 17 or harvesting reel. The reel 17 is configured to engage with a top of the crop and encourage or guide the crop into the header. A conveyor 18 is configured to convey the crop, cut by the cutter bar 16, into the elevator housing 12. Generally, the reel comprises a plurality of tine bars (each comprising a plurality of tines) that sequentially engage with the top of the crop as the reel rotates. The tines of each tine bar act to lift and separate the crop, and to guide the crop towards the header, particularly the conveyor 18 of the header.

[0049] The crop stream produced in this way by the header 15 is conveyed up through the elevator housing 12 into the threshing unit 20. In the example shown, the threshing unit 20 is a transverse threshing unit, i.e. formed by rotating elements with an axis of rotation in the side-to-side direction of the combine harvester and for generating a tangential flow.

[0050] The operation of the combine harvester may be controlled by a control system (not shown). The control system may receive input from a user interface and/or sensing apparatus and control the operation of the various units and apparatus responsive to the received input.

[0051] The combine harvester 10 may also comprise a user support 80, e.g. a cab, for housing an operator/individual. The user support will often contain a user interface to allow the operator/individual to influence or control the operation of the elements of the combine harvester (e.g. via the control system). The user interface may also provide information about the combine harvester and/or the status of the combine harvester.

[0052] The combine harvester 10 may also comprise at least one camera 90, 91, for use in embodiments of the invention. The purpose of the camera will be elucidated later in this disclosure.

[0053] The threshing unit 20, separating device 30 and grain cleaning apparatus 40 are shown in more detail in Fig. 2.

[0054] FIG. 2 shows one particular design for a threshing unit, namely a traverse threshing unit. The transverse threshing unit 20 includes a rotating, tangential-flow, threshing cylinder 22 and a concave-shaped grate 24, sometimes simply called a concave. The threshing cylinder 22 includes rasp bars (not shown) which act upon the crop stream to thresh the grain or seeds from the remaining material, the majority of the threshed grain passing through the underlying grate 24 and onto a stratification pan 42 (also known as the grain pan), which for convenience is in this disclosure considered to be part of the grain cleaning apparatus 40.

[0055] The threshing unit 20 also comprises a beater cylinder (also with a transverse rotation axis and creating a tangential flow), downstream of the threshing cylinder and a tangential-flow multi-crop separator cylinder (also with a transverse rotation axis and creating a tangential flow) downstream of the beater cylinder.

[0056] The threshing unit 20 shown in this example thus has a well-known set of three transversely mounted rollers and cylinders (otherwise known as drums). However, there are other transverse rotation (and hence tangential flow) threshing units. Typically, there is at least one threshing cylinder, and often also a beater cylinder.

[0057] The remainder of the crop material including straw, tailings and un-threshed grain are passed from the threshing unit 20 into the separating unit 30.

[0058] In the example shown, the separating unit 30 includes a plurality of parallel, longitudinally-aligned, straw walkers 32, and this is suitable for the case of a so-called strawwalker combine. However, the separating unit 30 may instead include one or two longitudinally-aligned rotors which rotate about a longitudinal axis and convey the crop stream rearwardly in a ribbon passing along a spiral path. This is the case for a so-called axial or hybrid combine harvester.

[0059] In all cases, the separating unit 30 serves to separate further grain from the crop stream, and this separated grain passes through a grate-like structure onto an underlying return pan 44. The residue crop material, predominantly made up of straw, exits the machine at the rear. Although not shown in Fig. 1, a straw spreader and/or chopper may be provided to process the straw material as required.

[0060] The threshing apparatus 20 and separating unit 30 do not remove all material other than grain, "MOG", from the grain so that the crop stream collected by the stratification pan 42 and return pan 44 typically includes a proportion of straw, chaff, tailings and other unwanted material such as weed seeds, bugs, and tree twigs. The remainder of the grain cleaning apparatus 40 (i.e. a grain cleaning unit 50) is provided to remove this unwanted material thus leaving a clean sample of grain to be delivered to the tank. [0061] For clarity, the term 'grain cleaning apparatus' is intended to include the stratification pan 42, the return pan 44 and other parts which form the grain cleaning unit 50 (also known as a cleaning shoe).

[0062] The grain cleaning unit 50 also comprises a fan unit 52 and sieves 54 and 56. The upper sieve 54 is known as the chaffer.

[0063] The stratification pan 42 and return pan 44 are driven in an oscillating manner to convey the grain and MOG accordingly. Although the drive and mounting mechanisms for the stratification pan 42 and return pan 44 are not shown, it should be appreciated that this aspect is well known in the art of combine harvesters and is not critical to disclosure of the invention. Furthermore, it should be appreciated that the two pans 42, 44 may take a ridged construction as is known in the art.

[0064] The grain passing through concave grate 24 falls onto the front of the stratification pan 42 as indicated by arrow A in Fig. 2. This material is conveyed rearwardly (in the direction of arrow B in Fig. 2) by the oscillating motion of the stratification pan 42 and the ridged construction thereof. Material passing through the grate of the separator apparatus 30 falls onto the return pan 44 and is conveyed forwardly by the oscillating motion and ridged construction thereof as shown by arrow C.

[0065] It is noted that "forwardly" and "rearwardly" refer to direction relative to the normal forward direction of travel of the combine harvester.

[0066] When the material reaches a front edge of the return pan 44 it falls onto the stratification pan 42 and on top of the material conveyed from the threshing unit 20 as indicated by arrow B.

[0067] The combined crop streams thus progress rearwardly towards a rear edge of the stratification pan 42. Whilst conveyed across the stratification pan 42, the crop stream, including grain and MOG, undergoes stratification wherein the heavier grain sinks to the bottom layers adjacent stratification pan 42 and the lighter and/or larger MOG rises to the top layers.

[0068] Upon reaching the rear edge of the stratification pan 42, the crop stream falls onto the chaffer 54 which is also driven in a fore-and-aft oscillating motion. The chaffer 54 is of a known construction and includes a series of transverse ribs or louvers which create open channels or gaps therebetween. The chaffer ribs are angled upwardly and rearwardly so as to encourage MOG rearwardly whilst allowing the heavier grain to pass through the chaffer onto an underlying second sieve 56.

[0069] The chaffer 54 is coarser (with larger holes) than second sieve 56.

[0070] It is known for chaffer 54 to include an inclined rear extension section (not shown), and MOG which reaches the rear section either passes over the rear edge and out of the machine or through the associated grate before being conveyed to a returns auger 60 for re-threshing in a known manner. The majority of materials passing through the rear end of the chaffer 54 is un-threshed tailings. The material (MOG) that exits the machine is deposited as a swath of material alongside/behind the combine harvester.

[0071] Grain passing through chaffer 54 is incident on the lower sieve 56 which is also driven in an oscillating manner and serves to remove tailings from the stream of grain before being conveyed to an on-board tank (not shown) by grain collecting auger 70 which resides in a transverse trough 72 at the bottom of the grain cleaning unit 50. Tailings blocked by sieve 56 are conveyed rearwardly by the oscillating motion thereof to a rear edge from where the tailings are directed to the tailings processing unit 60 or returns auger for reprocessing in a known manner.

[0072] The flow of material over the end of the stratification pan 42, shown as arrow D, is known as a cascade. It is desirable for this cascade to form a thin layer so that the airflow from the fan unit 52 is able to pass through the layer and lift the MOG away from the grains. [0073] To assist this operation it is also known to have an additional cascade pan between the stratification pan 42 and the chaffer 54. The grain and chaff then initially falls from the stratification pan 42 onto the cascade pan before falling from the rear edge thereof onto the chaffer 54. The cascade pan has a grid to convey long straw and weeds rearwardly and away from the cascading grain flow. The cascade pan assists the separation of grain from MOG.

[0074] In this case, fan unit 52 delivers a portion of a cleaning airstream rearwardly between the stratification pan 42 and the cascade pan and another portion rearwardly between the chaffer 54 and the cascade pan 46, and between the sieves.

[0075] The fan unit 52 thus generates a cleaning air stream which is directed through the falling grain and chaff cascade. The fan 52 rotates on a transverse axis in a known manner and includes a plurality of impellor blades which draw in air from the transverse ends open to the environment and generate an air stream as explained above in a generally rearward direction. The air stream creates a pressure differential across the chaffer 54 and sieve 56 to encourage lighter MOG rearwardly and upwardly whilst allowing the grain to pass through the chaffer 54 and the sieve 56.

[0076] The operation of the various units and elements of the combine harvester may be controlled by a control unit (not shown). For instance, the control unit may modify one or more operational components of the combine harvester.

[0077] The present disclosure relates to approaches for predicting a value responsive to one or more dimensions of a swath produced by a harvesting machine. As previously explained, for a combine harvester, the swath is typically produced from the non-grain (MOG) material that is output from the harvesting machine. Other harvesting machines may provide or deposit swath using other techniques.

[0078] FIG. 3 conceptually illustrates a portion of a harvesting machine for further contextualization. In particular, FIG. 3 illustrates the rear of the harvesting machine following a harvesting procedure.

[0079] It has been previously mentioned how material (e.g., non-grain material or MOG) is ejected from a harvesting machine after harvesting of crop. The material may be ejected through at least one output aperture 75 of the harvesting machine 10. The ejected material forms a swath 310 on a ground surface 350. In this way, the harvesting machine produces a swath of material.

[0080] It would be desirable to determine or predict a value responsive to one or more dimensions of the swath. This value may, for instance, be a predicted measure of the swath such a predicted height hi, width, cross-sectional area and/or volume (e.g., measure of amount of material) of the swath. As another example, the value may be a measure of uniformity of the swath, e.g., a measure of whether the swath has been deposited evenly. It will be appreciated that such measures are responsive to the dimension(s) of the swath, e.g., changes in the value of a particular dimension will indicate changes in the uniformity of the swath.

[0081] Knowledge of such values is beneficial for monitoring an operation of the harvesting machine. For instance, changes in the dimensions or uniformity of the swath may indicate or imply that the crop has changed (e.g., different moisture levels) or is being processed differently. For instance, a change in height or volume of the swath may indicate a different amount of damage to the material making up the swath during processing by the harvesting machine.

[0082] Information on the value responsive to one or more dimensions of the swath is also useful for later capturing or collection of the swath, e.g. to select appropriately sized gathering (e.g., baling) equipment to collect or gather the swath.

[0083] There is therefore a clear desire to facilitate the capture of one or more pieces of information that represents or responds to changes in the dimension(s) of the swath produced by the harvesting machine.

[0084] FIG. 4 is a flowchart illustrating a computer-implemented method 400 of predicting a value responsive to one or more dimensions of a swath produced by a harvesting machine.

[0085] The method 400 comprises a step 410 of obtaining first monocular image data containing a representation of a portion of the swath produced by the harvesting machine. The first monocular image data being captured using a single camera mounted on the harvesting machine.

[0086] The first monocular image data is captured by a single camera mounted on the harvesting machine, e.g., the camera 91 illustrated in FIG. 1. The single camera may iteratively produce monocular images of a view that includes any swath produced behind or to the side the harvesting machine. Step 410 may comprise obtaining the first monocular image data from this camera.

[0087] The method 400 also comprises a step 420 of processing the first monocular image data using one or more image processing algorithms to predict the value responsive to one or more dimensions of a swath produced by a harvesting machine. More detailed examples of how to perform this step are provided later.

[0088] Optionally, the method 400 also comprises a step 430 of obtaining second monocular image data containing a representation of stubble and/or ground surface remaining after harvesting of the crop by the harvesting machine and/or a representation of the cutting bar during harvesting of the crop. In this approach, step 420 comprises processing the first monocular image data and the second monocular image data to predict the value responsive to one or more dimensions of a swath produced by a harvesting machine.

[0089] Like the first monocular image data, the second monocular image data is captured by a single camera mounted on the harvesting machine. The single camera may iteratively produce monocular images of a view that includes the stubble or ground surface remaining after harvesting of the crop by the harvesting machine and/or the cutting bar during harvesting. By way of example, the second monocular image data may capture monocular images of the stubble or ground surface between the cutting bar and the remainder of the harvesting machine.

[0090] In preferred examples, a single camera captures the first and second monocular image data. However, this is not essential. Rather, different cameras may capture the first and second monocular image data - e.g., a first camera on the rear of the harvesting machine may capture the first monocular image data and a second camera on the front of the harvesting machine may capture the second monocular image data.

[0091] The method 400 may comprise other steps (not shown), e.g., obtaining movement information or the like. The inclusion of other steps may depend upon the precise procedure that is used in step 420 to predict the value for the swath, as will become clear from the following examples.

[0092] Various approaches for performing step 420 are envisaged, a non-exhaustive number of which are hereafter described.

[0093] FIG. 5 illustrates an embodiment of method 400 that employs one example approach for performing step 420. This approach makes use of the first monocular image data (obtained in step 410) and first movement information (obtained in a step 505).

[0094] The first monocular image data comprises a first sequence of monocular images. Each monocular image contains a representation of the portion of the swath produced by the harvesting machine.

[0095] The first movement information identifies a movement of the harvesting machine between capturing adjacent monocular images in the first sequence of monocular images.

[0096] By way of example only, the first movement information may be distance information that identifies a distance between adjacent monocular images in the first sequence of images. Such distance information may, for example, be derived from a speed of the harvesting machine and a time elapsed between capturing each monocular image. Suitable components for obtaining such forms of data are well known to the skilled person, e.g., using a tachometer, speedometer, accelerometer (and/or other form of inertial measurement unit) and/or satellite navigation systems. As another example, the distance information may be derived from navigation information identifying, for each monocular image, a position at which the monocular image was captured.

[0097] As another example, the first movement information may be inertia information that identifies a change in inertia between adjacent monocular images within the sequence. Such information can, for instance, be generated by an accelerometer or other form of inertial measurement unit.

[0098] Accordingly, the method 400 may comprise a step 505 of obtaining first movement information. As previously explained, the first movement information may be produced by deriving distance information or inertia information from other forms of data (e.g., speed information and/or navigation information).

[0099] The step 420 of processing the first monocular image data comprises a substep 510 of processing the first sequence of monocular images (and no other image data) together with the first movement information using a first monocular simultaneous localization and mapping, SLAM, algorithm to produce a plurality of first points. Each first point identifies a predicted location of a portion of the swath with respect to a co-ordinate system, e.g. a 3D co-ordinate system or 2D co-ordinate system. Examples of suitable SLAM algorithms for producing such a plurality of first points are described later in this disclosure.

[0100] The step 420 also comprises a sub-step 420 of fitting a contour (sometimes called a surface) to the first points to produce a first contour that represents the top of the swath. Approaches to fitting a contour or surface to a plurality of first points are well known in the art. Such approaches may, for instance, comprise ignoring outlying points and fitting a contour or surface of best fit that aligns with the first points.

[0101] Examples of suitable approaches are described by: Hackel, Timo, Jan D. Wegner, and Konrad Schindler. "Contour detection in unstructured 3D point clouds." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016; Mitra, Niloy J., and An Nguyen. "Estimating surface normals in noisy point cloud data." Proceedings of the nineteenth annual symposium on Computational geometry. 2003; or Berger, Matthew, et al. "A survey of surface reconstruction from point clouds." Computer graphics forum. Vol. 36. No. 1. 2017.

[0102] The method 400 then performs the step 530 of using the first contour to predict a value for the swath produced by the harvesting machine. The value for the swath is a value responsive to one or more dimensions of the swath produced by a combine harvester. [0103] One effect of using a SLAM algorithm is that the relative position of the camera capturing the monocular images processed by the SLAM algorithm is determined (i.e., localized). If the position of the camera with respect to a ground surface is determinable or otherwise known (e.g., at a known fixed position - as would be the case for a camera on a harvesting machine), then the relative position of the first contour with respect to a ground surface can be trivially determined. Thus, it is possible to determine, using the first contour, a measure of a dimension of the swath using the known camera position. For instance, a height of the swath can be readily determined by identifying the difference between the top of the swath (defined by the first contour) and the ground surface.

[0104] The region or space between the identified top of the swath (defined by the first contour) and the ground surface may represent a space occupied by the swath. One or more dimensions of this space may be calculated in determining a value responsive to one or more dimensions of the swath.

[0105] A measure of a width of the swath can be readily determined by identifying a width of the top of the swath, e.g., a width across the contour representing the top of the swath.

[0106] A measure of a cross-sectional area and/or volume of the swath can be readily and trivially determined using the contour and a known position for the ground. The volume of the swath may, for instance, be a volume with a predefined travel distance of the harvesting machine, e.g., of a fixed depth.

[0107] A measure of uniformity could, for instance, be determined by determining a size of a range of a measure of one or more dimensions or space (e.g., height, width, cross- sectional area and/or volume) of the swath across a predetermined distance or time of travel of the harvesting machine.

[0108] FIG. 6 illustrates another embodiment of method 400 that employs another example approach for performing step 420. This approach makes use of the first monocular image data (obtained in step 410), first movement information (obtained in a step 505), second monocular image data (obtained in a step 430) and second movement information (obtained in a step 605).

[0109] As in the previous embodiment, the first monocular image data comprises a first sequence of monocular images, each monocular image containing a representation of the portion of the swath produced by the harvesting machine. Similarly, the first movement information identifies a movement of the harvesting machine between capturing adjacent monocular images in the first sequence of monocular images.

[0110] The second monocular image data comprises a second sequence of monocular images. The second monocular image data comprises a second sequence of monocular images each containing a representation of stubble and/or ground surface remaining after harvesting of crop by the harvesting machine and/or the cutting bar of the header.

[0111] The second movement information identifies a movement travelled by the harvesting machine between capturing adjacent monocular images in the second sequence of monocular images. The second movement information may be analogous to the first movement information, and may comprise/contain similar data obtained in a similar manner. [0112] The step 420 in method 400 illustrated by FIG. 6 also comprises sub-steps 510, 520, which are embodied as previously described.

[0113] The step 420 in method 400 illustrated by FIG. 6 further comprises a sub-step 610 of processing the second sequence of monocular image data, and no other image data, together with the second movement information using a second monocular simultaneous localization and mapping, SLAM, algorithm to produce a plurality of second points. Each identifying a predicted location of a portion of a top of the stubble and/or ground surface and/or cutting bar represented in the second sequence of monocular images.

[0114] The step 420 in method 400 illustrated by FIG. 6 further comprises a sub-step 620 of fitting a contour to the second points to produce a second contour that represents the stubble and/or ground surface and/or cutting bar with respect to the co-ordinate system.

[0115] The step 530 of predicting the value for the swath comprises processing the first contour and the second contour to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data.

[0116] It will be appreciated that step 530 may be iteratively performed to produce a plurality of different values, e.g., at different crop harvesting locations. It may not be necessary to repeat steps 410 and 420 to repeat step 530, but this can, of course, be optionally performed.

[0117] FIG. 7 conceptually illustrates an approach for performing step 530. FIG. 7 conceptually illustrates the co-ordinate system 700 in which the first contour 710 and second contour 720 is defined. [0118] Step 530 may be performed by identifying a first portion 715 of the first contour 710 and identifying a second portion 725 of the second contour 720. The first portion of the first contour wholly overlays the second portion of the second contour in a vertical direction within the co-ordinate system.

[0119] Subsequently, a vertical distance or average vertical distance within the coordinate system between the first portion of the first contour and the second portion of the second contour can be determined to predict a measure of a height hi of the swath.

[0120] This approach effectively provides a mechanism for predicting a (e.g., average) distance or height hi between the top of the swath, at a particular swath location or swath locations, and the top of the stubble or ground surface produced after harvesting crop (i.e., the position of the cutting bar during harvesting said crop) at the same said swath location or swath locations.

[0121] The height may, for instance, be used to generate a measure of cross-sectional area and/or volume of the swath and/or further processed to produce a measure of uniformity of the swath.

[0122] In preferred examples, the first portion 715 of the first contour 710 comprises more than one point of the first contour 710 and the second portion 725 of the second contour 720 comprises more than one point of the second contour. The vertical distance or average vertical distance may, in this case, therefore be the average vertical difference. This approach helps to reduce the impact or effect of noise on a determined measure of swath height.

[0123] In preferred examples, the first portion 715 of the first contour represents a portion of the top of swath produced by the harvesting machine within a time period of between 1 second and 10 minutes, e.g., between 5 seconds and 60 seconds.

[0124] It will be apparent that in the previously described approach, the measure of swath height is only determined or determinable after crop has been harvested.

[0125] The approaches illustrated by FIGS. 5 to 7 make use of one or more SLAM algorithms to process (first and optionally second) monocular image data in order to produce a cloud of points. Suitable SLAM algorithms that are able to process monocular image data in this way are known in the art.

[0126] By way of example, one suitable SLAM algorithm for use in processing monocular image data is set out by Campos, Carlos, et al. "Orb-slam3: An accurate open- source library for visual, visual-inertial, and multimap slam." IEEE Transactions on Robotics 37.6 (2021): 1874-1890. As an open-source function, the Orb-slam3 library is continually updated to reflect developments of the SLAM algorithm, any of which can be used in proposed embodiments.

[0127] Another suitable SLAM algorithm for use in processing monocular image data is set out by Davison, Andrew J. "Real-time simultaneous localisation and mapping with a single camera." Computer Vision, IEEE International Conference on. Vol. 3. IEEE Computer Society, 2003.

[0128] FIG. 8 conceptually illustrates a procedure or method 800 performed by a monocular SLAM algorithm. The method 800 is performed on a series or sequence of monocular images that forms monocular image data.

[0129] The method 800 comprises a feature detection step 810 comprising detecting, in each monocular image, the most significant features (e.g., in a region of interest). Suitable algorithms for detecting features within a monocular image are well established in the art, including SIFT (Scale Invariant Feature Transform), ORB, BRISK (Binary Robust Invariant Scalable Keypoints) or KAZE.

[0130] Various approaches for feature detection are set out by Tareen, Shaharyar Ahmed Khan, and Zahra Saleem. "A comparative analysis of sift, surf, kaze, akaze, orb, and brisk." 2018 International conference on computing, mathematics and engineering technologies (iCoMET). IEEE, 2018.

[0131] One approach for detecting significant features that could be used is suggested by Leutenegger, Stefan, Margarita Chli, and Roland Y. Siegwart. "BRISK: Binary robust invariant scalable keypoints." 2011 International conference on computer vision, leee, 2011. Another approach is suggested by Alcantarilla, Pablo Fernandez, Adrien Bartoli, and Andrew J. Davison. "KAZE features." Computer Vision-ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part VI 12. Springer Berlin Heidelberg, 2012. Yet another approach is suggested by Huang, Feng-Cheng, et al. "High- performance SIFT hardware accelerator for real-time image feature extraction." IEEE Transactions on Circuits and Systems for Video Technology 22.3 (2011): 340-351.

[0132] The method 800 also comprises a feature matching step 820 comprising matching features from image to image, resulting in M match pairs of matched features between each pair of adjacent images. The method 800 also comprises a localization problem solving step 830 that comprises estimating, using the known (from the distance information) physical distance between two adjacent images in each pair, the (3D) position of each matching feature in a co-ordinate system. These follow standard approaches used in SLAM algorithmic processing.

[0133] In this way, a cloud of points is constructed, each point representing a location of a feature that is identified or represented in at least two images of the sequence of monocular images.

[0134] With reference back to FIG. 4, although above-described embodiments make use of one or more monocular SLAM algorithms to process the (first) monocular image data in order to predict the value responsive to one or more dimensions of the swath, alternative approaches could be used.

[0135] By way of example, an alternative approach for performing step 420 is to make use of a machine-learning method. Thus, the step 420 of processing the first monocular image data may comprise processing the first monocular image data using a machine-learning method to predict the value responsive to one or more dimensions of the swath. Thus, the first monocular image data may be input to the machine-learning method, which outputs the predicted value responsive to one or more dimensions of the swath.

[0136] In preferred examples of this approach, step 420 comprises processing no other image data that contains a representation of the portion of the swath using the machine-learning method. However, other forms of image data, e.g., containing a representation of the stubble or ground surface after harvesting of crop, could be used as input to the machine-learning method in other examples.

[0137] Some embodiments therefore make use of one or more machine-learning algorithms. Any suitable machine-learning algorithm may be used in different embodiments for the present disclosure. Suitable machine-learning algorithms include (artificial) neural networks, support vector machines (SVMs), Naive Bayesian models and decision tree algorithms, although other appropriate examples will be apparent to the skilled person.

[0138] There are a number of well-established approaches for training a machinelearning algorithm. Typically, such training approaches make use of a large database of known input and output data. The machine-learning algorithm is modified until an error between predicted output data, obtained by processing the input data with the machine-learning algorithm, and the actual (known) output data is close to zero, i.e. until the predicted output data and the known output data converge. The value of this error is often defined by a cost function. The precise mechanism for modifying the machine-learning algorithm depends upon the type of model. Example approaches for use with a neural network include gradient descent, backpropagation algorithms and so on.

[0139] For use in the above-described approach, the known input data comprises example instances of monocular image data. The corresponding known output data comprises, for each example instance of monocular image data, a value responsive to one or more dimensions of the swath represented in the monocular image data. The value may be produced by an individual or device physically/manually obtaining and recording (e.g., using measuring tools) a measure of the value for the swath represented in the instance of monocular image data.

[0140] The skilled person will appreciate how different forms of known input and output data can be used dependent upon the function or purpose of the machine-learning algorithm.

[0141] FIG. 9 illustrates a computer-implemented method 900 of controlling one or more parameters of a combine harvester.

[0142] The computer-implemented method 900 comprises a process 400 of determining the value responsive to one or more dimensions of the swath (produced by the harvesting machine) by performing a previously described method.

[0143] The computer-implemented method 900 also comprises a step 910 of adjusting one or more parameters responsive to the determined value (produced by process 400). The one or more parameters comprises one or more of: a rotation speed of a threshing cylinder of the combine harvester; a distance between the threshing cylinder and a concave of the combine harvester; and/or a number of rotations performed by a harvested crop during threshing by the combine harvester before being ejected from the combine harvester. [0144] Approaches for controlling these parameters are readily apparent to the skilled person in the field of combine harvesting. For instance, the number of rotations performed by a harvested crop can be controlled by changing the position of a guide vane with respect to a longitudinally-aligned rotor of a threshing unit.

[0145] In particular, the parameter(s) may be controlled to control or define a level of damage to the material by the combine harvester making up the swath. It has been recognized that these parameters affect the level of damage to the material making up the swath. For instance, increased rotation speed causes increase damage, reduced distance between threshing cylinder and concave increases damage and an increased number of rotations also increases damage.

[0146] In some examples, step 910 comprises controlling the parameter(s) to maintain the value responsive to the one or more dimensions of the swath. For instance, a desired value may be defined and maintained with the parameter(s), using the value determined in process 400 as feedback.

[0147] Step 910 may be iteratively performed, e.g., once a minute, once an hour, once every 6 hours or once a day.

[0148] In some examples, the method 900 further comprises a step 920 of receiving an indication of grain quality and/or grain loss. This can be produced using any known technique, e.g., using a grain quality sensor or the like. A grain loss may relate to the amount of grain lost in the tailings discharged from the harvesting machine.

[0149] The step 910 may be adapted to compromise between swath damage and grain quality and/or loss, as reducing swath damage may reduce grain quality or increase grain loss (e.g., as the crop may be insufficiently threshed).

[0150] FIG. 10 illustrates a processing system 1000 according to an embodiment. The processing system is configured to perform any herein described method 400, 900.

[0151] The processing system 1000 may thereby obtain first monocular image data containing a representation of a portion of the swath produced by the harvesting machine, the first monocular image data being captured using a single camera mounted on the harvesting machine. The processing system may process (at least) the first monocular image data using one or more image processing algorithms to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data.

[0152] Of course, where relevant, the processing system may receive the first movement information, the second monocular image data and/or the second movement information.

[0153] The processing system may receive these values from a memory or storage unit 1010 and/or from the camera 1020.

[0154] The processing system 1000 may comprise an input interface 1001 configured to receive all of the above-identified data. [0155] The processing system 1000 is configured to process at least the first monocular image data using one or more image processing algorithms to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data. Procedures for performing this method/process have been previously described. This process may be carried out by a processing unit 1002 of the processing system 1000.

[0156] The processing system 1000 may be configured to output the predicted value responsive to one or more dimensions of the swath represented in the first monocular image data. Any output of the processing system may be provided via an output interface 1003. In particular, the output of the processing system may be defined by the processing unit 1002 of the processing system via the processing unit.

[0157] In some examples, the processing system 1000 is configured to control the operation or parameters of one or more features of a harvesting machine via control circuitry 1050. Thus, the processing system may provide a control signal to control circuitry 1050 for modifying one or more parameters of the harvesting device (not shown).

[0158] In some examples, the predicted value responsive to one or more dimensions of the swath is used to control a user interface, such as a display. This may be used, for instance, to provide at a user interface, a visual representation of the predicted value responsive to one or more dimensions of the swath.

[0159] In some examples, any data produced by the processing unit, e.g., the predicted value responsive to one or more dimensions of the swath, is stored in the memory 1010.

[0160] FIG. 11 illustrates an embodiment of the processing system 1000 described with reference to FIG. 10. The processing system is able to carry out or perform one or more embodiments of an invention, e.g. for predicting a value responsive to one or more dimensions of the swath.

[0161] The processing system 1000 comprises an input interface 1001 that receives communications from one or more inputting devices. Examples of suitable inputting devices include external memories, cameras and so on.

[0162] The processing system 1000 also comprises a processing unit 1002.

[0163] In one example, the processing unit 1002 may comprise an appropriately programmed or configured single-purpose processing device. Examples may include appropriately programmed field-programmable gate arrays or complex programmable logic devices.

[0164] As another example, the processing unit may comprise a general purpose processing system (e.g. a general purpose processor or microprocessor) that executes a computer program 1115 comprising code (e.g. instructions and/or software) carried by a memory 1110 of the processing system 1000.

[0165] The memory 1110 may be formed from any suitable volatile or non-volatile computer storage element, e.g. FLASH memory, RAM, DRAM, SRAM, EPROM, PROM, CD- ROM and so on. Suitable memory architectures and types are well known to the person skilled in the art.

[0166] The computer program 1115, e.g. the software, carried by the memory 1110 may include comprise a sequence of instructions that are executable by the processing unit for implementing logical functions to carry out the desired method or procedure. Each instruction may represent a different logical function, step or sub-step used in performing a method or process according to an embodiment. The computer-program may be formed from a set of sub-programs, as would be known to the skilled person. The computer program 1115 may be written in any suitable programming language that can be interpreted by the processing unit 1002 for executing the instructions. Suitable programming languages are well known to the skilled person.

[0167] The processing system 1000 also comprises an output interface 1003. The processing system may be configured to provide information via the output interface. In some examples, the processing system may be configured to control one or more other devices connected to the output interface 1003 by providing appropriate control signals to the one or more other devices. Suitable control examples include controlling a user-perceptible output such as a visual or audible representation at a user interface.

[0168] Different components of the processing system 1000 may interact or communicate with one another via one or more intra-system communication systems (not shown), which may include communication buses, wired interconnects, analogue electronics, wireless communication channels (e.g. the internet) and so on. Such intra-system communication systems would be well known to the skilled person. [0169] It is not essential for the processing system 1000 to be formed on a single device, e.g. a single computer. Rather, any of the system blocks (or parts of system blocks) of the illustrated processing system may be distributed across one or more computers.

[0170] The skilled person would be readily capable of developing a processing system for carrying out any herein described method. Thus, each step of the flow chart may represent a different action performed by a processing system, and may be performed by a respective module of the processing system.

[0171] It will be understood that disclosed methods are preferably computer- implemented methods. As such, there is also proposed the concept of a computer program comprising code means for implementing any described method when said program is run on a processing system, such as a computer. Thus, different portions, lines or blocks of code of a computer program according to an embodiment may be executed by a processing system or computer to perform any herein described method.

[0172] A computer program may be stored on a computer-readable medium, itself an embodiment of the invention. A "computer-readable medium" is any suitable mechanism or format that can store a program for later processing by a processing unit. The computer readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device. The computer- readable medium is preferably non-transitory.

[0173] In some alternative implementations, the functions noted in the block diagram(s) or flow chart(s) may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. [0174] Variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. If a computer program is discussed above, it may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. If the term "adapted to" is used in the claims or description, it is noted the term "adapted to" is intended to be equivalent to the term "configured to". If the term "arrangement" is used in the claims or description, it is noted the term "arrangement" is intended to be equivalent to the term "system", and vice versa.

Any reference signs in the claims should not be construed as limiting the scope.

[0175] All references cited herein are incorporated herein in their entireties. If there is a conflict between definitions herein and in an incorporated reference, the definition herein shall control.

Claims

CLAIMS What is claimed is:

1. A computer-implemented method of determining a value responsive to one or more dimensions of a swath produced by a harvesting machine, the computer- implemented method comprising: obtaining first monocular image data containing a representation of a portion of the swath produced by the harvesting machine, the first monocular image data being captured using a single camera mounted on the harvesting machine; and processing the first monocular image data using one or more image processing algorithms to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data.

2. The computer-implemented method of claim 1, further comprising obtaining second monocular image data containing a representation of stubble and/or ground surface remaining after harvesting of crop by the harvesting machine and/or a cutting bar of the harvesting machine, wherein processing the first monocular image data comprises processing the first monocular image data and the second monocular image data to predict the value responsive to one or more dimensions of the swath.

3. The computer-implemented method of claim 1 or 2, wherein: the first monocular image data comprises a first sequence of monocular images, each monocular image containing a representation of the portion of the swath produced by the harvesting machine; the computer-implemented method further comprises obtaining first movement information identifying a movement of the harvesting machine between capturing adjacent monocular images in the first sequence of monocular images; the step of processing the first monocular image data comprises: processing the first sequence of monocular images, and no other image data, together with the first movement information using a first monocular simultaneous localization and mapping, SLAM, algorithm to produce a plurality of first points, each identifying a predicted location of a portion of a top of the swath with respect to a co-ordinate system; and processing the plurality of first points to predict the value responsive to the one or more dimensions of the swath represented in the first monocular image data.

4. The computer-implemented method of claim 3, wherein the step of processing the plurality of first points comprises: fitting a contour to the first points to produce a first contour that represents the top of the swath; and processing the first contour to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data.

5. The computer-implemented method of any of claims 3 or 4, when dependent upon claim 2, wherein the second monocular image data comprises a second sequence of monocular images each containing a representation of stubble and/or ground surface remaining after harvesting of crop by the harvesting machine, and further comprising: obtaining second movement information identifying a movement of the harvesting machine between capturing adjacent monocular images in the second sequence of monocular images; and processing the second sequence of monocular image data, and no other image data, together with the distance information using a second monocular simultaneous localization and mapping, SLAM, algorithm to produce a plurality of second points, each identifying a predicted location of a portion of a top of the stubble and/or ground surface represented in the second sequence of monocular images, wherein the step of processing the plurality of first points comprises processing the plurality of first points and the plurality of second points to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data.

6. The computer-implemented method of claim 5, when dependent upon claim 4, wherein: wherein the step of processing the plurality of first points comprises fitting a contour to the second points to produce a second contour that represents the stubble and/or ground surface with respect to the co-ordinate system; and the step of processing the first contour comprises processing the first contour and the second contour to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data.

7. The computer-implemented method of claim 6, wherein the step of processing the first contour and the second contour comprises: identifying a first portion of the first contour; identifying a second portion of the second contour, the first portion of the first contour being located proximate to the second portion of the second contour within the coordinate system; and determining a vertical distance or average vertical distance within the coordinate system between the first portion of the first contour and the second portion of the second contour to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data.

8. The computer-implemented method of claim 7, wherein: the first portion of the first contour comprises more than one point of the first contour; and the second portion of the second contour comprises more than one point of the second contour.

9. The computer-implemented method of any of claims 5 to 8, wherein the first sequence of monocular images and the second sequence of monocular images are the same, such that each monocular image contains a representation of a portion of the swath produced by the harvesting machine and a portion of a top of the stubble and/or ground surface after harvesting of crop by the harvesting machine.

10. The computer-implemented method of claim 1, wherein the step of processing the monocular image data comprises processing the first monocular image data, and no other image data, using a machine-learning method to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data.

11. The computer-implemented method of any of claims 1 to 10, wherein the value is a measure of one of the one or more dimensions, the one or more dimensions optionally comprising a height of the swath and/or a volume of the swath.

12. A computer-implemented method of controlling one or more parameters of a combine harvester, the computer-implemented method comprising: determining the value responsive to one or more dimensions of a swath produced by the combine harvester by performing the method of any of claims 1 to 10; and adjusting, responsive to the determined value responsive to one or more dimensions of the swath, one or more of: a rotation speed of a threshing cylinder of the combine harvester; a distance between the threshing cylinder and a concave of the combine harvester; and/or a number of rotations performed by a harvested crop during threshing by the combine harvester before being ejected from the combine harvester.

13. A computer program product comprising computer program code means which, when executed on a computing device having a processing system, cause the processing system to perform all of the steps of the computer-implemented method according to any of claims 1 to 12.

14. A processing system configured to determine a value responsive to one or more dimensions of a swath produced by a harvesting machine, the processing system being configured to: obtain first monocular image data containing a representation of a portion of the swath produced by the harvesting machine, the first monocular image data being captured using a single camera mounted on the harvesting machine; and process the first monocular image data using one or more image processing algorithms to predict the value responsive to one or more dimensions of the swath represented in the first monocular image data.

15. An image processing system comprising the processing system of claim 14 and the single camera for capturing the first monocular image data.