AU2019200948A1 - Method and apparatus for estimating mowing area - Google Patents

Abstract

This invention relates to estimating the extent of a grass-covered area to be treated, such as an area of lawn for mowing. It provides a device that implements a method which employs artificial intelligence for identifying from a satellite image features consistent with features of lawns and other grass-covered areas suitable for mowing. An algorithm calculates the extent of an area having the identified features. SHEET 3 OF 3 . . . . . . . . 44,E I,5 - - - - Y ' ' .. .. . .. .. .. I I

Description

METHOD AND APPARATUS FOR ESTIMATING A GRASSED AREA

Field of invention [01] This invention relates to apparatus for and a method of estimating the extent of a grassed area to be treated, such as an area of lawn for mowing.

Background to the invention [02] US granted patent no. 8126819, to Ersek, uses previously gathered images of a suburban plot of land, for use in reckoning mowing or other treatment services. Customers are required to post their estimate inquiry on a website associated with the service provider. Using current GPS satellite imaging aerial photographs of area lot sizes and customer demographic data stored in the lawn care service provider's database, a near real time estimate is provided for the estimate inquirer.

[03] Ersek's invention requires new customers to enter their demographic information and lot sizes into the lawn care service provider's system and obtain an estimate for the lawn care services by return. If the customer address is not in the lot size database (not shown) of the lawn care service provider but it is in a zip coded area that it services, then the customer will be so notified and requested to provide their own estimate for their yard size. Using that information, the service provider's system will provide an estimate for the type of lawn service that is requested.

[04] A pending application to Matthews, US20140229213-A1, mentions mowing services as an example, but requires a user to enter the lengths of two sides only, for determining an area to be mown. From these limited data, assumptions about the remainder are made.

[05] Both above systems can lead to considerable inaccuracies and the likelihood of a price being under-quoted or over-quoted, benefiting either the customer or the service provider unfairly.

-22019200948 11 Feb 2019 [06] US patent 6532672 (Gottlieb) discloses a wheeled area measurement device with direction recording means and software to calculate the area of a zone traversed. It provides an example of a tool available to measure area by a person on site walking the area to be calculated.

[07] US published patent application no. 2003/0220734 (Harrison) discloses that Ά cursor drawing tool allows the Internet user to draw and edit boundaries on [a] map and image data by simply clicking the cursor on the corner points of the boundary. An acreage calculation tool is also provided that calculates the acreage of an enclosed boundary.' There is no suggestion that the tool may be applied for use in administering a treatment to the area, let alone the specialised treatment involved in the art of grass mowing. It also does not teach that the person requesting a mowing service may enter the data points defining the area to be mown. This appears to be a responsibility too burdensome to be placed on a customer.

[08] Chinese published utility model no. CN 2014634854 teaches that although GPS coordinates gathered from farm machinery can be used for calculating extents of areas in a farming environment, this level of technology is not suitable for use by peasants. A handheld device is proposed, which includes a processing module, GPS receiver module and a display module. The processing module is used for calculating the farmland working area according to the coordinate information received, and then calculating corresponding operating costs according to the area size and predetermined billing parameter/s. The device is claimed suitable for the precision measurement of various shape farmlands, greenery patches, forest, waters, mud flats and irregular areas. The GPS coordinates are entered at discrete points by the user, when moving around the area to be measured. The device is a standalone and dedicated single application piece of equipment. There is no suggestion it would be suited for a use outside the farming environment. Again, it requires a person to carry out the collection of data by physically attending on site and walking about the area of interest, while operating the device.

Objects of the invention [09] It is an object of this invention to address the shortcomings of the prior art and, in doing so, to provide a more accurate estimating system that does not rely on customer estimates of the extent of a grassed area to be mown.

2019200948 11 Feb 2019

-3 [010] The preceding discussion of the background to the invention is intended to facilitate an understanding of the present invention. However, it should be appreciated that the discussion is not an acknowledgement or admission that any of the material referred to was part of the common general knowledge in Australia or elsewhere as at the priority date of the present application.

[Oil] Further, and unless the context clearly requires otherwise, throughout the description and the claims, the words 'comprise', 'comprising', and the like are to be construed in an inclusive sense - that is to say, in the sense of including, but not being limited to - as opposed to an exclusive or exhaustive sense, meaning including this and nothing else.

Summary of invention [012] According to a first aspect of the invention, there is provided a method of estimating size of a grassed area to be mown comprising the steps of:

a. Providing an aerial image of a locality that includes an address at which a grassed area to be mown is located,

b. Interpreting the image thereby to identify a perimeter of the grassed area, and

c. Calculating the area enclosed by the perimeter, whereby, from the area calculated, a price is determinable for providing a mowing service in respect of the area.

[013] In preferred form of the invention, the step of interpreting the image comprises comparing the image it with known images of related localities.

[014] In a preferred embodiment of the invention, the known images include colour images of grassed areas near the locality of the required mowing.

-42019200948 11 Feb 2019 [015] In a further preferred form of the invention, the method comprises the step of applying a monetary rate to the area calculated and generating a mowing price being the product of rate and calculated area.

[016] In a still further preferred form of the invention, the method includes adjusting the price on the basis of elevation data pertaining to the area. The elevation data may include contour elevations or colour coded altitude bands plotted on a map.

[017] In an embodiment, the method includes calculating a slope of the area and applying a monetary rate according to slope severity.

[018] Preferably, the method includes providing a computer processor that executes the app for determining geo-coordinates of points located on the identified perimeter, and uses said coordinates in calculating the extent of a grassed area bounded by the perimeter.

[019] The compared known image may be a satellite image, an aerial photograph or a map, or a combination of two or more thereof.

[020] In an embodiment, the perimeter is identified by constructing a polyline between pairs of adjacent peripheral points disposed on an interface between adjacent zones of different colours.

[021] In an embodiment, different colours are applied to differentiate areas having different vegetation.

[022] The method preferably further includes operating a programmed computer processor to calculate automatically the grassed area from geo-coordinates representative of points on the identified perimeter.

[023] The method extends to the step of applying a monetary rate to the area calculated and generating a mowing price, the price being the product of rate and calculated area.

2019200948 11 Feb 2019

-5 [024] According to a second aspect of the invention, there is provided a mowing estimate-generating device comprising executable instruction code comprising a mowing area-calculating algorithm, and a smartphone on which the code is executable, the instruction code, when running the algorithm, causing the smartphone to:

a. capture, from an aerial image of a locality within which the mowing area is located, location coordinates defining a grassed area to be subjected to mowing;

b. to execute said algorithm to generate a size estimate of the grassed area, based on the coordinates; and

c. to apply a monetary rate to the size estimate to generate a price for mowing the grassed area.

[025] In a preferred form of the invention, the locality for mowing is identified using an inputted address.

[026] In a further preferred form of the invention, the algorithm causes the smartphone to generate the size estimate by interpreting the image for detecting and recognizing the grassed area.

[027] Preferably, the algorithm causes the smartphone to interpret the image data by comparing it with a locality image selected from images of related localities known to contain a grassed area.

[028] The related locality may be related by geographical proximity to the locality of the grassed area to be mown. The related locality may be related on the basis of climatic similarity to the locality of the grassed area to be mown or it may be related by topographical similarity, or a combination of any of the aforesaid.

[029] Preferably, the instruction code comprises instructions for downloading the monetary rate from a remote server.

2019200948 11 Feb 2019

-6 [030] In a preferred embodiment, the images of related localities known to contain a grassed area include colour images of grassed areas.

[031] In a still further preferred form of the invention, the monetary rate is adjustable.

[032] Further preferably, the monetary rate is adjustable according to slope of the grassed area. Slope may be calculated from topographical data of the locality image of the grassed area requiring mowing.

[033] In an embodiment of the invention, the instruction code comprises instructionsfor allowing the user to select a portion of the area to be excluded from within the area marked to be mown. The instruction code may comprise instructions to apply a second monetary rate for calculating a price for service to be provided in relation to the excluded portion. The second rate may differ from the rate to be applied to the area minus the excluded portion.

Brief description of drawings [034] In order that the invention may be readily understood, and put into practical effect, reference will now be made to the accompanying figures. Thus:

Figure 1 shows in schematic plan view a diagram of a yard to be mown according to a preferred embodiment of the invention.

Figure 2 is a schematic diagram of a variation of the yard of Figure 1, showing portions to be excluded from the quote for mowing services, according to a further embodiment of the invention.

Figure 3 is a schematic process diagram of the method of the invention according to a further preferred embodiment.

Detailed description of an embodiment of the invention [035] In this invention, there are proposed methods of capturing the extent of a grassed area within a locality, for purposes of determining a quantity associated with

-7 2019200948 11 Feb 2019 rendering a service in respect of the grassed area. Specifically, the quantity determined is used for quoting the cost of mowing a defined grassed area, such as a lawn.

[036] An executable software application, or 'app', when run on a mobile computer device such as a smartphone of a user desirous of obtaining a price for a mowing service, prompts the user to supply location geo-coordinates of the locality where the service is required, or a street address thereof. Artificial intelligence features in the application interface with satellite imaging to detect, recognize and capture grassed areas to be mown, using address data supplied by a customer.

[037] The app obtains image data of the locality specified to recognize a grassed area thereat to be mown. Recognition is assisted by comparing features in the image data with features in images of grassed areas in related localities. The localities may be related by geographic proximity, topographical feature similarity, climatic similarity, vegetation similarity, colour similarity, and the like.

[038] The method of operation is explained with reference to Figure 1 in which a location generally indicated by means of the numeral 10 is to be measured according to the invention. The location is defined by a boundary line 12, within which is a house 14 surrounded by a paved area 16 and occasional shrubbery 18. The remainder of the bounded locality 10 is made up of a lawn 20, shown in two parts 20A and 20B.

[039] The invention is implemented by a user downloading the application (app) supplied by a lawn service provider or a service platform with which the provider is associated. This may be accomplished and organised in various different ways prevalent in commerce, using a job portal to introduce suppliers to customers. The user may be desirous of having his lawn mown and may already have approached a service provider which has introduced him to the app and directed him to download it to enable business to proceed, or he may be seeking a service provider and may have downloaded the app to estimate his lawn area before submitting the measurements to a service provider interactively listed on, or proposed by, the app.

-8 2019200948 11 Feb 2019 [040] It will be assumed for this embodiment that the app user is the owner or lessor occupying bounded location 10 and is desirous of engaging a lawn mowing service provider. It will be appreciated that the user may be an agent of the occupier, or even a servant of the service provider, summoned to the location to provide an on-site measurement.

[041] The user opens the app for loading it to the device processor and receives an on-screen prompt to enter to identify the area 20 to be mown. The user may do this by entering a street address or, if he knows them, a geo-spatial coordinate pairing from the locality. In response, the processor retrieves and displays an aerial image in the form of a satellite image of the locality in which the address or point coordinate pairing is contained. The image may be obtained from a service such as one known under the brand Google Maps or Apple Maps. The user may be offered the option of zooming the image in for small features to become identifiable.

[042] Location marker 22 in Figure 1 shows a first point on a perimeter 26 of an area 20A, 20B requiring a mowing service. The smartphone displays the retrieved locality image to the user and invites the user to confirm this as the locality of interest in generating a mowing quote. The prompt will also request the user whether there is a further area to be added, or whether there is an area to be excluded, for example if there is a non-grassed area such as a building 14, paving 16, rockery, shade tree 18, swimming pool, or fishpond. The user may respond by tapping a button to indicate nothing more to add, or a second button to add or subtract an area. In the latter case, the process will be repeated, with the user being invited to move the onscreen cursor to a new virtual starting point, and so on.

[043] The app runs an algorithm to propose a perimeter around various points and features identified on the image as defining the grassed area. It does this by having the algorithm perform comparisons of features in the obtained image with features found on images of comparable localities, identifying the likely grassed area and its perimeter. The perimeter thus defined is made up of a connected sequence of line segments, referred to as a 'polyline'.

2019200948 11 Feb 2019

-9 [044] Once the area to be mown has been defined by its perimeter 26, an algorithm is applied for calculating the area thereby enclosed. The algorithm may employ geometrical formulae or be structured to divide an irregularly shaped grassed area into two or more areas of recognized geometrical shape. Hence in the case of lawn 20a,20B in Figure 1, the lawn perimeter may first be smoothed into a series of linked straight lines. Thereafter, the app causes the processor to recognise the area complete 20 as two separable rectangles 20A and 20B, divided by broken line 28 (for example).

[045] In more irregularly shaped lawns, such as those with curved boundaries, the algorithm may employ integration or similar calculus-based techniques, or a combination of techniques involving dividing the lawn into finite elements of known area and adding the areas of the elements to arrive at an aggregate representing a close approximation of the area of the whole lawn. As indicated previously, the areas of non-mowable zones within the main perimeter 26 may be similarly calculated and then subtracted from the whole to arrive at the total mowable area within boundary 12.

[046] When run on a PC, smartphone or tablet device associated with the servicerequesting user, the app utilises downloaded or streamed satellite imagery to display a user-defined locality, as identified from the user's inputted street address or location coordinates.

[047] Figure 2 illustrates the implementation of the method in the backyard layout of Figure 1, as shown on a touchscreen display 40. System prompts appear in a message zone 42. Again, let the numeral 22 signify a point identified in the aerial image as being on the perimeter of the locality shown in the touchscreen display.

[048] Based on image recognition, the areas marked 44, 46, 48 50 and 52 are identified as corner features of the zone defined by the perimeter including point 22. The prompt zone displays a query which asks the user whether the definition of the external perimeter of the grassed area (lawn) has been completed.

-102019200948 11 Feb 2019 [049] If the user responds that the lawn external perimeter definition is complete, the algorithm of the program works further on the assumption that the area to be mown is within the perimeter-bounded area.

[050] The algorithm also causes the processor to seek features within the perimeter that are recognizable as unmowable areas, and thus to be excluded from mowing and from the price calculation. Such features may for example be a flower bed, rose garden, vegetable patch, pathway or a rockery. These are distinguishable according to features such as colour and shape, when compared with the expected shade of a grassed area. The shade may vary according to season.

[051] The enclosed area within the polylinear perimeter defines the main area to be mowed/quoted. If there is an area within the polyline perimeter to be excluded from the quote, for example feature 56 in Figure 2, representing a fishpond, the user is prompted by means of a menu display to confirm that it is an area to be excluded. The area extent of excluded area 56 is then subtracted from the area calculated earlier and the difference is multiplied by a cost rate per unit area to arrive at a monetary proposal.

[052] Further variables may be requested from the user, for inputting to the processor before the quote is issued. By way of non-limiting example, the user may define the length or shortness of the cut, or the frequency of required mowing, such as weekly, fortnightly or at intervals of a specified number of days. Further parameters may also be set - such as a date range. Concurrent treatments such as fertilizer spreading and topsoil distribution may be included as requirements, using the same area measurements.

[053] Further, in the embodiments above, there may be included altitude data associated with the recorded or captured coordinates, so that the slope of the terrain to be mowed is calculable. With this data, the true surface area to be mown is more accurately ascertainable. The service provider is also in a position to adjust the rated price according to the steepness of the slope of the ground to be mown, as this influences the fuel consumption and time to be allocated to the job in question.

-112019200948 11 Feb 2019 [054] With reference to Figure 3, the invention extends to the use of application software to cause a processor to interpret image data obtained of a general locality at which a mowing task is required. In this embodiment, the user submits at step 102 a request for a quotation (RFQ), providing the address of the premises where the service is required.

[055] A system server uses the address to stream or download an aerial view or satellite image (104) of the locality to the user device. A supplier of suitable images is known under the trade name Google Maps. The image shows the property in the context of its surrounds, for example at a suburban block level. The number of blocks shown would depend on the size of the blocks. Generally, the image should be selected to cover about 4km². The size should suit the context of the address and should show a number of surrounding properties and their lawns.

[056] At 106, the processor executes instructions provided by the app and makes a prediction as to the perimeter of the lawn area to be mown. This will be put to the customer in a proposal, including a quoted price for the service to be provided. To arrive at this outcome, it performs a programmed algorithm to analyse the aerial image. In doing so, it detects colours, patterns and inconsistencies 108. The algorithm zooms in digitally on the address given in the RFQ to magnify and seek out features, testing them against images contained in an image library 110. The image library contains images corresponding to objects such as fences, roads, trees, paths, buildings and dams, etc. From comparing images of surrounding environs, it determines probable characteristics of the property being quoted on and determines a proposed perimeter of the lawn to be mowed. The proposed perimeter would include internal perimeters determined to surround identified obstacles such as pond 56 (see Figure 2) in the overall expanse of lawn 114.

[057] The system software also causes the processor to perform colour recognition in respect of lawns in the vicinity of the premises 112. The programming instructions cause colours within the predicted perimeter to be recorded and differences noted. A steep colour gradient, exceeding a predefined threshold, indicates a change in terrain. By

-122019200948 11 Feb 2019 applying logic further, the program causes the processor to identify the colour of the lawn of the locality, by determining the predominant colour of the designated greater area, and colour-matching it to colours of images from other known or recognized areas of lawn in the vicinity. Images of these comparable lawns and of the premises in question would preferably have been recorded on the same recording occasion to eliminate differences that may have developed overtime.

[058] An example of a system of mapping and imaging that is suitable for use in the present invention is the 'Google Earth' system. Further assumptions that may be applied in performing the image comparison include:

a. That the perimeter of the outermost area (when there are two or more areas) is immediately adjacent the grassed area to be mown.

b. That the grassed area corresponds predominantly to a green shade (if the shade is too yellow or brown, then probably insufficient grass exists for mowing).

[059] From its calculations, the processor provides the perimeter to be proposed for mowing, including exclusion zones, and superimposes these on an enlarged aerial image of the premises 116. It causes the combined superimposed image to be displayed to the user and prompts the user to approve the area demarcated (see step 118), using zone 42 in screenshot 40 of Figure 2.

[060] Slope data is preferably included in the data characterizing the locality to be mown. As discussed previously, the altitude profile of the area to be mown has influence on numerous variables that should be taken into account in preparing a quote for a customer.

[061] Any suitable device may be employed by the user for downloading and executing the app used in the invention, but particularly convenient are touchscreen enabled mobile computing devices. This is not to exclude the use of a device having an optical tracking device such as a mouse for identifying locations on the display.

-132019200948 11 Feb 2019 [062] When the image of his property is displayed to him 118, the user is prompted to approve the representation of the lawn area perimeter as identified, or to adjust the mowable area by tapping or clicking (as the case requires) and dragging on points of the perimeter, such as corners. The user will also approve exclusion areas as previously described, as a part of the requirements of this step 120.

[063] When the user has signalled satisfaction with the representation of the mowing area and the price, he will be invited to accept a quote 122 for a price calculated on the basis of the area within the approved perimeter. Finally, at step 124, the user signals his acceptance or rejection of the quote and this is communicated via the app on the user device to the server of the service provider.

[064] The present invention thus provides quoting tools directly to the end-user. In use, these tools provide instant pricing output, without need for manual intervention by or review from the service provider or his agent.

[065] These embodiments merely illustrate specific examples of the method of the invention and related programmed device for the determination of a grassed area to be mown. With the insight gained from this disclosure, the person skilled in the art is well placed to discern further embodiments by means of which to put the claimed invention into practice.

Claims (15)

1. A method of estimating size of a grassed area to be mown comprising the steps of:

a. Obtaining a satellite image of a locality within which a portion comprises a grassed area to be mown, and

b. Inputting the image in digital form into a computer running a software application that

i. processes the captured image for detecting terrestrial features therein, applying artificial intelligence, whereby detected features that are consistent with machine-learned features of a grassed area are identified;

ii. determines the area within the portion that is covered by grass, iii. acquires spatial coordinates of the grass-covered area within the portion, and iv. calculates the area of said grass-covered area.

2. A method according to claim 1 wherein the step of applying artificial intelligence comprises comparing detected features in the image with stored images of features of grassed areas.

3. A method according to claim 2, wherein the stored images include colour images of grassed areas in or adjacent the locality of the portion containing the grass-covered area.

4. A method according to any one of claims 1 to 3 further comprising applying a monetary rate to the area calculated and generating a mowing price being the product of rate and calculated grass-covered area.

5. A method according to claim 4 comprising the step of adjusting the price on the basis of elevation contours pertaining to the grass-covered area.

2019200948 11 Feb 2019

6. A mowing estimate-generating device comprising executable instruction code comprising:

a. an area-calculating algorithm and

b. artificial intelligence features applicable to the processing of a satellite image, the instruction code, when running the algorithm, causing the smartphone to:

acquire a digital satellite image of a locality within which a portion comprises a grassed area to be mown, process the image for detecting terrestrial features therein, apply artificial intelligence whereby detected features that are consistent with machine-learned features of a grassed area are identified;

determine the area within the portion that is covered by grass, acquire spatial coordinates of the grass-covered area within the portion, and calculates the area of said grass-covered area.

7. A device according to claim 6, wherein the artificial intelligence, when applied, causes comparison of detected features in the image with stored images of features of grassed areas.

8. A device according to claim 7, wherein the stored images include colour images of grassed areas in or adjacent the locality of the portion containing the grass-covered area.

9. A device according to claim 7 or claim 8, wherein the algorithm causes the smartphone to apply a monetary rate to the area calculated for the grass-covered area and to generate a mowing price being the product of rate and the calculated grass-covered area.

2019200948 11 Feb 2019

10. A device according to claim 9, wherein the algorithm causes the smartphone to adjust the monetary rate on the basis of elevation contours pertaining to the grass-covered area.

11. A device according to claim 10, wherein the monetary rate is adjustable according to the slope of the grass-covered area.

12. A device according to any one of claims 6 to 11, wherein features that are consistent with machine-learned features of a grassed area are identified by comparing them with features selected from images of related localities known to contain a grassed area.

13. A device according to claim 12, wherein the related locality is related

a. by geographical proximity,

b. by climatic similarity,

c. by topographical similarity, or

d. a combination of any of the aforesaid to the locality containing the portion in which the grassed area to be mown is located.

14. A device according to any one of claims 6 to 13, wherein the instruction code comprises instructions for allowing a user of the device to select a section within the determined grass-covered area to be excluded from the estimate.

15. A device according to claim 14, wherein the instruction code comprises instructions to apply a second monetary rate to the section for exclusion, for calculating a price for service to be provided in relation to the excluded portion.