WO2019180255A1

WO2019180255A1 - Ware identification and handling apparatus

Info

Publication number: WO2019180255A1
Application number: PCT/EP2019/057336
Authority: WO
Inventors: Douglas Geoffrey YOUNG; James Edward BUCKLEY; Henry Matthew Lawrence FLETCHER; Christopher John Roberts
Original assignee: Elior Group
Priority date: 2018-03-22
Filing date: 2019-03-22
Publication date: 2019-09-26
Also published as: GB201804637D0; GB2572199A

Abstract

A warewasher system for identifying and handling wares is disclosed. The system comprises a first apparatus for identifying and locating a ware to be transported from a first location to a second location in the warewasher system, and a second apparatus for transporting the ware. The first apparatus comprises means for identifying a region of an image that contains an image part corresponding to at least a part of the ware. The identified region has a perimeter that does not coincide with the perimeter of the image part. The first apparatus also comprises means for determining a location of the region, within the image, and for generating information representing a corresponding spatial location in a receiving area. The second apparatus comprises means for receiving the generated information, and a resilient handling tool for transporting the ware, even when the estimated spatial location is not coincident with the actual spatial location.

Description

Ware Identification and Handling Apparatus

The present invention relates to a system and apparatus for identifying and handling wares for a warewashing system and has particular, although not exclusive, relevance for use as a system and apparatus for identifying objects, such as crockery or glassware, and releasably attaching to the objects for transport from one location to another.

In large kitchens such as those found in hotels or hospitals there is a need to rapidly clean very large quantities of dining ware. In these high-volume environments, it is prohibitively time consuming to wash the objects by hand, and so the process is typically automated using a warewasher system.

Warewasher systems process large numbers of objects, such as crockery or glassware, to clean the objects of debris such as food waste. In order to increase the efficiency of the cleaning process, the objects that enter the machine are often separated into different types. For example, crockery and glassware may be separated from dining trays and other objects. In low-capacity warewasher systems, objects may be manually separated by the user when placing the objects into the machine. High capacity warewasher systems on the other hand are highly complex machines able to process hundreds to thousands of objects per hour, and may separate the objects automatically before the cleaning process begins. For example, objects may be automatically routed into different parts or lanes of the warewasher using gating mechanisms or other similar switching devices. However, these types of separation systems often require large amounts of space in which to operate, which is a particularly undesirable characteristic when the warewasher is installed in a kitchen environment where the available space is limited.

When the separation process is performed automatically by the warewasher, there is a need to selectively transport wares, such as crockery or glassware, from one area of the warewasher to another relatively swiftly. The surface of a ware that enters a warewasher is typically contaminated by a range of different types of debris, such as food waste, liquids, cutlery, napkins, yoghurt pots, drinks cans, receipts or other inorganic waste, which makes automatic handling of the ware challenging. This challenge is made more complex by the need for the warewasher system to be compact, and to be able to process large numbers of objects as rapidly and reliably as possible. Currently, there are a number of grippers available that are able to releasably attach to wares in a warewasher for the purposes of such transport. However, these grippers are often unable to reliably transport wares that are contaminated by large amounts of food waste, or that are partially obstructed by items such as cutlery. This can result in an increased risk of a ware becoming detached from a gripper, which may result in damage to the ware or machinery, and possibly require interruption of the cleaning process to retrieve wares that have been dropped. It can be seen that this may reduce the throughput of the system undesirably.

The currently available grippers typically make use of precise positional alignment with respect to the ware that is being transported for a secure attachment to be made. However, these systems are often unable to process wares at the high volumes required in a warewasher system. These grippers may also require an impractical amount of space in which to operate and are often prohibitively expensive.

The use of high precision grippers to selectively transport wares imposes a constraint that the warewasher system must be capable of accurately identifying and classifying an item that is to be transported, and capable of determining a precise position for the identified and classified item. However, this process can be undesirably slow as there is a trade-off between identification and classification accuracy, positioning precision, and speed. Thus, in order to achieve the level of precision imposed by typical handling systems the throughput in the warewasher system has to be compromised.

The present invention seeks to provide a warewasher system, apparatus for use in a warewasher system, associated methods, and a program for addressing or at least partially ameliorating one or more of the above issues.

In a first aspect the invention provides a warewasher system for identifying and handling wares, the system comprising: first apparatus for identifying and locating a ware to be transported from a first location to a second location in the warewasher system; and second apparatus for transporting a ware identified and located by the first apparatus from the first location to the second location; wherein the first apparatus comprises: a receiving area for receiving objects including wares to be transported; means for capturing an image of at least part of the receiving area, the image including an image part corresponding to at least part of a ware to be transported that has been received in said receiving area; means for identifying, from the captured image, the ware to be transported, and for identifying a region of the captured image that contains the image part, wherein the region has a perimeter that does not coincide with a perimeter of the image part for at least a major part the perimeter of the region; means for determining a location of the region, within the image, and for generating information representing a spatial location in the receiving area corresponding to the determined location within the image; and means for outputting the generated information representing the spatial location for use by the second apparatus as an estimated spatial location of the identified ware; wherein the second apparatus comprises: means for receiving the generated information representing the spatial location; a handling tool for engaging with, and releasably securing to, the identified ware for transporting the ware from a first location to a second location; and means for moving the handling tool to transport the ware from the first location to the second location; wherein the moving means is configured to move the handling tool, responsive to receiving the generated information, to the estimated spatial location of the identified ware; and wherein the handling tool is configured to engage with and releasably secure to the identified ware at the estimated spatial location even when the estimated spatial location deviates, as a result of the identified region having a perimeter that does not coincide with the perimeter of the image part, from a corresponding actual spatial location of the ware.

In a second aspect the invention provides apparatus for identifying and locating a ware to be transported from a first location to a second location in the warewasher system of the first aspect of the invention, the apparatus comprising: the receiving area for receiving objects including wares to be transported; the means for capturing an image of at least part of the receiving area, the image including an image part corresponding to at least part of a ware to be transported that has been received in said receiving area; the means for identifying, from the captured image, the ware to be transported, and for identifying a region of the captured image that contains the image part, wherein the region has a perimeter that does not coincide with a perimeter of the image part for at least a major part the perimeter of the region; the means for determining a location of the region, within the image, and for generating information representing a spatial location in the receiving area corresponding to the determined location within the image; and the means for outputting the generated information representing the spatial location for use by the second apparatus as an estimated spatial location of the identified ware. The means for determining may be configured to determine, for at least some identified wares, as a result of the identified region having a perimeter that does not coincide with the perimeter of the image part, a location of the region that deviates, in a given direction, from a corresponding actual spatial location of an identified ware by at least 5% (or preferably at least 10%, 15% or 25%) of a width of the identified ware in the given direction.

The means for determining may be configured for determining the determined location of the region to be a geometric centre of the region, and for generating information representing a spatial location in the receiving area corresponding to the geometric centre for use as an estimated geometric centre of the identified ware.

The means for identifying may be implemented using a neural network. The means for determining a location may be implemented as part of said neural network.

The neural network may be a neural network that has been pre-trained using at least one object other than wares of a type that are washed in the warewasher system, and the neural network may be adapted for use with wares of the type washed in a warewasher system, using transfer learning.

The neural network may be a convolutional neural network (e.g. a‘y°^u only look once’ convolutional neural network).

The apparatus may be configured to capture and process images at a rate greater than one image per second.

The means for identifying may be configured to identify as said region of the captured image that contains the image part, a region that has a shape that is different to a shape of the ware (or part thereof) as captured in said image part.

The means for identifying may be configured to identify as said region of the captured image that contains the image part, a region that has a geometric centre that is not coincident with a geometric centre of the ware (or part thereof) as captured in said image part.

The means for identifying may be configured to identify as said region of the captured image that contains the image part, a region that has an area that is at least 10% larger than an area of the image part corresponding to at least part of a ware. The means for identifying may be configured to identify as said region of the captured image that contains the image part, a region that has a shape that is polygonal. The region may have shape is a quadrilateral (e.g. a rectangle or square).

The apparatus may further comprise means for processing the captured image to at least one of: resize the image, alter the contrast of the image, or sharpen the image; and means for outputting the processed image for use by the means for identifying the ware to be transported and the means for determining the location of the region.

The ware to be transported may be an item of crockery or glassware.

In a third aspect the invention provides a handling tool for transporting a ware from a first location to a second location in the warewasher system of the first aspect of the invention, the handling tool comprising means for engaging with, and releasably securing to, the identified ware for transporting the ware from the first location to the second location, wherein the means for engaging with, and releasably securing to, the identified ware are configured to engage with and secure the identified ware when the estimated spatial location deviates from the actual spatial location, as a result of the identified region having a perimeter that does not coincide with the perimeter of the image part for at least a major part the perimeter of the region.

The means for engaging with, and releasably securing to, the identified ware may comprise a resilient ware engaging surface and means for coupling to a vacuum source for applying at least a partial vacuum between the ware engaging surface and a surface of the identified ware to releasably secure the identified ware to the handling tool.

In a fourth aspect the invention provides a method of identifying and handling wares in the warewasher system of the first aspect of the invention, the method comprising: capturing an image of at least part of the receiving area, the image including an image part corresponding to at least part of a ware to be transported that has been received in said receiving area; identifying, from the captured image, the ware to be transported, and for identifying a region of the captured image that contains the image part, wherein the region has a perimeter that does not coincide with a perimeter of the image part for at least a major part the perimeter of the region; determining a location of the region, within the image, and for generating information representing a spatial location in the receiving area corresponding to the determined location within the image; and outputting the generated information representing the spatial location for use by ware handling apparatus as an estimated spatial location of the identified ware; receiving, in the ware handling apparatus, the generated information representing the spatial location; using a handling tool to engage with, and releasably secure to, the identified ware for transporting the ware from a first location to a second location; and moving the handling tool to transport the ware from the first location to the second location; wherein the handling tool is moved, responsive to receiving the generated information, to the estimated spatial location of the identified ware; and wherein the handling tool engages with and releasably secures to the identified ware at the estimated spatial location even when the estimated spatial location deviates, as a result of the identified region having a perimeter that does not coincide with the perimeter of the image part, from a corresponding actual spatial location of the ware.

In a fifth aspect the invention provides a method for identifying and locating a ware to be transported from a first location to a second location in a warewasher system, the method comprising: receiving objects, in a receiving area, including wares to be transported; capturing an image of at least part of the receiving area, the image including an image part corresponding to at least part of a ware to be transported that has been received in said receiving area; identifying, from the captured image, the ware to be transported, and identifying a region of the captured image that contains the image part, wherein the region has a perimeter that does not coincide with a perimeter of the image part for at least a major part the perimeter of the region; determining a location of the region, within the image, and for generating information representing a spatial location in the receiving area corresponding to the determined location within the image; and outputting the generated information representing the spatial location for use by ware handling apparatus as an estimated spatial location of the identified ware.

In a sixth aspect the invention provides a computer implementable program product causing a programmable apparatus to perform the method of the fourth aspect of the invention or the fifth aspect of the invention.

Aspects of the invention extend to computer program products such as computer readable storage media having instructions stored thereon which are operable to program a programmable processor to carry out a method as described in the aspects and possibilities set out above or recited in the claims and/or to program a suitably adapted computer to provide the apparatus recited in any of the claims. Each feature disclosed in this specification (which term includes the claims) and/or shown in the drawings may be incorporated in the invention independently (or in combination with) any other disclosed and/or illustrated features. In particular but without limitation the features of any of the claims dependent from a particular independent claim may be introduced into that independent claim in any combination or individually.

Embodiments of the invention will now be described by way of example only with reference to the attached figures in which:

Figure 1a shows a simplified schematic diagram of an area of a warewasher system;

Figure 1 b shows a simplified schematic diagram of an apparatus for separating wares in a warewasher system;

Figure 2 shows a simplified block diagram of the control unit of Figure 1 ;

Figure 3a shows a simplified schematic diagram of an area in which an item is identified;

Figure 3b shows a simplified schematic diagram showing an area identified for an item;

Figure 4 shows a 3D view of a handling tool;

Figure 5 schematically illustrates a cross section of the handling tool of Figure 4;

Figure 6 schematically illustrates the handling tool of Figure 4 attached to an item of crockery;

Figure 7 shows a top-down view and a 3D view of a variation of the handling tool of Figure 4;

Figure 8 shows a cross section of another variation of the handling tool of Figure 4; Figure 9 shows a 3D view of a second handling tool;

Figure 10 schematically illustrates a cross section of the handling tool of Figure 9;

Figure 1 1a schematically illustrates the handling tool of Figure 9 attached to an item of glassware in a first orientation; Figure 1 1 b schematically illustrates the handling tool of Figure 9 attached to an item of glassware in a second orientation;

Figure 1 1c schematically illustrates the handling tool of Figure 9 attached to an item of glassware in a third orientation;

Figure 12 shows a cross-section view of the handling tool of Figure 9 attached to an item of glassware contaminated by a liquid and an item of cutlery;

Figure 13a shows a cross-sectional view of a variation of the handling tool of Figure 9; and

Figure 13b shows a simplified 3D view of the variation of the handling tool of Figure 9.

Overview

Figure 1a shows, for illustrative purposes, a simplified schematic diagram of an area of a warewasher system generally at 1. The system comprises a ware separation area comprising a first conveyor 3 for transporting, in the direction indicated by arrow A, items (‘wares’) through an automated ware separation apparatus 5 and a warewasher transport area comprising a second conveyor 7 for transporting, in the direction indicated by arrow B, items into a warewasher 9.

Figure 1 b shows the ware separation apparatus 5 in more detail. As seen in Figure 1 b, the ware separation apparatus 5 comprises a vision system 51 for capturing images, a handling system 53 for picking up, handling, and transporting items to the warewasher transport area, and a control unit 55 for identifying items and controlling the vision system 51 and the handling system 53.

The vision system 51 , in this example, comprises a digital camera positioned over conveyer 3 for capturing images of items passing through the automated ware separation apparatus 5.

The handling system 53, in this example, comprises a moveable arm 57 coupled to a handling tool 10 for picking up and holding items during transport, by the arm 57 to the warewasher transport area. The handling tool 10 is configured for securing the items to be transported using suction provided by a high-power vacuum pump 59.

The control unit 55 receives images from the vision system 51 and, based on the received images, identifies an item to be transported, identifies the location of the item, and controls the moveable arm 57 and the vacuum pump 59 to pick up the identified item from the first conveyor 3, using the handling tool 10, and to transport the item to the second conveyor 7.

As seen in Figure 1 b, in this example, the items (wares) transported by the first conveyor 3 include crockery 43 and glassware 44 arranged on a tray 46. The items may be contaminated by debris such as food waste 42 and liquids 45. The skilled person will appreciate that other items may be present on the conveyor 3 or tray 46, such as napkins or cutlery.

As described in more detail later, the control unit 55 has a particularly beneficial configuration comprising an identification module configured to identify an item that is to be transported, and a location determination module configured to determine the approximate, relatively imprecise, location of the item. Beneficially, the determination of the approximate location of the item, as opposed to the precise centre of the item, significantly reduces the time required for the control unit 55 to process an image received from the vision system 51 , to identify and estimate the position of an item. It will be appreciated that this reduced processing time beneficially allows the throughput of the warewasher system to be increased.

As described in more detail later, the handling tool 10 has a beneficial configuration which is particularly tolerant to a relatively wide variation in positional alignment relative to an item that the handling tool 10 is required to pick up.

A first particularly beneficial configuration of the handling tool 10, described in more detail later, comprises a resilient skirt supported by at least one structural element. The skirt comprises an opening to a suction path, through which air may be drawn by the pump 59, to form a partial vacuum between the skirt and an item that is to be transported. This allows the apparatus to be securely and releasably attached to the item, so that the item can be rapidly moved to another location.

The resilient skirt is formed of a relatively soft and compliant material that is able to deform around debris present on the surface of the item, thus enabling a seal or partial seal to form between the skirt and the item. Because of the geometry and flexibility of the skirt, the partial vacuum can be formed even when the surface of the item is contaminated by relatively large objects such as cutlery, food waste, or when the surface of the item has significant curvature, for example if the item is a bowl. Beneficially, the use of the resilient skirt and the relative geometry of the handling tool, and in particular the suction path, also greatly reduces the amount of air and debris that flows into the apparatus during operation. The configuration of the handling tool, and in particular the flexibility of the skirt, also allows the apparatus to reliably attach to objects even when there are large positional errors or variations in the orientation of the objects.

Soft compliant material towards the periphery of the skirt conforms, in operation, to the shape of the ware underneath, including cutlery or debris. This helps to ensure that a sufficient vacuum seal generally occurs at some point in the entire (relatively large) area. This also ensures that any air path under the skirt is relatively convoluted, leading to greatly reduced air flow into the system towards the pump thereby reducing the risk of blocking and ingestion of large quantities of waste into the vacuum system.

Whilst the skirt is locally flexible so that it may deform around debris or irregular shapes, the skirt remains globally rigid such that the overall structure of the skirt is maintained even when the partial vacuum has formed, allowing undesirable folding of the skirt to be prevented.

It can be seen, therefore, that because of its design, the first configuration of the handling tool 10 is particularly position tolerant. More specifically, the handling tool 10 will successfully engage with, secure, and hold onto wares even when the tool is not precisely centred. For example, the handling tool 10 can successfully pick up a ware such as a plate or bowl even when the centre of the tool 10 engages with a location away from the centre (e.g. radially towards the edge of the ware). The handling tool 10 even has the potential to successfully pick up items when it engages with a non- flat or non-horizontal surface of the ware (e.g. the sloping side of a bowl or a raised edge of a plate or saucer).

A second particularly beneficial configuration of the handling tool 10, described in more detail later, comprises a resilient wall arranged such that, when the tool is inserted into a vessel such as glassware, the diameter of a part of the wall section inserted into a vessel increases as the tool is inserted further into the vessel. The handling tool, in one example, also beneficially comprises a resilient skirt at the lower end of the tool that assists secure engagement with an item that is in a non-standard orientation (e.g. a glass on its side or upside down). The skirt comprises an opening to a suction path, through which air may be drawn by the pump 59, to form a partial vacuum between the tool 10 and an item that is to be transported. This allows the tool 10 to be securely and releasably attached to vessels of a variety of sizes, shapes and orientations, so that the vessel can be rapidly moved to another location.

The resilient skirt and resilient wall of the second particularly beneficial configuration of the handling tool 10 are locally flexible such that they are able to deform around debris, thus enabling a seal or partial seal to form between the tool 10 and the item. Because of the relative geometry and flexibility of the tool, the partial vacuum can be formed even when the item is obstructed by relatively large objects such as cutlery, or by food waste. Beneficially, the relative geometry of the handling tool 10 greatly reduces the amount of air and debris that flows into the apparatus during operation. The configuration of the handling tool 10 and in particular the flexibility of the resilient skirt and the resilient wall also allows the apparatus to reliably attach to objects even when there are large positional errors or variations in the orientation of the objects.

Whilst the second configuration of the tool 10 is locally flexible so that it may deform around debris, the tool 10 remains globally rigid such that the overall structure of the tool 10 is maintained even when the partial vacuum has formed, allowing undesirable folding or collapse of the tool into a vessel to be inhibited.

It can be seen, therefore, that because of its design, the second configuration of the handling tool 10 is particularly position and orientation tolerant. More specifically, the handling tool 10 will successfully engage with, secure, and hold onto wares even when the tool 10 is not precisely centred. For example, the handling tool 10 can successfully pick up a ware such as a drinking glass 44 even when the centre of the tool 10 is not aligned with the central axis of the drinking glass 44 (e.g. radially towards the edge of the drinking glass 44). The handling tool 10 is also able to successfully pick up an item of glassware, cup, mug, jug, or similar liquid carrying vessel, regardless of its orientation, such as by creating a vacuum in the interior of, or attaching to an external surface of a base or outer wall of, the vessel 44.

The second particularly beneficial configuration of the handling tool 10 can pick up a glass or other drinking style or similar vessel, regardless of its orientation. The handling tool 10 can pick them up on their side, right way up, upside down, if there are large objects sticking out from them (e.g. cutlery), or if they are full or partially full of a liquid. The handling tool 10 achieves this using a continuous vacuum system to allow for a partial seal to be formed. The handling tool 10 is, in effect, a compliant end effector that can deform around objects. The handling tool 10 is designed to be locally compliant and flexible, but globally stiff to avoid collapse into the drinking vessels.

Moreover, because the first and second configurations of the handling tool 10 are particularly tolerant to a lack of precision in a location determined, for an identified ware, by the control unit 55 the use of the position tolerant handling tool 10 can significantly reduce the precision constraints that have to be imposed on the control unit 55. This in turn means that the control unit 55 has the potential to accurately identify wares and determine sufficiently precise position information relatively quickly, thereby increasing the possible throughput achievable using the ware separation apparatus 5. Alternatively, or additionally, the reduction in the position constraints allows more processing time to be devoted to accurately identifying wares and classifying them (e.g. as plates, bowls, cups, glasses, etc.) thereby providing the potential to improve separation reliability without a commensurate reduction in throughput.

In summary, therefore, the handling tool 10 can be used as an end-effector that allows a robot to pick up items in a warewasher system. The handling tool 10 is designed to handle items that are covered by a wide range of food waste, liquid, debris, cutlery and other items that can be expected on an item in a warewasher system. The handling tool 10 can then move the entire item to another location at a rapid pace. The use of a large high powered vacuum system helps to ensure sufficient suction to lift the item even with leakage in the system. The handling tool 10 is beneficially designed to operate successfully even with a partial seal, thereby mitigating issues associated with contamination and strange curvature surfaces. The handling tool 10 has a geometry which can either align the object being picked up to the surface of the tool 10, or does not need to be placed in an exact position for an item to be successfully secured and picked up. It is designed to be compliant to large positional errors (typically, but not limited to, >1 mm, up to 10-20mm). This means that the wider handling system does not need to position the handling tool as precisely and therefore does not require high precision motors, thereby reducing system installation and maintenance costs. Moreover, the tolerance of the tool 10 to positional errors enables the use of a particularly beneficial configuration of the control unit 55 in which an approximate, relatively imprecise, location of an item can be determined. Beneficially, the determination of the approximate location of the item, as opposed to the precise centre of the item, reduces the time required for the control unit 55 to process an image received from the vision system 51 , to identify and estimate the position of an item. It will be appreciated that this reduced processing time beneficially allows the throughput of the warewasher system to be increased or the accuracy with which an item is identified / classified to be improved for a given throughput.

Illustrative Examples - Control Unit

The control unit 55 will now be described in more detail, by way of example only, with reference to Figures 2, 3a and 3b.

Referring to Figure 2, this shows a simplified block diagram of the control unit 55. The control unit 55 comprises at least one vision system interface 551 , at least one handling system interface 552, a transceiver circuit 553, a controller 554 and memory 555.

Software stored in the memory 555 includes, among other things, an operating system 556, a communication control module 557, an image processing module 558, an identification module 559, a location determination module 560 and a handling system control module 561. It will be appreciated that whilst, for ease of understanding, the controller 554 is described as operating under the control of a number of discrete software modules, the functionality attributed to these modules may be built into the overall operating system 556 or as separate code in such a way that the modules may not be discernible as discrete entities.

The transceiver circuit 553 is operable to transmit signals to, and to receive signals from, the vision system 51 via the vision system interface 551 , and the handling system 53 via the handling system interface 552. It will be appreciated that the handling system interface 552 may comprise separate interfaces for the moveable arm 57 and the vacuum pump 59. The operation of the transceiver circuit 553 is controlled by a controller 554 in accordance with the software stored in the memory 555.

The communication control module 557 controls communication with the vision system 51 to receive image data from the vision system 51. The communication control module 557 also controls communication with the handling system 53, for example to send instructions to the vacuum pump 59 or the moveable arm 57.

The image processing module 558 processes the image received from the vision system 51 to produce a processed image for use by the identification module 559 and the location determination module 560. This processing may include, for example, resizing or altering the contrast of the received image. The skilled person will appreciate that other forms of image processing such as image sharpening may also be performed. However, it will also be appreciated that if no such processing is necessary, then the image received from the vision system 51 may be used directly by the identification module 559 or the location determination module 560, without any additional image processing being performed.

The identification module 559 identifies (and classifies) an object that is present in the image captured by the vision system 51. In other words, the identification module 559 determines that an item of potential interest is present in the image, and classifies the item, for example, as an item of crockery or as an item of glassware. Referring to Figures 3a and 3b, in this example the vision system has captured an image 30 of an area that contains a tray 46, glassware 44 and crockery 43. The identification module 559 identifies an item of crockery 43 in the image 30, although the skilled person will appreciate that the identification module 559 may also be configured to identify other objects such as glassware or dining trays.

The location determination module 560 determines the approximate location 33 of the item of crockery 43 identified by the identification module 560. The location determination module 560 determines a bounding box 31 that contains at least a major portion of the identified crockery 43. In this example, the location determination module 560 determines the geometric centre of the bounding box 31 , and assigns the geometric centre as the approximate position 33 of the crockery 43. The approximate position 33 may then be used by the handling system control module 561 as an estimate of the position of the actual centre 32 of the item of crockery 43. It will be appreciated that the bounding box can represent a region of the image that is significantly larger in area than an identified object within that region. For example, the bounding box may represent a region of the image that is 10% (or more) larger in area than an identified object within that region. Moreover, the bounding box can have a centre that is significantly offset relative to the centre of the identified object. Accordingly, the positional estimate provided using this technique can deviate significantly from an actual location of the object. For example, the estimate of the position of an object may typically deviate, on average, by approximately 10% of the size of the object from the actual centre of the object.

Beneficially, the identification module 559 and the location determination module 560 utilise a neural network to identify the item and determine a bounding box 31 that contains the item. For example, a convolutional neural network may be used. The determination of a bounding box 31 to estimate the position of the item beneficially allows the detection system to process images at the relatively high speeds required in a warewasher system, so that the throughput of the warewasher system is not undesirably reduced. For example, the detection system may be able process at least 5 images per second (e.g. 60 images per second). In contrast, whilst greater positional accuracy may be achievable using an alternative algorithm, for example an algorithm that determines a precise centre of an object with sub-pixel precision based on precisely identifying the edges of an object, such algorithms would process images at relatively low speeds and would thus have a negative impact on the throughput of the warewasher system.

A further advantage of utilising a neural network to perform the object identification and location determination is that the network is often able to accurately identify and classify objects even when there is a large variety in the types or styles of objects entering the machine, and when the objects are heavily contaminated by debris. For example, if a new style of crockery, such as a dining plate that has a shape not previously encountered by the neural network, is introduced into the warewasher system, the network is beneficially still able to identify and classify the object as an item of crockery.

Whilst the location determination module 560 and the identification module 559 have been described for clarity as separate modules in the present example, the skilled person will appreciate that the location determination and identification may be performed by a single neural network. In other words, a single neural network may be used to identify and classify an object, and determine the location of the object, in an integrated process in which the identification and location determination cannot necessarily be separated into separate software modules albeit that the neural network will produce separate outputs for the purposes of identification and location determination.

Whilst a variety of neural networks could be adapted for use in the system of the present example, one particularly beneficial neural network based system uses a ‘You Only Look Once’ (‘YOLO’) based network, such as the YOLO9000 real-time object detection system. The neural network of the YOLO9000 system may be pre- trained to classify and detect objects using publicly-available datasets, and adapted using transfer learning for use in a warewasher system. It will be appreciated by the skilled person that although the use of networks that determine a bounding box 31 to estimate the position of an item allows images to be processed at the relatively high rate required in a warewasher system, the position determined for the item is relatively imprecise. As indicated by the arrow 34 shown in Figure 3b, the geometric centre of the bounding box 31 used to estimate the position 33 of an item may be a significant distance from the actual centre 32 of the item. This means that is impractical to use a high precision gripper, such as a claw style gripper able to transport a ware contaminated by debris, in combination with the fast, but relatively imprecise, determination of the location of the ware. Beneficially, the handling tool of the present invention does not require precise position alignment with respect to the item that is being transported, even when the item is contaminated by significant amounts of debris. It will be appreciated therefore, that the use of the handling tool 10 of the present invention in combination with the use of the control unit 55 that is able to quickly, although relatively imprecisely, process an image to estimate the position of an item, is a particularly beneficial combination.

Using the position 33 estimated by the location determination module 560, the handling system control module 561 controls the handling system 53 to attach the handling tool 10 to the item of crockery 43 at the location estimated by the location determination module 560 (or at a different location that is determined relative to the estimated location e.g. based on a fixed positional offset). In the present example, the handling system control module 561 controls the handling system 53 to align the central axis of the handling tool 10, indicated by the dashed line in Figure 5, with the estimated position 33 of the item of crockery 43. It will be appreciated that, since the items in the image 30 captured by the vision system 51 may be located on a moving conveyor 3, the handling system control module 561 also accounts for movement of the conveyor 3 when controlling the handling system 53 to attach the handling tool 10 to the crockery 43.

Illustrative Examples - Handling Tool - first configuration

A first illustrative example 100 of the handling tool 10 will now be described in more detail, by way of example only, with reference to Figures 4 to 8.

Referring to Figure 4, this shows a simplified 3D view of the handling tool 100. The tool 100 comprises a resilient skirt 14, a supporting part 13 and a conduit 12.

The supporting part 13 is attached to, and coaxially aligned with, the resilient skirt 14 and provides structural support for the resilient skirt 14. The conduit 12 is coaxially aligned with, and passes through the supporting part 13 to the resilient skirt 14 as indicated by the dashed lines. In the present example, the supporting part 13 is illustrated as having a frustoconical configuration, although the skilled person will appreciate that any other suitable shape such as a hemispherical or cylindrical configuration may be used. In the present example, the resilient skirt 14 is illustrated as being a generally flat circular (or‘disciform’) shape, although the skilled person will appreciate that any other suitable non-circular shape such as an elliptical or polygonal shape may be used (e.g. to provide better performance with particular types or shapes of crockery).

The resilient skirt 14 may beneficially be formed of any suitable resilient material such as food-safe polyurethane. However, it will be appreciated that any other suitable material such as silicone may be used. The supporting part 13 and the conduit 12 may be formed of any suitable material, for example a material such as food-safe polyurethane, rubber or silicone may be used.

The diameter of the skirt 14 is beneficially between 200mm and 350mm. However, a particularly advantageous diameter of the skirt 14 is 275mm, which provides a suitable area of the lower surface 14-1 of the skirt to engage with a typical item of crockery 43 found in a warewasher system.

The thickness of the skirt 14 is beneficially between 1 mm and 3mm, such that the flexibility of the skirt 14 is suitable for deforming around debris found on the surface of an item of crockery, whilst maintaining the shape of the skirt 14 when not engaged with an item of crockery. However, a thickness of the skirt 14 of 2mm was found to be particularly beneficial for maintaining global rigidity whilst allowing localised flexibility.

The internal diameter of the conduit 12 is beneficially between 25mm and 40mm. However, a particularly advantageous diameter was found to be 30mm, to provide sufficient fluid flow through the conduit 12 and to inhibit the formation of obstructions inside the conduit 12 by debris.

The supporting part 13 beneficially has a largest diameter of 100mm. However, it will be appreciated that the diameter of the supporting part 13 may be larger or smaller depending on the size of the item that is to be picked up. Figure 5 schematically illustrates a cross section of the handling tool 100. The resilient skirt 14 comprises an opening that is aligned with the opening at the lower end of the conduit 12, so that a path for fluid flow 11 indicated by the dashed line is formed. When the pump 59 is connected to the upper end of the conduit and turned on it thus attempts to draw fluid (air) through the opening of the resilient skirt 14 and into the conduit 12 thereby providing suction. A filter 15 is provided in the conduit, proximate the opening in the skirt, to inhibit debris from entering the conduit 12 and forming an obstruction to fluid flow or damaging the pump 59. In the present example the filter 15 is shown positioned inside the lower end of the conduit 12 (e.g. for ease of access and cleaning). However, the skilled person will appreciate that the filter 15 may be located at any other suitable position inside the conduit 12, or may instead by provided separately by the pump. The pore size of the filter 15 is beneficially configured to inhibit the debris most likely to form an obstruction in the conduit 12 from passing through the conduit 12. For example, the width of the pores of the filter 15 may be smaller than the width of a typical grain of cooked rice. Beneficially, the filter may be formed of a relatively stiff, metallic material such as stainless steel, to provide increased resistance to deformation under pressure. However, it will be appreciated that any other suitable material may be used. Whilst beneficial, the filter 15 need not necessarily be provided. For example, if the handling tool 100 is used to transport clean, dry crockery at a later stage of the warewashing process then the filter may not be necessary.

The resilient skirt 14 has a first, generally planar but resilient, ware engaging surface 14-1 (on the underside of the skirt as viewed in Figure 3) for engaging with and conforming to a surface of a ware (such as a plate, bowl, or other item of crockery) during operation to form a sufficiently strong vacuum interface with the ware, to secure the handling tool 100 to the ware for reliable lifting and transporting of the ware to another location. The resilient skirt 14 has a second generally flat surface 14- 2 (on the upper side of the skirt as viewed in Figure 3) that interfaces with the supporting part 13.

The supporting part 13 has a transverse cross-sectional area (perpendicular to a main central longitudinal axis through the conduit 12), where the supporting part 13 interfaces with the second surface 14-2 of the skirt 14, that is relatively smaller than the transverse cross-sectional area of the skirt 14 (in the plane of the skirt). Thus, the skirt 14 has a portion 14-3 (that is annular in the illustrated example) that extends outwardly, in a radial direction, beyond the interface 14-2 with the supporting part 13. This extended portion 14-3 of the skirt is thus more flexible than the portion that interfaces with the supporting part 13, allowing it to conform closely to the contours of a ware when, in operation, it is brought into contact with the ware and a vacuum is applied. The extended portion 14-3 is, nevertheless, rigid enough (by virtue of its thickness) to maintain its generally flat and planar shape when not engaged with a ware.

It will be appreciated that whilst the skirt 14 and supporting part 13 are described separately they may be formed as a single integrated part (e.g. using an appropriate moulding process or the like).

Figure 6 schematically illustrates the handling tool 100, in operation, attached to an item of crockery 43.

In operation, to arrive at the illustrated positon, it will be appreciated that the control unit 55 controls the moveable arm 57 to move the handling tool 100 into position such that the resilient skirt 14 engages with an item of crockery identified by the control unit 55. The control unit 55 also controls the pump 59 to draw air through the conduit 12 to form a vacuum or partial vacuum. As can be seen in Figure 6, the resilient skirt 14 is in contact with an item of crockery 43 contaminated by food debris 42 and an item of cutlery 41. The resilient skirt 14 deforms to form a seal or partial seal with the crockery 43, despite the irregular surface presented by the food debris 42 and the item of cutlery 41. When the high-power vacuum pump 59 pumps air out of the conduit 12 a partial vacuum is formed in the area between the resilient skirt 14 and the crockery 43, and the handling tool 100 becomes secured to the crockery 43. The control unit 55 then controls the moveable arm 57 to transport the crockery 43 to another location. For example, the control unit may control the moveable arm 57 to transport the crockery 43 from the first conveyor 3 to the second conveyor 7.

After the crockery 43 has been transported to the desired location using the moveable arm 57, the partial vacuum is destroyed by allowing air to flow into the area between the resilient skirt 14 and the crockery 43, and the crockery 43 becomes detached from the resilient skirt 14. For example, air may be allowed to flow into the conduit 12 via a valve provided in the wall of the conduit. The valve may be controlled by the control unit 55. However, the skilled person will appreciate that air may be introduced into the area between the resilient skirt 14 and the crockery 43 by any other suitable method, to destroy the partial vacuum formed in the area between the resilient skirt 14 and the crockery 43. Illustrative Examples - Handling Tool - second configuration

A second illustrative example 200 of the handling tool 10 will now be described in more detail, by way of example only, with reference to Figures 9 to 13b.

Referring to Figure 9, this shows a simplified 3D view of the handling tool 200. The tool 200 comprises a resilient skirt 23, a resilient wall 22 and a conduit 21.

In the present example, the resilient wall 22 is illustrated as having a generally hollow funnel shaped configuration, having a generally frustoconical boundary, such that the diameter of the conical frustum increases from a lower (or‘distal’) end of the tool towards the upper (or‘proximal’) end of the tool (when the tool is in its normal operational orientation as seen in Figure 9). However, it will be appreciated that any other suitable shape such as a hemispherical configuration may be used.

The conduit 21 comprises a generally elongate tubular member of circular cross- section (in this example) configured for direct or indirect coupling, for fluid flow, to the pump 59. The frustoconical shape formed by the resilient wall 22 and conduit 21 are coaxially aligned with one another along mutual longitudinal axis as indicated by the dashed line (X-X’).

The resilient skirt 23 extends around a perimeter of a narrow end of the conical frustum formed by the resilient wall 22 and flares outwardly, from that narrow end of the conical frustum, towards a base of the handling tool 200. The resilient skirt 23 thus has a generally annular shape having a narrower portion that has an external circular perimeter that coincides with a corresponding perimeter of the narrow end of the conical frustum, and a wider portion at the base of the handling tool. The narrower portion of the annular skirt 23 has an internal circular aperture of the same (or similar) size to that of the conduit 21.

In the present example, the handling tool 200 is illustrated as having a generally circular shaped transverse cross-section, although the skilled person will appreciate that any other suitable non-circular shape, such as an elliptical or polygonal shaped cross-section, may be used for one or more of the components 21 , 22, 23 of the handling tool 200. (e.g. to provide better performance with particular types or shapes of glassware).

The largest diameter of the resilient wall 22 is beneficially between 20mm and 40mm larger than the diameter of the largest opening that the tool is to be inserted into, for example when the tool 200 is inserted into an item of glassware 44 as shown in Figure 11 a. For example, a diameter of 110mm may be used.

The largest diameter of the resilient skirt 23 is beneficially between 10mm and 20mm smaller than the diameter of the smallest opening that the tool 200 is to be inserted into, for example when the tool 200 is inserted into an item of glassware 44 as shown in Figure 1 1 a, to prevent the resilient skirt 23 from interfering with the insertion of the tool 200 into the item. However, a largest diameter of the resilient skirt 23 of 40mm was found to be particularly beneficial, as this reduced the interference by the resilient skirt 23 to the insertion of the tool 200 into items of glassware 44 commonly found in a warewasher, whilst still allowing a sufficient seal to form between the resilient skirt 23 and an item of glassware 44 when the tool 200 is attached to the item of glassware in one of the orientations shown in Figures 1 1 b or 11 c.

The angle of the resilient wall 22 to the central axis of the tool X-X’ shown in Figure 9 may beneficially be between 5 degrees and 20 degrees (i.e. between 70 degrees and 85 degrees to a transverse plane orthogonal to the central axis). However, a particularly beneficial angle between the resilient wall 12 and the central axis of the tool X-X’ was found to be 10 degrees, to improve the seal formed between the tool 200 and an item of glassware 44.

The thickness of the resilient wall 22 and the resilient skirt 23 is beneficially between 2mm and 4mm. However, a thickness of 3mm was found to be particularly beneficial for allowing the tool 200 to remain locally flexible, whilst preventing the tool 200 from collapsing into an item of glassware 44.

It will be appreciated that whilst the resilient skirt 23, resilient wall 22 and the conduit 21 are described separately, one or more of these components may be formed as a single integrated part (e.g. using an appropriate moulding process or the like). It will also be appreciated that the conduit could extend longitudinally through and out of the aperture in the frustum or other shape formed by the resilient wall 22 and that the resilient skirt 23 could extend from the conduit rather than (or as well as) from the resilient wall 22.

The resilient skirt 23 and resilient wall 22 may beneficially be formed of any suitable resilient material such as silicone. The conduit 21 may be formed of any suitable material, for example a material such as silicone, rubber or plastic may be used. Figure 10 schematically illustrates a cross section of the handling tool 200. The resilient skirt 23 comprises an opening that is aligned with the opening at the lower end of the conduit 21 , so that a path for fluid flow 24 indicated by the dashed line X- X’ is formed. When the pump 59 is coupled directly or indirectly to the upper end of the conduit 21 and turned on it thus attempts to draw fluid (air) through the opening of the resilient skirt 23 and into the conduit 21 thereby providing suction. A filter 25 is provided in the conduit 21 , proximate the opening in the skirt 23, to inhibit debris from entering the conduit 21 and forming an obstruction to fluid flow or damaging the pump 59. In the present example the filter 25 is shown positioned inside the lower end of the conduit 21 (e.g. for ease of access and cleaning). However, the skilled person will appreciate that the filter 25 may be located at any other suitable position inside the conduit 21 , or may instead by provided separately by the pump 59. Beneficially, the filter may be formed of a relatively stiff, metallic material such as stainless steel, to provide increased resistance to deformation under pressure. Whilst beneficial, the filter 25 need not necessarily be provided. For example, if the handling tool 200 is used to transport clean, dry glassware at a later stage of the warewashing process, or if the vacuum pump 59 is designed to tolerate the ingestion of liquids and solid debris, then the filter 25 may not be necessary. When a filter is not provided, a larger transverse cross sectional area of the path for fluid flow may be provided in order to inhibit the obstruction of the path for fluid flow by debris.

Figures 1 1 a, 1 1 b and 11 c schematically illustrate the handling tool 200, in operation, attached to an item of glassware 44 in three different orientations.

In operation, to arrive at the illustrated positions, it will be appreciated that the control unit 55 controls the moveable arm 57 to move the handling tool 200 into position such that the tool 200 engages with an item of glassware 44 identified by the control unit 55. The control unit 55 also controls the pump 59 to draw air through the conduit 21 to form a vacuum or partial vacuum. As can be seen in Figures 4a, 4b and 4c the relative geometry of the handling tool 200 enables it to securely attach to an item of glassware 44, regardless of the orientation of the glassware 44.

Figure 1 1a illustrates the handling tool 200 attached to an item of glassware 44 in the upright position. As can be seen in Figure 1 1a, the tool 200 is partially inserted into the cavity formed by the inner wall of the glassware 44, such that the resilient wall 22 of the tool 200 is engaged with a rim at the top of a wall of the glassware 44 to form a seal or partial seal with the glassware 44. When the high-power vacuum pump 59 pumps air out of the conduit 21 a vacuum or partial vacuum is formed in the area between the tool 200 and the glassware 44, and the handling tool 200 becomes secured to the glassware 44. The control unit 55 then controls the moveable arm 57 to transport the glassware 44 to another location. For example, the control unit may control the moveable arm 57 to transport the glassware 44 from the first conveyor 3 to the second conveyor 7.

Figures 11 b and 1 1 c illustrate the handling tool attached to an external surface of the wall of an item of glassware 44. To arrive at the illustrated positon, it will be appreciated that the control unit 55 controls the moveable arm 57 to move the handling tool 200 into position such that the resilient skirt 23 engages with the outer wall of the glassware 44. The resilient skirt 23 deforms to form a seal or partial seal with the glassware 44, despite the curved surface presented by glassware 44 in Figure 1 1 b. When the high-power vacuum pump 59 pumps air out of the conduit 21 a partial vacuum is formed in the area between the resilient skirt 23 and the glassware 44, and the handling tool 200 becomes secured to the glassware 43.

It can be seen that, beneficially, even if the handling tool 200 is imprecisely located such that it is moved into (or onto a surface of) the glassware 44 at a non-ideal location (e.g. non-centrally relative to the opening in Figure 1 1a), the effect of applying suction and/or moving the tool into position can result in a beneficial realignment of the glassware 44.

Figure 12 schematically illustrates the handling tool 200, in operation, attached to an item of glassware 44 that is contaminated by a liquid 45 and an item of cutlery 41. As can be seen from Figure 12, the local flexibility and global rigidity of the resilient wall 22 allows a seal or partial seal to form, and hence a partial vacuum to form in the area between the tool 200 and the inner wall of the glassware 44, despite the obstruction formed by the item of cutlery 41.

After the glassware 44 has been transported to the desired location using the moveable arm 57, the partial vacuum is destroyed by allowing air to flow into the area between the tool 200 and the glassware 44, and the glassware 44 becomes detached from the tool 200. For example, air may be allowed to flow into the conduit 21 via a valve provided in the wall of the conduit 21. The valve may be controlled by the control unit 55. However, the skilled person will appreciate that air may be introduced into the area between the tool 200 and the glassware 44 by any other suitable method, to destroy the partial vacuum formed in the area between the tool 200 and the glassware 44. Modifications and alternatives

An exemplary warewasher system, control unit, and handling tool have been described above in detail. As those skilled in the art will appreciate, a number of modifications and alternatives can be made to the above examples and variations whilst still benefiting from the inventions embodied therein.

In the above description, the control unit 55 is described for ease of understanding as having a number of discrete software modules. However, it will be appreciated that the functionality performed by part or all of the software may be performed using one or more dedicated hardware circuits for example using one or more dedicated integrated circuits such as an application specific integrated circuit (ASIC) or the like. The use of software modules is, nevertheless, preferred as it facilitates the updating of the warewasher.

It will be appreciated that the controller 554 referred to in the description of the control unit 55 may comprise any suitable controller such as, for example an analogue or digital controller. Each controller may comprise any suitable form of processing circuitry including (but not limited to), for example: one or more hardware implemented computer processors; microprocessors; central processing units (CPUs); arithmetic logic units (ALUs); input/output (IO) circuits; internal memories / caches (program and/or data); processing registers; communication buses (e.g. control, data and/or address buses); direct memory access (DMA) functions; hardware or software implemented counters, pointers and/or timers; and/or the like.

It will be appreciated that whilst in the example shown in Figure 3a and 3b a rectangular box has been used as the bounding box, any other suitable shape, such as a circle, could be used.

Beneficially, a resilient mechanism (such as, but not limited to, a damped spring) may be provided to provide tolerance for positional errors in the direction along the central axis of the tool 10. The resilient mechanism may be provided, for example, at an interface between the moveable arm 57 and the handling tool 10 or may be provided at any other suitable position (e.g. as a resilient portion of the conduit 1 1 ). For example, in operation, the moveable arm 57 may move the handling tool 10 to a fixed height, and the provision of the resilient mechanism allows the height of the handling tool 10 to adjust to the height of the ware that is to be picked up. This beneficially reduces the precision required in the movement of the moveable arm 57 along the central axis of the tool 10. Additionally, the provision of the resilient mechanism (especially in combination with the flexibility of the handling tool 10) beneficially improves the ability of the handling tool 10 to attach to wares that are tilted such that, before the tool 10 is engaged with the ware, the ware is offset by a significant angle from one of the positions shown in Figures 6, 1 1a, 11 b or 1 1 c. Beneficially, the resilient mechanism may tolerate positional errors of up to 30mm in the direction along the central axis of the tool, for example a displacement of 12mm may be tolerated.

The handling system 5 may be provided with a sensor for sensing if the handling tool 10 is engaged with the surface of a ware. For example, the handling system 5 may be provided with a sensor for sensing a displacement of the resilient mechanism, from an equilibrium position of the resilient mechanism when the handling tool 10 is not engaged with a ware. The sensor may be provided, for example, in the form of a linear encoder, a microswitch, a strain gauge or a flag breaking an optical beam. The sensor may be provided, for example, by the resilient mechanism. The provision of the sensor beneficially enables the handling system 5 to determine if an attempted engagement of the handling tool 10 with a ware is successful. Alternatively, the sensor for sensing if the handling tool 10 is engaged with the surface of a ware may be provided as a pressure sensor or an air flow sensor. For example, a change in the air pressure inside the conduit 1 1 , or a change in the flow rate of fluid through the conduit 1 1 , may be used to determine if the handling tool 10 is engaged with the surface of a ware.

The handling system 5 may be provided with a sensor for sensing the weight of an object that the handling tool 10 is attached to. For example, when the handling tool is attached to an item of crockery contaminated by food debris, the weight indicated by the sensor may beneficially be used to determine the amount of food debris attached to the crockery, or to confirm the identity of the crockery.

Figure 7 shows top-down and simplified 3D views of the handling tool 100 in which struts 16 are beneficially provided on the upper surface of the resilient skirt 14. The struts 16 provide additional structural support for the resilient skirt 14 and help prevent undesirable folding of the resilient skirt 14. Although in the example shown in Figure 7 radial struts are provided, the skilled person will appreciate that any other suitable configuration may be used to provide structural support for the resilient skirt 14. For example, concentric rings around the central axis of the handling tool 100 may be provided on the upper surface of the resilient skirt 14. In Figure 8 an alternative configuration of the resilient skirt 14 and support 13 is shown in which the resilient skirt comprises a main, outer annular section 142 that is substantially perpendicular to a main central axis of the handling tool 100, and an inner concave section 141. Both sections 141 , 142 are coaxially aligned around the central axis of the fluid conduit 12 with the opening to the conduit in the skirt being located in the concave section 141. The concave section 141 is configured to form, when the handling tool 100 engages with a ware, a cavity centred on the opening to the fluid conduit 12, between the surface of the ware and an external surface of the concave section 141 of the skirt. Beneficially, the height of the cavity in the longitudinal direction, from a plane defined by the lower surface of the outer annular section 142, to the distal end of the conduit 12, may be between 5mm and 20mm, for example the height may be 10mm. The largest diameter of the cavity, in the transverse direction, is beneficially between 120mm and 200mm, depending on the dimensions of the ware that is being transported and the dimensions of the outer annular section 142. The formation of this cavity beneficially helps to reduce the amount of bridging and associated blockage that occurs across the opening in the skirt by relatively small particles of debris such as rice. The formation of the cavity was also found to beneficially improve the reliability of the attachment of the handling tool 100 to wares contaminated by significant amounts of debris such as food waste.

In the above examples the handling tool 100 has been described as transporting an item of crockery 43. However, the skilled person will appreciate that the handling tool 100 may be used to transport any other compatible items such as glassware 44 or dining trays 46.

Although the resilient skirt 14 of the handling tool 100 has been illustrated as having a uniform thickness, the thickness of the resilient skirt 14 may beneficially decrease towards the outer edge of the skirt 14 to provide increased flexibility at the outer edge of the skirt 14. This allows for larger local deformations closer to the edge of the skirt 14 to improve the seal formed between the resilient skirt 14 and the object to which the handling tool 100 is attached.

The supporting part 13 of the handling tool 100 may be hollow to reduce the weight of the apparatus, or may be solid in order to provide increased structural support to the resilient skirt 14.

Although the lower surface of the skirt 14, of the handling tool 100, has been illustrated as being smooth, the lower surface of the skirt 14 may beneficially be textured. When the lower surface of the skirt 14 is textured, the amount of debris that moves towards the opening at the centre of the skirt when the high-power vacuum pump is switched on may be reduced. However, providing a smooth lower surface of the skirt 14 may beneficially improve the reliability of the release of wares from the handling tool 100, and reduce the time taken for a ware to disengage with the surface of the handling tool, when the partial vacuum between the tool 100 and the ware is destroyed.

In the above examples the resilient skirt 14 of the handling tool 100 has been described as comprising a single opening for fluid flow into the conduit 12. However, the skilled person will appreciate that the resilient skirt may beneficially comprise a plurality of openings, so that fluid may flow into the conduit 12 even when one of the openings becomes blocked by debris.

Beneficially, the inner width of the conduit 12 of the handling tool 100 may increase towards the opening in the skirt 14, to inhibit the formation of obstructions of the path for fluid flow 1 1. In other words, a section of the conduit 12 proximal to the opening in the skirt 14 may be tapered such that the cross sectional area of the conduit 12 increases towards the opening in the skirt 14.

Figures 13a and 13b, show cross-sectional and simplified 3D views of the second exemplary handling tool 200 in which the resilient wall is arranged to have a generally hemispherical configuration 26, and a resilient skirt is not provided. The hemispherical configuration of the resilient wall beneficially increases the global rigidity of the tool 200 such that the resistance to collapse of the overall structure is increased, whilst maintaining the local flexibility that enables the resilient wall to deform around debris.

In the above examples the handling tool 200 has been described as transporting an item of glassware 44. However, the skilled person will appreciate that the handling tool 200 may be used to transport any other compatible items such as crockery 43 or dining trays 46.

Although the resilient wall 22 and resilient skirt 23 of the handling tool 200 have been illustrated as having a uniform thickness, the thickness of the resilient wall 22 and resilient skirt 23 may beneficially be non-uniform to provide increased local flexibility in particular areas of the tool 200. This allows the local flexibility of the tool 200 to be tailored to the shape of the object to which the tool 200 is to be attached, to improve the seal formed between the tool 200 and the object. Although the lower surface of the skirt 23 has been illustrated as being smooth, the lower surface of the skirt 23 may beneficially be textured. When the lower surface of the skirt 23 is textured, the amount of debris that moves towards the opening at the centre of the skirt when the high-power vacuum pump is switched on may be reduced. However, providing a smooth lower surface of the skirt 23 may beneficially improve the reliability of the release of wares from the handling tool 200, and reduce the time taken for a ware to disengage with the surface of the handling tool, when the partial vacuum between the tool 200 and the ware is destroyed.

Beneficially, a section of relatively stiff material may be provided at the interface between the resilient skirt 23 and the conduit 21 (or at the end of the distal end of the conduit 21 for the hemispherical configuration in Figure 6). For example, a ring of relatively stiff material such as, but no limited to, plastic, may be provided between the resilient skirt 23 and the conduit 21. This beneficially inhibits the collapse of the path for fluid flow 14 when the vacuum or partial vacuum has formed, and may also act as a hard end-stop to inhibit folding of the skirt 23 into the path for fluid flow 24. Alternatively, the stiffness of a part of the conduit 21 proximal to the resilient skirt 23 may be increased by providing additional material, for example by increasing the thickness of the part of the conduit 21 or by providing ribs on the surface of the conduit 21. Providing the section of relatively stiff material beneficially allows the tool 200 to remain locally flexible, whilst increasing the amount of force required to deform the tool 200 when the deformation becomes undesirably large. However, it will be appreciated that a section of relatively stiff material need not necessarily be provided.

Although the resilient wall 22 of the handling tool 200 has been described as having a hollow frustoconical configuration, it will be appreciated that a solid frustoconical configuration may be provided such that conduit 21 passes through the centre of a solid, resilient conical frustum, in order to increase the rigidity of the tool 200.

In the above examples the resilient skirt 23 of the handling tool 200 has been described as comprising a single opening for fluid flow into the conduit 21 . However, the skilled person will appreciate that the resilient skirt 23 may beneficially comprise a plurality of openings, so that fluid may flow into the conduit 21 even when one of the openings becomes blocked by debris.

Various other modifications will be apparent to those skilled in the art and will not be described in further detail here. A number of examples of handling tools have been described in detail along with detailed description of examples of apparatus and methods in which those handling tools may be used. A summary of those examples, in broad terms, is set out below.

In one of the above examples, there is described in broad terms a handling tool for handling wares in a warewasher system, the handling tool comprising: means for engaging with a ware to be handled for releasably securing the handling tool to the ware for transport of the ware from a first location to a second location in the warewasher system; wherein the engaging means comprises: a resilient ware engaging plate having a ware engaging surface for mutually engaging with a surface of the ware to be handled; and means for coupling the handling tool to a vacuum source for application of at least a partial vacuum between the ware engaging surface and the surface of the ware to be handled for releasably securing the ware to the handling tool during operation; and wherein the ware engaging plate is formed of compliant resilient material whereby, in operation, the ware engaging surface will conform to a surface of the ware to be handled.

The ware engaging surface may be generally planar over a greater part of the ware engaging surface when not engaged with a ware to be handled.

The ware engaging surface may be smooth.

The ware engaging plate may comprise an aperture via which said at least a partial vacuum can be applied. The means for coupling may comprise a conduit for forming a fluid path from the aperture to the vacuum source. The ware engaging surface may have a surface area that is greater than a transverse cross-sectional area of the aperture. The ware engaging surface may have a surface area that is at least five times the transverse cross-sectional area of the aperture. The ware engaging surface may have a surface area that is at least ten times the transverse cross-sectional area. The aperture may be located generally centrally to, and through the plane of, the ware engaging plate. The aperture may have at least one of the following geometries: circular with a diameter ~25mm to ~40mm, for example ~30mm (with tolerances of plus or minus 20%); and a transverse cross-sectional area of ~500mm² to -1250mm², for example ~700mm² (with tolerances of plus or minus 40%).

The ware engaging plate may comprise a concave portion extending from the aperture to the ware engaging surface. The height of the concave portion along an axis generally orthogonal to a plane of the ware engaging surface, from the ware engaging surface to the aperture, may be between 5mm and 20mm, (e.g. 10mm).

The largest width of the concave portion, along an axis generally parallel to the plane of the ware engaging surface (e.g. the diameter of a concave portion, having a circular cross-section, at its widest part) , may be between 120mm and 200mm.

The fluid conduit may be provided with a filter for inhibiting material from being drawn through the conduit towards the vacuum source. The filter may be formed of a metallic material.

The ware engaging plate may have a geometry (e.g. thickness and transverse cross- sectional area) whereby the ware engaging plate is sufficiently rigid to maintain a generally planar shape when not engaged with a ware, and sufficiently compliant to conform to a surface of the ware to be handled during operation to releasably secure the ware to the handling tool. The ware engaging plate may, for example, have a thickness in the range ~1 mm to ~3mm, for example ~2mm (with tolerances of plus or minus -20%). The ware engaging plate may, for example, have at least one of the following geometries: circular with a diameter of ~200mm to ~350mm, for example ~275mm (with tolerances of plus or minus 20%); and a transverse cross-sectional area of -30000mm² to -100000mm², for example -60000mm² (with tolerances of plus or minus 40%). The ware engaging plate may, for example, have a transverse cross-sectional area that is configured to be similar to (e.g. within 20% of) an area of a surface, of a largest ware that the handling tool is configured to handle, which surface the ware engaging surface will engage with in operation. The ware engaging plate may be generally disc shaped. The ware engaging plate may be formed of a material from the list: food safe polyurethane; rubber; and silicone.

The engaging means may further comprise at least one supporting element extending from a generally central region on the ware engaging plate, whereby to provide additional rigidity across that central region. The supporting element and the ware engaging plate may be formed of the same compliant resilient material. The supporting element may be one of: frustoconical in shape; disc shaped; and hemispherical in shape. The supporting element may have at least one of the following geometries at an interface with the ware engaging plate: circular with a diameter of -100mm (with tolerances of plus or minus 20%); and a transverse cross- sectional area of -7850mm² (with tolerances of plus or minus 40%). The supporting element may have a transverse cross-sectional area that is configured to be smaller than an area of a part of a smallest ware that the handling tool is configured to handle, which part the ware engaging surface will engage with in operation.

The handling tool may further comprise a resilient mechanism configured to allow, in operation, reciprocal movement of the ware engaging plate along an axis generally orthogonal to a plane of the ware engaging plate whereby, in operation, the relative height of the ware engaging plate will adjust automatically to ware-to-ware variation in a maximum height of wares to be handled. The resilient mechanism may be further configured to allow, in operation, rotational movement of the ware engaging plate about an axis generally parallel to the plane of the ware engaging plate whereby, in operation, an angle of the ware engaging surface will adjust automatically to ware-to- ware variation in an angle of a surface of the wares to be handled.

The resilient mechanism may be configured to tolerate reciprocal movement of up to 30mm (e.g. 12mm) along the axis generally orthogonal to a plane of the ware engaging plate.

The handling tool may comprise a sensor for sensing if the handling tool is engaged with the surface of a ware.

The handling tool may comprise a sensor for sensing the weight of an object that the handling tool is attached to.

In one of the above examples, there is described in broad terms a handling tool for handling wares in a warewasher system, the handling tool comprising: means for engaging with a ware to be handled for releasably securing the handling tool to the ware for transport of the ware from a first location to a second location in the warewasher system; wherein the engaging means comprises: a resilient ware engaging element for mutually engaging with the ware to be handled; and means for coupling the handling tool to a vacuum source for application of at least a partial vacuum between the ware engaging element and a surface of the ware to be handled for releasably securing the ware to the handling tool during operation; and wherein the ware engaging element has a distal end, a proximal end, and a resilient wall having an external ware engaging surface; wherein the resilient wall of the ware engaging element is configured to form a shape that increases in transverse cross- sectional area in a longitudinal direction, from the distal end to the proximal end whereby, in operation, the distal end of the ware engaging element is capable of insertion into a cavity in the ware to be handled to bring the ware engaging surface into contact with a perimeter of the cavity; and wherein the resilient wall of the ware engaging element is formed of compliant resilient material that, in operation, will conform to a contour of the perimeter of the cavity and/or to any object obstructing that perimeter to form at least a partial seal for establishing at least a partial vacuum in the cavity.

The resilient wall of the ware engaging element may be configured to form a shape that is one of frustoconical; and at least partially spherical.

The resilient wall of the ware engaging element may be configured to form an aperture, at the distal end, via which said at least a partial vacuum can be applied and wherein the means for coupling may comprise a conduit for forming a fluid path from the aperture to the vacuum source. The aperture and conduit may be located generally coaxially with the ware engaging element. The aperture may have at least one of the following geometries: circular with a diameter ~25mm to ~40mm, for example ~30mm (with tolerances of plus or minus 20%); and a transverse cross- sectional area of ~500mm² to -1250mm², for example ~700mm² (with tolerances of plus or minus 40%). The resilient wall of the ware engaging element may have a geometry (e.g. wall thickness) whereby the ware engaging element is sufficiently rigid to maintain its shape when not engaged with a ware, and sufficiently compliant to conform to the perimeter of the cavity and/or to an object obstructing that perimeter during operation to releasably secure the ware to the handling tool.

The ware engaging element may have a wall thickness in the range ~2mm to ~4mm, for example ~3mm (with tolerances of plus or minus 20%). The ware engaging element may have at least one of the following geometries: circular with a diameter of ~80mm to ~140mm, for example -110mm (with tolerances of plus or minus 20%); and a transverse cross-sectional area of -5000mm² to -15000mm², for example -9500mm² (with tolerances of plus or minus 40%). The resilient wall of the ware engaging element may be configured to form a generally frustoconical shape in which the wall extends from the distal end to the proximal end at an angle of between -5° to -30° (with a tolerance of 10%) to an axis in said longitudinal direction, for example at an angle of -10° (with a tolerance of 10%) to an axis in said longitudinal direction.

The ware engaging element may be formed of a material from the list: polyurethane; silicone; or rubber. The ware engaging element may be a first ware engaging element; wherein the ware engaging means may further comprise a second ware engaging element, the second ware engaging element comprising a resilient and compliant skirt that is configured to, in operation, engage with and conform to a surface of a ware to be handled whereby to form at least a partial seal with that surface for establishing at least a partial vacuum between the skirt and the surface. The skirt may be configured to have a transverse cross-sectional area that is smaller than that of a cavity into which, in operation, the distal end of the first ware engaging element is to be inserted.

The skirt may have at least one of the following geometries: annular with an external diameter of ~30mm to ~50mm, for example ~40mm (with tolerances of plus or minus 20%); and a transverse cross-sectional area of ~700mm² to -1950mm², for example -1250mm² (with tolerances of plus or minus 40%). The skirt may depend, at a distal end of the handling tool, from at least one of: the first ware engaging element; and the coupling means. The skirt may flare outwardly, at the distal end of the handling tool, to increase in transverse cross-sectional area in a direction opposite to the longitudinal direction from the distal end of first ware engaging element to the proximal end.

The surface of the skirt may be smooth.

The handling tool may further comprise a resilient mechanism configured to allow, in operation, reciprocal movement of the handling tool along an axis generally parallel to a longitudinal axis between the distal end and proximal end of the ware engaging element whereby, in operation, the relative height of the ware engaging means will adjust automatically to ware-to-ware variation in a maximum height of wares to be handled. The resilient mechanism may be further configured to allow, in operation, rotational movement of the handling tool about an axis generally orthogonal to the longitudinal axis between the distal end and proximal end of the ware engaging element whereby, in operation, an angle of the ware engaging means will adjust automatically to ware-to-ware variation in an orientation of the wares to be handled.

The resilient mechanism may be configured to tolerate reciprocal movement of up to 30mm (e.g. 12mm) along the axis generally parallel to a longitudinal axis between the distal end and proximal end of the ware engaging element.

The handling tool may comprise a sensor for sensing if the handling tool is engaged with the surface of a ware. The handling tool may comprise a sensor for sensing the weight of an object that the handling tool is attached to.

In one of the above examples, there is described in broad terms apparatus for handling wares in a warewasher system, the apparatus comprising: means for identifying a ware to be transported from a first location to a second location in the warewasher system and for determining a position of the identified ware; means for transporting the identified ware from the first location to the second location using a handling tool broadly as set out above; and a pump for providing said vacuum source for said handling tool. In one of the above examples, there is described in broad terms a method of handling wares in a warewasher system, the method comprising: identifying a ware to be transported from a first location to a second location in the warewasher system and determining a position of the identified ware; transporting the identified ware from the first location to the second location using a handling tool broadly as set out above.

Claims

1. A warewasher system for identifying and handling wares, the system comprising: first apparatus for identifying and locating a ware to be transported from a first location to a second location in the warewasher system; and second apparatus for transporting a ware identified and located by the first apparatus from the first location to the second location; wherein the first apparatus comprises: a receiving area for receiving objects including wares to be transported; means for capturing an image of at least part of the receiving area, the image including an image part corresponding to at least part of a ware to be transported that has been received in said receiving area; means for identifying, from the captured image, the ware to be transported, and for identifying a region of the captured image that contains the image part, wherein the region has a perimeter that does not coincide with a perimeter of the image part for at least a major part the perimeter of the region; means for determining a location of the region, within the image, and for generating information representing a spatial location in the receiving area corresponding to the determined location within the image; and means for outputting the generated information representing the spatial location for use by the second apparatus as an estimated spatial location of the identified ware; wherein the second apparatus comprises: means for receiving the generated information representing the spatial location; a handling tool for engaging with, and releasably securing to, the identified ware for transporting the ware from a first location to a second location; and means for moving the handling tool to transport the ware from the first location to the second location; wherein the moving means is configured to move the handling tool, responsive to receiving the generated information, to the estimated spatial location of the identified ware; and wherein the handling tool is configured to engage with and releasably secure to the identified ware at the estimated spatial location even when the estimated spatial location deviates, as a result of the identified region having a perimeter that does not coincide with the perimeter of the image part, from a corresponding actual spatial location of the ware.

2. Apparatus for identifying and locating a ware to be transported from a first location to a second location in the warewasher system of claim 1 , the apparatus comprising: the receiving area for receiving objects including wares to be transported; the means for capturing an image of at least part of the receiving area, the image including an image part corresponding to at least part of a ware to be transported that has been received in said receiving area; the means for identifying, from the captured image, the ware to be transported, and for identifying a region of the captured image that contains the image part, wherein the region has a perimeter that does not coincide with a perimeter of the image part for at least a major part the perimeter of the region; the means for determining a location of the region, within the image, and for generating information representing a spatial location in the receiving area corresponding to the determined location within the image; and the means for outputting the generated information representing the spatial location for use by the second apparatus as an estimated spatial location of the identified ware.

3. The apparatus according to claim 2, wherein the means for determining is configured to determine, for at least some identified wares, as a result of the identified region having a perimeter that does not coincide with the perimeter of the image part, a location of the region that deviates, in a given direction, from a corresponding actual spatial location of an identified ware by at least 5% (or preferably at least 10%, 15% or 25%) of a width of the identified ware in the given direction.

4. The apparatus according to claim 2 or 3, wherein the means for determining is configured for determining the determined location of the region to be a geometric centre of the region, and for generating information representing a spatial location in the receiving area corresponding to the geometric centre for use as an estimated geometric centre of the identified ware.

5. The apparatus according to any of claims 2 to 4, wherein the means for identifying is implemented using a neural network.

6. The apparatus according to claim 5, wherein the means for determining a location is implemented as part of said neural network.

7. The apparatus according to claim 5 or 6, wherein the neural network is a neural network that has been pre-trained using at least one object other than wares of a type that are washed in the warewasher system, and wherein the neural network is adapted for use with wares of the type washed in a warewasher system, using transfer learning.

8. The apparatus according to any of claims 5 to 7, wherein the neural network is a convolutional neural network (e.g. a‘y°^u only look once’ convolutional neural network).

9. The apparatus according to any of claims 2 to 8 configured to capture and process images at a rate greater than five images per second.

10. The apparatus according to any of claims 2 to 9, wherein the means for identifying is configured to identify as said region of the captured image that contains the image part, a region that has a shape that is different to a shape of the ware (or part thereof) as captured in said image part.

1 1. The apparatus according to any of claims 2 to 10, wherein the means for identifying is configured to identify as said region of the captured image that contains the image part, a region that has a geometric centre that is not coincident with a geometric centre of the ware (or part thereof) as captured in said image part.

12. The apparatus according to any of claims 2 to 1 1 , wherein the means for identifying is configured to identify as said region of the captured image that contains the image part, a region that has an area that is at least 10% larger than an area of the image part corresponding to at least part of a ware.

13. The apparatus according to any of claims 2 to 12, wherein the means for identifying is configured to identify as said region of the captured image that contains the image part, a region that has a shape that is polygonal.

14. The apparatus according to claim 13, wherein the region has shape is a quadrilateral (e.g. a rectangle or square).

15. The apparatus according to any of claims 2 to 14, further comprising means for processing the captured image to at least one of: resize the image, alter the contrast of the image, or sharpen the image; and means for outputting the processed image for use by the means for identifying the ware to be transported and the means for determining the location of the region.

16. The apparatus according to any of claims 2 to 15, wherein the ware to be transported is an item of crockery or glassware.

17. A handling tool for transporting a ware from a first location to a second location in the warewasher system of claim 1 , the handling tool comprising means for engaging with, and releasably securing to, the identified ware for transporting the ware from the first location to the second location, wherein the means for engaging with, and releasably securing to, the identified ware are configured to engage with and secure the identified ware when the estimated spatial location deviates from the actual spatial location, as a result of the identified region having a perimeter that does not coincide with the perimeter of the image part for at least a major part the perimeter of the region.

18. The handling tool according to claim 17, wherein the means for engaging with, and releasably securing to, the identified ware comprises a resilient ware engaging surface and means for coupling to a vacuum source for applying at least a partial vacuum between the ware engaging surface and a surface of the identified ware to releasably secure the identified ware to the handling tool.

19. A method of identifying and handling wares in a warewasher system, the method comprising: capturing an image of at least part of the receiving area, the image including an image part corresponding to at least part of a ware to be transported that has been received in said receiving area; identifying, from the captured image, the ware to be transported, and for identifying a region of the captured image that contains the image part, wherein the region has a perimeter that does not coincide with a perimeter of the image part for at least a major part the perimeter of the region; determining a location of the region, within the image, and for generating information representing a spatial location in the receiving area corresponding to the determined location within the image; and outputting the generated information representing the spatial location for use by ware handling apparatus as an estimated spatial location of the identified ware; receiving, in the ware handling apparatus, the generated information representing the spatial location; using a handling tool to engage with, and releasably secure to, the identified ware for transporting the ware from a first location to a second location; and moving the handling tool to transport the ware from the first location to the second location; wherein the handling tool is moved, responsive to receiving the generated information, to the estimated spatial location of the identified ware; and wherein the handling tool engages with and releasably secures to the identified ware at the estimated spatial location even when the estimated spatial location deviates, as a result of the identified region having a perimeter that does not coincide with the perimeter of the image part, from a corresponding actual spatial location of the ware.

20. A method for identifying and locating a ware to be transported from a first location to a second location in the warewasher system of claim 1 , the method comprising: receiving objects, in the receiving area, including wares to be transported; capturing an image of at least part of the receiving area, the image including an image part corresponding to at least part of a ware to be transported that has been received in said receiving area; identifying, from the captured image, the ware to be transported, and identifying a region of the captured image that contains the image part, wherein the region has a perimeter that does not coincide with a perimeter of the image part for at least a major part the perimeter of the region; determining a location of the region, within the image, and for generating information representing a spatial location in the receiving area corresponding to the determined location within the image; and outputting the generated information representing the spatial location for use by ware handling apparatus as an estimated spatial location of the identified ware.

21. A computer implementable program product causing a programmable apparatus to perform the method of claim 19 or 20.