WO2021034190A1

WO2021034190A1 - Method and system feeding a flatwork item to a transport conveyor and/or a flatwork treating device

Info

Publication number: WO2021034190A1
Application number: PCT/NL2020/050512
Authority: WO
Inventors: Alexander Martinus VAN NIEUWENHUIJSEN; Walter Paul TEN HAGEN; Rudolf Kobel; Markus KOBEL; Christoph Hofer; Marc Johannes Hendrik Marinus VAN BOEKHOLT; Erwin Cornelis Hermanus Johannes MAASSEN
Original assignee: Laundry Robotics B.V.
Priority date: 2019-08-16
Filing date: 2020-08-14
Publication date: 2021-02-25
Also published as: EP4013575A1; JP2022544535A

Abstract

The invention relates to a system and method for feeding a flatwork item to a transport conveyor and/or a flatwork treating device. Therefore first and second corners of a flatwork item are detected and first and second grippers are used to grip a detected first or second corner of the flatwork item which can subsequently be fed to a transport conveyor and/or a flatwork treating device.

Description

Method and system feeding a flatwork item to a transport conveyor and/or a flatwork treating device

The invention relates to a method for feeding a flatwork item to a transport conveyor and/or a flatwork treating device. The invention also relates to a system for feeding a flatwork item to a transport conveyor and/or a flatwork treating device. The invention further relates to the use of such system.

Industrial laundry services are specialized in providing a full service laundry solution for, for example, hotels, restaurant, gyms, spas, schools, care and nursing homes or any further. This means that the dirty washing of such party is provided to the industrial laundry service and will be returned once clean. Therefore, batches of used or soiled laundry need to be sorted, washed, dried, optionally ironed or pressed and subsequently folded such that they can be returned to their place of use. The washing and drying process is almost fully automated by making use of a sequence of washing machines, dryers and/or presses and automated transport systems. Also folding of laundry can be automated by making use of a folding device. However, for some steps the laundry item needs to be provided to the following processing unit in a rather specific arrangement. For example the sorting of a bunch of laundry is done manually. Another example is feeding of laundry to a feed conveyor or a mangle, or the insertion of textile in a folding machine. For the latter, it is of importance that for example a sheet (or towel) is positioned such that a first side of the typically rectangular or square sheet is perpendicular to a transport direction and that two corners of the sheet are parallel to each other when inserted in two vertically positioned rollers or on a conveyor. This specific alignment is a typical requirement of conventional folding devices and further machinery used in this technical field. Since this specific orientation of sheets it is difficult, or even unachievable, to automate this step. Hence, said steps are nowadays still done manually. The overall process still having steps to be done manually is undesired for several reasons, this is for example both labor intensive and costly. Further, it is physically demanding work and also relatively time consuming, and thus a limiting factor in the overall process. Hence, there is a need for further automating of the entire system. It is a goal of the invention to provide a method and/or a system which can contribute to further automating of an industrial laundry process.

The invention provides hereto a method for feeding a flatwork item to a transport conveyor and/or a flatwork treating device, comprising the steps of: a) providing at least one neural network trained to recognize at least one corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one corner, b) providing at least one flatwork item, c) capturing at least one first image of at least part of said flatwork item, d) providing the first image to the neural network, e) receive a possible position and/or orientation of at least one corner of the flatwork item from the neural network; f) displacing a first gripper towards the determined corner of the flatwork item, g) gripping of the corner by said first gripper, h) displacing the flatwork item via the gripped corner preferably in a substantially horizontal orientation, i) retaining a distal end opposing the gripped, first corner by a clamp, such that at least one second corner of the flatwork item substantially hangs fee, j) displacing of the flatwork item over a carrying surface preferably such that the second corner is positioned upon said carrying surface, k) capturing at least one second image of at least part of the second corner of the flatwork item,

L) providing the second image to the neural network , m) receiving a possible position and/or orientation of at least one second corner of the flatwork item from the neural network, n) displacing a second gripper towards the determined second corner of said flatwork item, o) gripping of the determined second corner by said second gripper, and p) optionally feeding the flatwork item to a transport conveyor and/or a flatwork treating device.

The method according to the present invention enables gripping of a first corner and a second corner in an automated manner. As outline above, typically the first and second corner need to be substantially parallel to each other and a first side of the flatwork has to be perpendicular to a transport direction when fed to a subsequent device. By gripping of the first and second corner it is enabled that the flatwork item can be fed to a transport conveyor, a flatwork treating device and/or any further device configured for further processing or treatment of the flatwork item. The conventional manual steps of picking of flatwork item, gripping of the corners and feeding to the flatwork item can be substantially fully replaced by the method according the invention. Hence, the method can contribute to further automating of an industrial laundry process. An additional benefit thereof is that the overall efficiency of the laundry process can be improved.

The method according to the invention provides an accurate and replicable way to detect and grip subsequently a first corner and a second corner of a flatwork item. Due to the provision of a neural network trained to recognize at least one corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one corner which is configured to provide information about at least one first image and at least one second image the location and/or orientation of the corners of the flatwork item can be determined in a relatively simple and efficient manner. The use of the captured first or second image within at least one neural network, results in receiving a possible position and/or orientation of the first and second corner of the flatwork item from the neural network. Basically, the method according to the present invention makes use of image recognition in a practical manner. At first, the orientation and/or position is of at least one first corner of the flatwork item is to be determined. Once the position and/or orientation of the first corner is determined by the neural network, the first gripper is displaced and actuated to grip the corner of the flatwork item. A benefit of this method is that it enables determination of the position and/or orientation of a first corner of a flatwork item without having to arrange the flatwork item upfront. Due to the use of the trained neural network, it is of minor importance how the flatwork item is arranged. Herewith, an additional arrangement step can be omitted. The method even enables detection of a first corner of a flatwork item from a pile of multiple flatwork items. The specific gripping of the detected first corner by the first gripper can ensure that only one flatwork item is gripped. Further, the method enables that it can be ensured that a corner is detected and not just an edge or an folded part of the flatwork item. The method according to the invention provides a reliable and repeatable way to detect and grip a first corner of a flatwork item. With the term first corner, the first corner to be gripped is meant. The steps a) to g) for gripping the first corner benefit over the prior art in several respects. Methods or systems according to the state of the art typically randomly grip a flatwork item which is then hung out, such that the lowest hanging corner can be detected and gripped. This is relatively cumbersome and cannot ensure reliable and repeatable gripping of the first corner. Further, systems which make use of determination of surface profiles cannot ensure exact and replicable gripping of the corner itself. The method according to the present invention does not require that the flatwork items are marked or provided with identification means itself. A benefit of displacing the flatwork item via the gripped first corner preferably in a substantially horizontal orientation, is that a relatively controlled displacement of the flatwork item can be initiated. It is for example conceivable that the horizontal orientation is enabled by means of guiding the flatwork item through a guiding device. The flatwork item may be displaced through at least two adjacent guiding elements, such as guiding rollers. Therefore such guiding rollers can be positioned above another, such that the flatwork item can pass through in a substantially horizontal orientation. A benefit of displacing the flatwork item in a substantially horizontal position is that the flatwork item, and the movement thereof, can be more controlled compared to displacing the flatwork item in a vertical, or hanging, orientation. The substantially horizontal orientation can be sustained by retaining a distal end opposing the first corner for example by a clamp. This is typically done such that at least one second corner of the flatwork item substantially hangs free. This is the second corner which is to be gripped. It is also conceivable that aforementioned guiding rollers fulfil the retaining function of the clamp. Further the use of such clamp can ensure that the flatwork item at least temporarily can keep its substantially horizontal orientation.

For example, during displacing the flatwork item over the carrying surface. This may contribute to achieving correct positioning of the flatwork item on the carrying surface. The distal end may for example be determined by using detection means and/or visual imaging means. It is also conceivable that an air knife is used at any step of the method according to the present invention to control and/or guide the position and/or orientation of the flatwork item. Subsequently, the orientation and/or position of least one second corner of the flatwork item is determined by the neural network using the input of the second image. The position and/or orientation of the second image is in particular its position and/or orientation with respect to the carrying surface. Typically, the flatwork item is at least partially in a substantially horizontal orientation at least during the displacement and/or detecting steps. It is more accurate to determine the position and/or orientation of the second corner with respect to a carrying surface than to determine such for a freely handing second corner. Further, this may increase the speed of detecting the second corner, and subsequently the speed and ease of gripping said second corner. Also the success rate of detection and/or gripping will increase with respect to the conventional methods, and thus the repeatability.

The steps of the method according to the present invention are typically successive steps. Capturing of at least one first or second image can for example be done by visual imaging means or a visual imaging device. This can be any conventional visual imaging means, non-limiting examples thereof are a 2D camera and a 3D camera. When it is referred to a flatwork item, this can be type of conventional flatwork item. Non-limiting examples of flatwork items according to the present invention are towels, cloths, (bed) sheets, napkins, bed linen, duvets, tabletops and/or pillowcases. Such flatwork item is typically substantially rectangular or square. Typically, the flatwork item has at least one corner, preferably at least two corners, and more preferably the flatwork item has four corners. The flatwork item is typically at least partially made of a textile or fabric, which can be either woven or nonwoven. However, it is also conceivable that the flatwork item is a garment. Hence, further non-limiting examples of the flatwork item are a top, shirt, blouse, sweater, trousers and/or a dress.

With respect to the present invention, the term corner is just exemplary, this can also be any predetermined part or area of the flatwork item. Non-limiting examples thereof are an edge, corner region, shoulder part, collar, sleeve, button and/or zipper. Hence, the invention also relates to a method for detecting and gripping a predetermined part of a flatwork item, wherein the neural network is trained to recognize a predetermined part of a flatwork item in an image and to indicate the position and/or orientation of said predetermined part and wherein a possible position and/or orientation is received of said predetermined part of the flatwork item from the neural network, such that a second gripper can be displaced towards the determined corner of the flatwork item. A further possible embodiment of the method comprises comprising the steps of taking multiple images of at least one flatwork item, wherein each image shows at least one (first) corner of the flatwork item, annotating, and in particular manually annotating each image with information about a position and/or orientation of at least one corner of said flatwork item, and including the annotated images of flatwork items in a neural network. Typically these step are performed prior to the neural network being provided in the method. Hence, these steps are done prior to step a) of the method according to the present invention. Since the (manual) annotating of the images only needs to be done once, this will not significantly limit the overall process in practice. The information about a position and/or orientation of at least one corner of the flatwork item can be 2D and/or 3D related. It is also conceivable that the annotating of the images is computer-controlled. It is conceivable that no corners are observed during annotating of the images. This is also useful since it is conceivable that the flatwork item is not correctly positioned upon the carrying surface such that no corners can be visually observed.

It is conceivable that the method further comprises the steps of determining if the gripping of the determined corner by the first gripper was successful, annotating at least one first image made at step b) with information of the position and/or orientation of at least one corner and adding of the annotated image to the neural network, and/or, determining if the gripping of the second corner by the second gripper was successful, annotating at least one second image made at step k) with information of the position and/or orientation of the second corner, and adding of the annotated second image to the neural network. Annotating of the captured first image with information about the position and/or orientation and adding this annotated image to the neural network can contribute to the ability of the method to faster and/or better recognize a corner in a newly captured first image. Including the annotated first image in the neural network may further enhance the reliability of the method. The annotating of the first image may for example be done manually. However, it is also conceivable that the annotating of the first image is computer- controlled. Basically, a continuous method for determining a corner of a flatwork item can be obtained. The method may even qualify as a continuous deep learning system. Gripping of a (first) corner can for example be classified as successful if the corner is gripped by the first gripper such that the first gripper can take along the flatwork item to a further processing step. If the corner was not sufficiently gripped the first gripper may lose the corner, and thus the flatwork item. If this is detected, for example by a further visual imaging device, the gripping cannot be classified as successful. It is conceivable that the image will not be annotated and/or added to the neural network if the gripping was not successful. Hence, the annotation and addition of an image is possibly only done when the gripping is classified as successful. The way of annotating the second image and/or determining if gripping of the second corner was success can equal the way as described above for the first corner.

The method according to the invention may also comprise the provision of a first neural network trained to recognize at least one corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one corner, wherein the first neural network is used to determine a possible position and/or orientation of at least one first corner of the flatwork item, and the method comprising the provision of a second neural network trained to recognize at least one corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one corner, wherein the second neural network is used to determine a possible position and/or orientation of at least one second corner of the flatwork item.

In a preferred embodiment, the method comprises the step of shaking the flatwork item prior to step j) such that a second corner which is at shortest distance from the first corner will hang free. It is also conceivable that this step is done after step i). This step can contribute to preparation of the flatwork item before it is displaced towards or over the carrying surface. It enables better starting position of the second corner of the flatwork item before the corner is brought into position for detection and gripping. The shaking will loosen the structure of the flatwork item in case the flatwork item is for example twisted, folded or wrinkled. Under the term shaking, any vibrating or oscillating movement should be understood. Also shuddering or swinging can be meant. The purpose of the shaking movement is preparing the flatwork item for the displacement and for the identification of the second corner.

In another preferred embodiment, step o) of the method comprises the (subsequent) steps of lifting the second corner of the flatwork item via suction or the intake of air, and gripping the second corner via a enclosing at least part of the second corner by the second gripper, in particular by a clamping element of the second gripper. Hence, the second gripper preferably comprises a suction element in combination with a (movable) clamping element. Lifting of the second corner of the flatwork item via suction can for example be done by applying a vacuum. With suction, typically suction of air is meant. The suction is preferably temporarily and may be stopped once the second corner is gripped by the second gripper. The clamping element can for example be a gripping mouth. Enclosing of at least part of the second corner can provide a reliable and firm engagement of the corner, and thus the flatwork item, by the second gripper. The combination of lifting and gripping of the second corner enables that an item of flatwork, which is mostly a textile or the like, can be gripped in a controlled manner. Due to the flatwork item typically being made of a textile or fabric it is difficult to grip a corner which is positioned on a substantially flat surface, such as a carrying surface. It is preferred that the second corner is gripped via enclosing at least part of the corner by the second gripper, such that controlled further displacing of the flatwork item is enabled. If the first and second corners are gripped via enclosing at least part of each corner by the first or second gripper, controlled further displacing of the flatwork item is enabled. Hence, the method equals the steps which are typically done manually, and can therefore replace this manual steps.

Preferably, the determined first corner is gripped by enclosing at least part of said corner of the flatwork item by the first gripper, in particular by a gripping mouth of said first gripper. Hence, the first gripper, and in particular the gripping mouth of the first gripper, can provide double-sided engagement of at least part of the corner. Enclosing of at least part of the first corner can provide a reliable and firm engagement of the corner, and thus the flatwork item, by the first gripper. Due to the determination of the position and/or orientation of the first corner via the method according to the present invention it is also possible to grip a corner of a randomly arranged flatwork item, or even from a single flatwork item in a pile of flatwork items.

The determined first corner may be at least partly lifted before it is gripped, preferably via providing a flow of air. Possibly, the determined first corner is gripped by providing a flow of air over the corner which is to be gripped. The first gripper may for example be a coanda gripper which makes use of the lifting force of air in order to (temporarily) lift the flatwork item, such that the item can be gripped. It is for example conceivable that the first gripper makes use of compressed air and/or pressurized air.

A conventional gripper making use of vacuum, or the suction of air, is not usable for gripping a corner of a flatwork item. Such gripper cannot ensure that only the top layer of the flatwork item is gripped, since multiple parts of the flatwork item may engage and/or stick together due to the suction.

It is beneficial if the method comprises the steps of suction of air, or intake of air, from underneath the carrying surface such that the flatwork item is pressed against the carrying surface after step j) and preferably before step k), and stopping, or discontinuing, the suction of air after the second gripper has gripped the second corner. For example, a vacuum can be applied underneath the carrying surface. Hence, suction of air can be created, thereby drawing the flatwork item towards the carrying surface. The carrying surface is typically at least partially permeable to air. The carrying surface for example comprises at least one air opening. The vacuum, or air intake, is typically released after the second corner has been detected and gripped. In a further possible embodiment, the method comprising the step of blowing air through the carrying surface such that the flatwork item is pushed away from the carrying surface. Hence, in addition to the suction of air, it is also possible to use an air flow for guiding the flatwork item after the second corner is gripped by the second gripper. Such air flow can for example be created via at least one fan. It is conceivable that at least one air knife is used in order to assist the positioning of the flatwork item onto the carrying surface and/or when removing the flatwork item from the carrying surface.

Possibly, the method comprises the steps of providing multiple flatwork items, picking at least one flatwork item by a picker, and positioning of the flatwork item on a substantially horizontal surface. These steps are typically done prior to step b). The multiple flatwork items can for example be provided on a conveyor. However, it is also conceivable that the flatwork items are provided in a storage device or hopper. The picker typically randomly grips at least one, and preferably one, flatwork item. The picker may for example comprise a claw for picking a flatwork item. The flatwork item is preferably provided in a laying position upon said substantially horizontal surface. The flatwork item can have a free orientation. It is of importance that the flatwork item is in a non-fixed position. As described above, the method enables that pre-arrangement of the flatwork item can be omitted.

Another possible embodiment of the method comprises the steps of receiving a score for each detected first corner of at least one flatwork item from the neural network, selecting a corner to be gripped which has a score which is higher than a predetermined threshold score, and/or discarding the flatwork item if the score is lower than the threshold score. The threshold score can for example be determined based upon the extend of resemblance between the captured first image and the images of the neural network. It is for example possible that a threshold score of at least 50% resemblance is set. However, this can also be another, preferably higher, threshold score. If a corner has a score below the threshold score, the corner can be discarded. If in the captured first image, not any corner has a score above the threshold score are detected, than the flatwork item as such can be discarded. It is however also conceivable that the flatwork item is moved a bit, and that subsequently a new image is captured and the corner determination is started again. Moving of the flatwork item can for example be displacing and/or shaking of the flatwork item.

It is possible that the first gripper is controlled to grip the nearest corner when multiple corners are determined, or detected. In particular, the first gripper can be controller to grip the nearest corner of the flatwork item with a score above the predetermined threshold score when multiple corners are detected. Gripping the nearest corner can enhance the efficiency of the overall process.

In order to further improve the efficiency of the method and possibly the production capacity of the overall system, it is conceivable that multiple first grippers are applied. Hence, it is possible that a secondary first gripper takes over the first corner, in particular from the (primary) first gripper, after step g). So, steps f) and g) can for example be performed by a primary first gripper following which the first corner is taken over by a secondary first gripper. The secondary first gripper can than at least part of step h). It is also conceivable that the secondary first gripper performs step j), and optionally step p). It is however, also possible that a tertiary first gripper takes over the first corner prior to step j). The use of more than one first gripper can significantly speed up the overall process of feeding a flatwork item to a transport conveyor and/or a flatwork treating device. It may further contribute to enabling continuous application of the method and/or increasing the output of flatwork items fed to a subsequent step. The first grippers can be substantially equal in their technical construction. It is however also conceivable that the secondary first gripper and/or the tertiary first gripper has a different design.

The invention also relates to a system for feeding a flatwork item to a transport conveyor and/or a flatwork treating device, preferably via a method according to the present invention as described above, the system comprising at least one first visual imaging device configured for capturing at least one first image of at least part of a flatwork item, at least one second visual imaging device configured for capturing at least one second image of at least part of a flatwork item, at least one neural network trained to recognize at least one corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one corner, , at least one displaceable first gripper configured for gripping a first corner of a flatwork item a displaceable second gripper configured gripping a second corner of the flatwork item, and a carrying surface configured for carrying at least part of a flatwork item. The system according to the present invention experiences the same or similar benefits as described for the method according to the present invention.

It is possible that at least one first gripper comprises a gripping mouth configured to enclose at least part of a corner of the flatwork item. The gripping mouth can be configured to clasp the corner of the flatwork item at a predetermined distance from at least one edge of the corner and preferably at a predetermined distance of both edges of the corner. Hence, a repeatable and reliable gripping of the corner of the flatwork item can be obtained. It is also possible that multiple first grippers are applied, for example a primary first gripper, a secondary first gripper and a tertiary first gripper. It is conceivable that all first grippers comprise a gripping mouth. Preferably the gripping mouth is configured such that it does not damage the flatwork item during gripping and/or displacing. The first gripper may also be configured for providing a flow of air. Hence, the determined corner can be gripped by providing a flow of air over the corner which is to be gripped. The first gripper may for example be a coanda gripper which makes use of the lifting force of air in order to (temporarily) lift the flatwork item, such that the item can be gripped. It is for example conceivable that the first gripper makes use of compressed air and/or pressurized air.

In a preferred embodiment the system further comprises a drive unit for displacing the first gripper towards the location determined by the neural network.

At least one first gripper, and in particular the gripping mouth, is preferably rotatable over at least 180 degrees, preferably 360 degrees, over an axis of rotation. This will result in the gripper having a relatively large degree of freedom. It also provides more flexibility in which corners can be reached and gripped by the first gripper.

The second gripper of the system is preferably configured for lifting at least part of the second corner before gripping said corner. Preferably, the second gripper comprises a suction element in combination with movable clamping element. The second gripper may for example comprise at least one suction element configured for at least partially lifting an edge and/or a corner of a flatwork item, and at least one clamping element for clampingly engaging said flatwork item, wherein the clamping element is configured to enclose at least part of an edge and/or a corner of the flatwork item.

It is beneficial if the carrying surface is at least partially permeable to air. The carrying surface can for example comprises at least one air opening and/or may be at least partially configured by a mesh or mesh structure. Hence, the carrying surface can enable that a flow of air through the carrying surface can be applied. This can be either attract (draw or pull) the flatwork item towards the carrying surface, or push the flatwork item away from it. This can be beneficial for controlling the motion or motionlessness of the flatwork item. Typically, the carrying surface is at least partially substantially flat. A substantially flat carrying surface can provide improved support of the flatwork item.

The system may further comprise at least one clamp configured to hold a distal end of the flatwork item opposing the first corner. The clamp can be any conventional clamp configured to hold or clamp part of a flatwork item. The function of the clamp is guiding the distal end of the flatwork during use of the system. The clamp can for example be a claw.

The first gripper and the second gripper are preferably independently movable and/or controllable. This may further improve the efficiency of the entire system. The first gripper, and in particular the gripping mouth, and/or the second gripper, and in particular the clamping element thereof, is preferably rotatable over at least 180 degrees, preferably 360 degrees, over an axis of rotation. This will result in the gripper(s) having a relatively high degree of freedom. The system may comprise at least one drive unit to drive one or more grippers and/or the clamp.

The first visual imaging device and/or the second visual imaging device may comprise a camera. The visual imaging device can be any conventional visual imaging means, non-limiting examples thereof are a 2D camera and a 3D camera.

Possibly the system comprises at least one air knife. An air knife can be applied in order to control and/or guide the position and/or orientation of the flatwork item in a relatively simple and reliable manner.

The invention also relates to use of a system according to the present invention.

A difficulty for further automating abovementioned laundry process is that it is difficult to develop a device which can detect and grab a corner of a piece of laundry, such as a flatwork item, in a controlled and repeatable manner. Distinguishing corners and edges appears to be difficult for automated system and this also applies to subsequent gripping of the detected edge or corner. Hence, it is a goal of the invention to provide a method and/or a system which can contribute to further automating of an industrial laundry process.

The invention provides hereto a method for detecting and gripping a predetermined part of a flatwork item, such as a first corner or a first edge of a flatwork item, in particular as described in clause 1 , comprising the steps of providing a neural network trained to recognize at least one corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one corner, providing at least one flatwork item, capturing at least one first image of at least part of said flatwork item, providing the first image to the neural network, receiving a possible position and/or orientation of at least one predetermined part of the flatwork item, in particular a corner of the flatwork item from the neural network; displacing a first gripper towards the determined part, in particular corner of the flatwork item, and gripping of the part, in particular corner by said first gripper.

Due to the provision of a neural network trained to recognize at least one predetermined part such but not limited to a corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one corner which is configured to provide information of at least one first image made during step C1) of the method according to the present invention, it is possible to determine the position, or location, and/or orientation of at least one corner of the flatwork item. The use of the captured first image within the neural network, results in receiving a possible position and/or orientation of at least one corner of the flatwork item (provided at step B1) ) from the neural network. Basically, the method according to the present invention makes use of image recognition and/or classification in a practical manner. The neural network is hereby trained to determine the orientation and/or position of at least one corner of the flatwork item which is to be determined. Once information regarding the position and/or orientation of the corner is provided by the neural network, the first gripper is displaced and actuated to grip the corner of the flatwork item. A benefit of this method is that it enables determination of the position and/or orientation of a corner of a flatwork item without having to arrange the flatwork item upfront. Due to the use of the trained neural network, it is of minor importance how the flatwork item is arranged. Herewith, an additional arrangement step can be omitted. The method even enables detection of a corner of a flatwork item from a pile of multiple flatwork items. The specific gripping of the detected corner by the first gripper can ensure that only one flatwork item is gripped. Further, the method enables that it can be ensured that a corner is detected and not just an edge or an folded part of the flatwork item. The method according to the invention provides a reliable and repeatable way to detect and grip a corner of a flatwork item. Said corner is in particular a first corner, which means the first corner to be gripped. The method benefits over the prior art in several respects. Methods or systems according to the state of the art typically randomly grip a flatwork item which is then hung out, such that the lowest hanging corner can be detected and gripped. This is relatively cumbersome and cannot ensure reliable and repeatable gripping of the corner. Further, systems which make use of determination of surface profiles cannot ensure exact and replicable gripping of the corner itself. The method according to the present invention does not require that the flatwork items are marked or provided with identification means itself.

In a preferred embodiment it is conceivable that the method further comprises the steps of taking multiple images of at least one flatwork item, wherein each image shows at least one flatwork item, (manually) annotating each image with information about a position and/or orientation of at least one corner and preferably all corners of said flatwork item, and including the annotated images of flatwork items in a neural network. Typically these step are performed prior to the neural network being provided in the method. Hence, these steps are done prior to step a) of the method according to the present invention. Since the annotating of the images, which may be done manually, only needs to be done once, this will not significantly limit the overall process in practice. It is also conceivable that the annotating of the first image is computer-controlled. It is conceivable that no corners are observed during annotating of the images. This is also useful since it is conceivable that the physical flatwork item which is to be determined is in a position that no corners are visually observable.

It is also possible that the method comprises the steps of determining if the gripping of the determined corner by the first gripper was successful, annotating at least one first image made at step C1) with information of the position and/or orientation of at least one corner, and adding of the annotated image to the neural network. Annotating of the captured first image with information about the position and/or orientation and adding this annotated image to the neural network can contribute to the ability of the method to faster and/or better recognize a corner in a newly captured first image. Including the annotated first image in the neural network may further enhance the reliability of the method. The annotating of the first image may for example be done manually. However, it is also conceivable that the annotating of the first image is computer-controlled. Basically, a continuous method for determining a corner of a flatwork item can be obtained. The method may even qualify as a continuous deep learning system. Gripping of a (first) corner can for example be classified as successful if the corner is gripped by the first gripper such that the first gripper can take along the flatwork item to a further processing step. If the corner was not sufficiently gripped the first gripper may lose the corner, and thus the flatwork item. If this is detected, for example by a further visual imaging device, the gripping cannot be classified as successful. It is conceivable that the image will not be annotated and/or added to the neural network if the gripping was not successful. Hence, the annotation and addition of an image is possibly only done when the gripping is classified as successful.

The determined corner may be at least partly lifted before it is gripped, preferably via providing a flow of air. Possibly, the determined part or corner is gripped by providing a flow of air over the corner which is to be gripped. The first gripper may for example be a coanda gripper. The first gripper may for example be a coanda gripper which makes use of the lifting force of air in order to (temporarily) lift the flatwork item, such that the flatwork item, and in particular the determined part thereof, can be gripped. It is for example conceivable that the first gripper makes use of compressed air and/or pressurized air. A conventional gripper making use of vacuum, or the suction of air, is not usable for gripping a corner of a flatwork item. Such gripper cannot ensure that only the top layer of the flatwork item is gripped, since multiple parts of the flatwork item may engage and/or stick together due to the suction.

Preferably, the determined corner is gripped by enclosing at least part of said corner of the flatwork item by the first gripper, in particular by a gripping mouth of said first gripper. Hence, the first gripper, and in particular the gripping mouth of the first gripper, can provide double-sided engagement of at least part of the corner. Enclosing of at least part of the corner can provide a reliable and firm engagement of the corner, and thus the flatwork item, by the first gripper. Due to the determination of the position and/or orientation of the corner via the method according to the present invention it is also possible to grip a corner of a randomly arranged flatwork item, or even from a single flatwork item in a pile of flatwork items.

It is conceivable that the flatwork item is provided in a laying position at step B1). This means for example that the flatwork item is provided on a substantially horizontal surface. The flatwork item can have a free orientation. It is also conceivable that the flatwork item is provided in a substantially horizontal orientation at step B1). It is of importance that the flatwork item is in a non-fixed position. As described above, the method enables that pre-arrangement of the flatwork item can be omitted.

In a possible embodiment, the method further comprises the step of displacing the flatwork item via the gripped corner in a substantially horizontal orientation. This is in particular done after step G1). It is for example conceivable that the horizontal orientation is enabled by means of guiding the flatwork item through a guiding device. The flatwork item may be displaced through at least two adjacent guiding elements, such as guiding rollers. Therefore such guiding rollers can be positioned above another, such that the flatwork item can pass through in a substantially horizontal orientation. A benefit of displacing the flatwork item in a substantially horizontal position is that the flatwork item, and the movement thereof, can be more controlled compared to displacing the flatwork item in a vertical, or hanging, orientation.

A further possible embodiment of the method according to the present invention comprises the steps of receiving a score for each corner of at least one flatwork item (which is captured during step C1) ) from the neural network, selecting a corner to be gripped which has a score which is higher than a predetermined threshold score, and/or discarding the flatwork item if the score is lower than the threshold score. The threshold score can for example be determined based upon the extend of resemblance between the captured first image and the images of the neural network. It is for example possible that a threshold score of at least 50% resemblance is set. However, this can also be another, preferably higher, threshold score. If a corner has a score below the threshold score, the corner can be discarded. If in the captured first image, no corners having a score above the threshold score are detected, than the flatwork item as such can be discarded. It is however also conceivable that the flatwork item is moved a bit, and that subsequently a new image is captured and the corner determination is started again. Moving of the flatwork item can for example be displacing and/or shaking of the flatwork item.

The invention also relates to a system for detecting and gripping a first corner of a flatwork item, preferably via a method as described above, the system comprising at least one visual imaging device configured for capturing at least one first image of at least part of a flatwork item, a neural network trained to recognize at least one corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one corner, and a displaceable first gripper configured for gripping a corner of a flatwork item. The system according to the present invention can experience the same benefits and embodiments as described above for the corresponding method according to the present invention.

The first gripper preferably comprises a gripping mouth configured to enclose at least part of a corner of the flatwork item. The gripping mouth can be configured to clasp the corner of the flatwork item at a predetermined distance from at least one edge of the corner and preferably at a predetermined distance of both edges of the corner. Hence, a repeatable and reliable gripping of the corner of the flatwork item can be obtained. Preferably the gripping mouth is configured such that it does not damage the flatwork item during gripping and/or displacing. The first gripper may also be configured for providing a flow of air. Hence, the determined corner can be gripped by providing a flow of air over the corner which is to be gripped. The first gripper may for example be a coanda gripper which makes use of the lifting force of air in order to (temporarily) lift the flatwork item, such that the item can be gripped.

It is for example conceivable that the first gripper makes use of compressed air.

In a preferred embodiment, the system further comprises at least one drive unit for displacing the first gripper towards the location determined by the neural network. The first gripper, and in particular the gripping mouth, is preferably rotatable over at least 180 degrees, preferably 360 degrees, over an axis of rotation. This will result in the gripper having a relatively large degree of freedom. It also provides more flexibility in which corners can be reached and gripped by the first gripper. The visual imaging device of the system may for example be a camera. The visual imaging device can be any conventional visual imaging means, non-limiting examples thereof are a 2D camera and a 3D camera. Another difficulty for further automating the laundry process as described above is that it is difficult to develop a device which can detect and grab a second corner of a piece of laundry, such as a flatwork item, in a controlled and repeatable manner once a first corner is yet detected and/or grabbed. Distinguishing corners and edges appears to be difficult for automated system and this also applies to subsequent gripping of the detected edge or corner. It is known to identify a second or third corner of a flatwork item once one or two corners are already identified. The flatwork item can for example be in a hanging position, such that the freely hanging corner can be detected and gripped. However, gripping a free hanging corner cannot be done with a very high accuracy. Hence, it is a goal of the invention to provide an improved method to identify a second or third corner of a flatwork item in order to contribute to further automating of an industrial laundry process.

The invention provides thereto a method for detecting and gripping a predetermined part of a flatwork item, such as a second corner of a flatwork item, in particular according to clause 16, the method comprising the steps of providing a neural network trained to recognize at least one corner of a flatwork item in an image and trained to indicate the position and/or orientation of said at least one corner, providing a flatwork item which is clamped at a first corner by a first gripper, and which is preferably held at a distal end opposing the first corner by a clamp, such that at least one second corner substantially hangs fee, displacing of the flatwork item over a carrying surface preferably such that the second corner is positioned upon said carrying surface, capturing at least one second image of at least part of the second corner of the flatwork item, providing the second image to the neural network, receiving a possible position and/or orientation of at least one second corner of the flatwork item (provided at step b) ) from the neural network; displacing a second gripper towards the determined second corner of said flatwork item, and gripping of the determined second corner by said second gripper.

The method according to the invention provides an accurate and replicable way to detect and grip a second corner of a flatwork item. Due to the provision of a neural network trained to recognize at least one corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one corner which is configured to provide information of at least one second image made during step d), the location and/or orientation of the second corner of the flatwork item can be determined in a relatively simple and efficient manner. The use of the captured second image within the neural network, results in receiving a possible position and/or orientation of the second corner of the flatwork item (provided at step B2) ) from the neural network.. Basically, the method according to the present invention makes use of image recognition and/or classification in a practical manner. The neural network is trained to determine the orientation and/or position of least one second corner of the flatwork item. The position and/or orientation of the second image is in particular its position and/or orientation with respect to the carrying surface. Typically, the flatwork item is at least partially in a substantially horizontal orientation. It is more accurate to determine the position and/or orientation of the second corner with respect to a carrying surface than to determine such for a freely handing second corner. Further, this may increase the detection speed, and subsequently the speed and ease of gripping. Also the success rate of detection and/or gripping will increase with respect to the conventional methods, and thus the repeatability.

Preferably the flatwork item is held at a distal end opposing the first corner by a clamp. When the first corner is clamped by a first gripper and the distal end of the flatwork item is clamped as well at least one second corner will typically hang free. Further the use of such clamp can ensure that the flatwork item at least temporarily can keep its substantially horizontal orientation. For example, during displacing the flatwork item over the carrying surface. This may contribute to achieving correct positioning of the flatwork item on the carrying surface. It is also conceivable that an air knife is used at any step of the method according to the present invention to control and/or guide the position and/or orientation of the flatwork item. The second gripper is also configured to release the second corner. The steps of the method according to the present invention are typically successive steps. Capturing of at least one second image can for example be done by visual imaging means or a visual imaging device. This can be any conventional visual imaging means, non limiting examples thereof are a 2D camera and a 3D camera.

A further embodiment of the method is conceivable, wherein the method comprises the step of displacing the first gripper and the second gripper with respect to each other, preferably such that the first and second corners of the flatwork item are substantially aligned or parallel to another. With this step the flatwork item can be prepared for further processing step in a relatively simple manner. Mutual displacement of the first and second gripper in a direction facing away from another enables alignment of the first and second corner of the flatwork item. Subsequently via this alignment of the first and second corner, a first edge, or first side, of the flatwork item situated between said first and second corner can be stretched.

Hence, the flatwork item is prepared for several further processing steps. The flatwork item can for example be brought in a position wherein it can be inserted in a further device, such as a conveyor, wherein the first edge is substantially perpendicular to the transport direction. The first and second gripper are typically independently controllable, which can further improve the efficiency of the entire method and/or system.

It is also possible that the method further comprises the steps of taking multiple images of at least one flatwork item preferably wherein the flatwork item is positioned upon a carrying surface, wherein each image shows at least one corner of the flatwork item, annotating, and in particular manually annotating each image with information about a position and/or orientation of at least one corner of said flatwork item, and including the annotated images of flatwork items in a neural network. Typically, these steps are done prior to the provision of the neural network. Manual annotating of images generally only needs to be done once, wherefore this will not significantly limit the overall process in practice. The information about a position and/or orientation of at least one corner of the flatwork item can be 2D and/or 3D related. It is also conceivable that the annotating of the first image is computer-controlled. It is conceivable that no corners are observed during annotating of the images. This is also useful since it is conceivable that the flatwork item is not correctly positioned upon the carrying surface such that no corners can be visually observed.

The method may further comprise steps wherein it is determined if gripping of the second corner by the second gripper was successful, annotating at least one image made at step D2) with information of the position and/or orientation of the second corner, and adding of the annotated second image to the neural network.

Preferably annotating of an image is only done when the gripping of the second was classified as successful. Annotating of the captured first image with information about the position and/or orientation and adding this annotated image to the neural network can contribute to the ability of the method to faster and/or better recognize a corner in a newly captured second image. Including the annotated second image in the neural network may further enhance the reliability of the method. The annotating of the second image may for example be done manually. However, it is also conceivable that the annotating of the first image is computer-controlled. Basically, a continuous method for determining a second corner of a flatwork item can be obtained. The method may even qualify as a continuous deep learning system. Gripping of a second corner can for example be classified as successful if the corner is gripped by the second gripper such that the second gripper can take along the flatwork item to a further processing step, preferably in co-acting with the first gripper holding the first corner. If the second corner was not sufficiently gripped the second gripper may lose the corner. If this is detected, for example by a further visual imaging device, the gripping cannot be classified as successful. The entire flatwork item may then be discarded. It is conceivable that the second image will not be annotated and/or added to the neural network if the gripping was not successful. Hence, the annotation and addition of an image is possibly only done when the gripping can be classified as successful. Displacing of the flatwork item over a carrying surface preferably such that the second corner is positioned upon said carrying surface, can be performed by displacing the first corner at least via the first gripper. The first gripper is preferably displaced substantially parallel to the carrying surface, in order to achieve a substantially horizontal orientation of at least part of the flatwork item. It is also conceivable that the displacement is achieved via displacing the first corner via the first gripper and via displacing the distal end of the flatwork item via the clamp. With the latter method a more controlled displacement of the flatwork item may be achieved. Further this enables a substantially horizontal orientation of the flatwork item during displacing thereof.

Preferably, at least part of the flatwork item is in substantially horizontal orientation during the detecting and/or gripping of the second corner. The flatwork item being in a substantially horizontal orientation during detection and/or gripping of the second corner can further simplify the detection and/or gripping of said second corner. The detection and gripping of a corner when the flatwork item is in a substantially horizontal orientation, for example because it is supported by the carrying surface, is easier than detection and gripping of a hanging corner due to a reduced degree of freedom.

It is further possible that the method comprises the step of determining the distance between the first corner and the second corner after step C2), and discarding the flatwork item if the distance does not exceed a predetermined threshold distance, and/or selecting the second corner which is at the closest distance to the first corner for gripping by the second gripper if more than one second corner is detected. It is possible that during more than one second corner is identified in the second image. This can for example be the situation when the flatwork item is substantially square and/or quadratic and/or relatively small. If two corners are detected, the corner closest to the first corner should be depicted. This means generally that the shortest side of the flatwork item is identified. If the determined distance does not exceed the predetermined distance, it may be possible that instead of a second corner the first corner is accidentally detected. It is also possible that something went wrong during displacing of the flatwork item. A non limiting measure for the threshold distance is for example 200 mm or lower, preferably 15 mm or lower and more preferably 100mm or lower.

It is further possible, that prior to step C2), the step of shaking the flatwork item prior such that at least a second corner which is at shortest distance from the first corner will hang free is applied. This step can contribute to preparation of the flatwork item before it is displaced towards or over the carrying surface. It enables better starting position of the second corner of the flatwork item before the corner is brought into position for detection and gripping. The shaking will loosen the structure of the flatwork item in case the flatwork item is for example twisted, folded or wrinkled. Under the term shaking, any vibrating or oscillating movement should be understood. Also shuddering or swinging can be meant. The purpose of the shaking movement is preparing the flatwork item for the displacement and for the identification of the second corner.

In a preferred embodiment of the method according the invention, step H2) comprises the (subsequent) steps of lifting the second corner of the flatwork item via suction, and gripping the second corner via a enclosing at least part of the second corner by the second gripper, in particular by a clamping element of the second gripper. Hence, the second gripper preferably comprises a suction element in combination with a (movable) clamping element. Lifting of the second corner of the flatwork item via suction can for example be done by applying a vacuum. With suction, typically suction of air is meant. The suction is preferably temporarily and may be stopped once the second corner is gripped by the second gripper. The clamping element can for example be a gripping mouth. Enclosing of at least part of the second corner can provide a reliable and firm engagement of the corner, and thus the flatwork item, by the second gripper. The combination of lifting and gripping of the second corner enables that an item of flatwork, which is mostly a textile or the like, can be gripped in a controlled manner. Due to the flatwork item typically being made of a textile or fabric it is difficult to grip a corner which is positioned on a substantially flat surface, such as a carrying surface. It is preferred that the second corner is gripped via enclosing at least part of the corner by the second gripper, such that controlled further displacing of the flatwork item is enabled. If the first and second corners are gripped via enclosing at least part of each corner by the first or second gripper, controlled further displacing of the flatwork item is enabled. Hence, the method equals the steps which are typically done manually, and can therefore replace this manual steps.

In a beneficial embodiment of the method, the method further comprises the steps of suction of air from underneath the carrying surface such that the flatwork item is pressed against the carrying surface after step C2) and preferably before step d), and stopping, or discontinuing, the suction of air after the second gripper has gripped the second corner. For example, a vacuum can be applied underneath the carrying surface. Hence, suction of air can be created, thereby drawing the flatwork item towards the carrying surface. The carrying surface is typically at least partially permeable to air. The carrying surface for example comprises at least one air opening. The vacuum, or suction of air, is typically released after the second corner has been detected and gripped. In a further possible embodiment, the method comprising the step of blowing air through the carrying surface such that the flatwork item is pushed away from the carrying surface. Hence, in addition to the suction of air, it is also possible to use an air flow for guiding the flatwork item after the second corner is gripped by the second gripper. Such air flow can for example be created via at least one fan.

Possibly the method also comprises the step of feeding the flatwork item to a transport conveyor and/or a flatwork treating device. A non-limiting example thereof is a folding machine and/or a mangle.

The invention also relates to a system for detecting and gripping a second corner of a flatwork item, preferably via a method as described above, the system comprising at least one visual imaging device configured for capturing at least one second image of at least part of a flatwork item, a neural network trained to recognize at least one corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one corner, , a displaceable first gripper configured for gripping a first corner of a flatwork item, a displaceable second gripper configured gripping a second corner of the flatwork item, and a carrying surface configured for carrying at least part of a flatwork item. The system according to the invention experiences the same benefits as described above for the corresponding method. Hence, the system enables the detection and gripping of a second corner of a flatwork item in a controlled and repeatable manner.

The system may further comprise at least one clamp configured to hold a distal end of the flatwork item opposing the first corner. The clamp can be any conventional clamp configured to hold or clamp part of a flatwork item. The function of the clamp is guiding the distal end of the flatwork during use of the system. The clamp can for example be a claw. The first gripper and the second gripper are preferably independently movable and controllable. This may further improve the efficiency of the entire system. The first gripper, and in particular the gripping mouth, and/or the second gripper, and in particular the clamping element thereof, is preferably rotatable over at least 180 degrees, preferably 360 degrees, over an axis of rotation. This will result in the gripper(s) having a relatively high degree of freedom. The system may comprise at least one drive unit to drive one or more grippers and/or the clamp. The visual imaging device of the system may for example be a camera. The visual imaging device can be any conventional visual imaging means, non-limiting examples thereof are a 2D camera and a 3D camera. The invention also relates to the use of a system according to the present invention.

The invention also relates to a gripper for gripping an edge and/or corner of a flatwork item, the gripper comprising a suction element configured for at least partially lifting an edge and/or a corner of a flatwork item, and a clamping element for clampingly engaging said flatwork item, wherein the clamping element is configured to hold and/or enclose at least part of said edge and/or a corner of the flatwork item. Such gripper is configured for and suitable for use in a method and system according to the present invention. The suction element is typically configured for suction of air. Due to the suction of air, at least part of the flatwork item can be drawn towards the suction element and at least temporarily retained. The suction element and the clamping element are preferably mutually displaceable. It is for example conceivable that the clamping element is pivotable with respect to the suction element. After the suction element has lifted at least part of the flatwork item, the clamping element can be displaced and used in order to clampingly engage the edge and/or corner of the flatwork item. The air intake of the suction element can be discontinued once a clamping engagement is established. When it is referred to a flatwork item, this can be type of conventional flatwork item. A flatwork item is typically a flat workpiece. Non-limiting examples of flatwork items according to the present invention are towels, cloths, (bed) sheets, napkins, bed linen, duvets, tabletops and/or pillowcases. Such flatwork item is typically substantially rectangular or square. Typically, the flatwork item has at least one corner, preferably at least two corners, and more preferably the flatwork item has four corners. The flatwork item is typically at least partially made of a textile or fabric, which can be either woven or nonwoven. However, it is also conceivable that the flatwork item is a garment. Hence, further non-limiting examples of the flatwork item are a top, shirt, blouse, sweater, trousers and/or a dress.

With respect to the present invention, the term corner is just exemplary, this can also be any predetermined part or area of the flatwork item. Non-limiting examples thereof are an edge, corner region, shoulder part, collar, sleeve, button and/or zipper. Where the term corner is used, also the term edge, corner region, shoulder part, collar, sleeve, button and/or zipper can be used. Hence, the invention also relates to a method for detecting and gripping a predetermined part of a flatwork item, wherein the neural network is trained to recognize a predetermined part of a flatwork item in an image and to indicate the position and/or orientation of said predetermined part and wherein a possible position and/or orientation is received of said predetermined part of the flatwork item from the neural network, such that a first gripper can be displaced towards the determined corner of the flatwork item.

The invention also relates to a gripper, in particular a first gripper, as described in the present document. This gripper is preferably configured to provide a laminar airflow. Such laminar airflow is preferably configured to temporarily lift at least part of a textile and/or flatwork item. This can be explained by the velocity of the airflow and the difference in air pressure, such that basically a coanda-effect is generated. At least part of the textile is typically lifted up in the direction of the provided airflow. Due to the lifting force of the laminar airflow at least part of the item will be lifted such that the item can be gripped in a relatively easy manner. As soon, as the part of the linen is where we want to grab it, we will clamp it and stop the airflow. The gripper typically comprises an air inlet for the intake of air, in particular pressurized air. Preferably the air inlet is provided at an upper region of the gripper such that the (compressed) air is let into the interior of the gripper from a top side of the gripper. The gripper, and in particular the interior of the gripper, preferably comprises an internal structure configured to enable air distribution. The gripper typically comprises at least one air outlet. At least one air outlet may for example comprise a distribution section, which may be substantially square, circular and/or rectangular. A non-limiting example is an air outlet which having a distribution section which has a width between 25 and 75 mm, for example of about 50mm.

The air opening of the air outlet can for example have a diameter in the range of 0.1 and 0.3 mm, for example of about 0.2mm. The air opening is preferably relatively small. Air is typically let out of the air opening at a relatively high velocity. The gripper, and in particular the air outlet, preferably comprises at least one guiding surface configured for a controlled outflow of air. The gripper may further comprise a gripping mouth which is configured for clampingly gripping at least part of a textile and/or flatwork item. The gripper, and in particular the gripping mouth, may optionally comprise at least one boundary element configured to at least partially interrupt the airflow. It is for example conceivable that the gripper comprises at least two opposing boundary elements, wherein the boundary elements are configured to at least partially interrupt the airflow and/or to guide the airflow. The gripper may also be provided with an external structure configured to enable air distribution. It is conceivable that the gripper comprises at least one turbulator or turbulator element, which is in particular configured to turn a laminar boundary layer into a turbulent boundary layer. At least one turbulator can be applied in order to prevent flutter of at least part of the textile in the airflow and/or to ensure that at least part of the textile follows the airflow in a stable manner. The dimension of such turbulator(s) typically depend on the type of textile, the predetermined part of the textile which is to be gripped and/or the associated speed of the airflow. The purpose of the turbulator(s) is to create a turbulent boundary layer at a locations near or at the gripper where fluttering of the textile and/or flatwork item should be prevented.

The described gripper can be used in combination with a systems or method according to the present invention. The present invention will be explained further on the basis of the nonlimiting embodiment examples presented in the following clauses.

1 . Method for detecting and gripping a predetermined part of a flatwork item, such as a corner, comprising the steps of:

A1 ) providing a neural network trained to recognize at least one predetermined part of a flatwork item, in particular a corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one predetermined part of a flatwork item, in particular said at least one corner, B1 ) providing at least one flatwork item,

C1) capturing at least one first image of at least part of said flatwork item,

D1) providing the first image to the neural network,

E1) receiving a possible position and/or orientation of at least one predetermined part, in particular at least one corner of the flatwork item from the neural network;

F1) displacing a first gripper towards the determined part, in particular the determined corner of the flatwork item, and

G1) gripping of the predetermined part, in particular corner by said first gripper. 2. Method according to clause 1 , further comprising the steps of:

H1) taking multiple images of at least one flatwork item, wherein each image shows at least one flatwork item,

11) annotating each image with information about a position and/or orientation of at least one predetermined part of a flatwork item, in particular a corner and preferably all corners of said flatwork item, and

J1) including the annotated images of flatwork items in a neural network.

3. Method according to clause 1 or 2, further comprising the steps of:

K1 ) determining if the gripping of the determined part, in particular the corner by the first gripper was successful,

L1) annotating at least one first image made at step C1) with information of the position and/or orientation of at least one predetermined part, in particular at least one corner,

M1 ) adding of the annotated image to the neural network. 4. Method according to any of the previous clauses, wherein the determined part, and in particular the determined corner, is at least partly lifted before it is gripped, preferably via providing a flow of air.

5. Method according to any of the previous clauses, wherein the determined part, in particular the determined corner, is gripped by enclosing at least part of said part, in particular said corner, of the flatwork item by the first gripper, in particular by a gripping mouth of said first gripper.

6. Method according to any of the previous clauses, wherein the flatwork item is provided in a laying position at step B1).

7. Method according to any of the previous clauses, further comprising the step of:

N1) displacing the flatwork item via the gripped part, in particular corner, in a substantially horizontal orientation.

8. Method according to any of the previous clauses, further comprising the steps of:

01) receiving a score of each predetermined part, in particular each corner of at least one flatwork item from the neural network,

P1 ) selecting a predetermined part, in particular a corner to be gripped which has a score which is higher than a predetermined threshold score, and/or Q1) discarding the flatwork item if the score is lower than the threshold score.

9. Method according to any of the previous clauses, wherein the first gripper is controlled to grip the nearest predetermined part, in particular corner when multiple predetermined parts, in particular corners are determined.

10. System for detecting and gripping a predetermined part of a flatwork item, such as a first corner of a flatwork item, preferably via a method according to any of clauses 1-9, comprising: at least one visual imaging device configured for capturing at least one first image of at least part of a flatwork item, a neural network trained to recognize at least one predetermined part, in particular a corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one predetermined part, in particular said corner, and a displaceable first gripper configured for gripping a predetermined part, in particular a corner of a flatwork item.

11. System according to clause 10, wherein the first gripper comprises a gripping mouth configured to enclose at least part of a predetermined part, in particular a corner of the flatwork item.

12. System according to any of clauses 10 or 11 , further comprising a drive unit for displacing the first gripper towards the location determined by the neural network.

13. System according to any of clauses 10-12, wherein the first gripper, and in particular the gripping mouth, is rotatable over at least 180 degrees, preferably 360 degrees, over an axis of rotation.

14. System according to any of clauses 10-13, wherein the visual imaging device comprises a camera.

15. Use of a system according to any of clauses 10-14.

16. Method for detecting and gripping a predetermined part of a flatwork item, such as a second corner, comprising the steps of:

A2) providing a neural network trained to recognize at least one corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one corner,

B2) providing a flatwork item which is clamped at a first corner by a first gripper, and which is preferably held at a distal end opposing the first corner by a clamp, such that at least one second corner substantially hangs fee,

C2) displacing of the flatwork item over a carrying surface preferably such that the second corner is positioned upon said carrying surface,

D2) capturing at least one second image of at least part of the second corner of the flatwork item E2) providing the second image to the neural network,

F2) receiving a possible position and/or orientation of at least one second corner of the flatwork item from the neural network;

G2) displacing a second gripper towards the determined second corner of said flatwork item, and

H2) gripping of the determined second corner by said second gripper.

17. Method according to clause 16, further comprising the step of:

I2) displacing the first gripper and the second gripper with respect to each other.

18. Method according to clause 16 or 17, further comprising the steps of:

J2) taking multiple images of at least one flatwork item preferably wherein the flatwork item is positioned upon a carrying surface, wherein each image shows at least one corner of the flatwork item,

K2) annotating each image with information about a position and/or orientation of at least one corner of said flatwork item, and

L2) including the annotated images of flatwork items in a neural network.

19. Method according to any of the previous clauses, further comprising the steps of:

M2) determining if the gripping of the second corner by the second gripper was successful,

N2) annotating at least one image made at step D2) with information of the position and/or orientation of the second corner, and

02) adding of the annotated second image to the neural network.

20. Method according to any of the previous clauses, wherein step C2) is performed at least via displacing the first corner via the first gripper.

21 . Method according to any of the previous clauses, wherein at least part of the flatwork item is in substantially horizontal orientation during the detecting and/or gripping of the second corner. 22. Method according to any of the previous clauses, comprising the step of: P2)determining the distance between the first corner and the second corner after step C2), and

Q2) discarding the flatwork item if the distance does not exceed a predetermined threshold distance, and/or R2) selecting the second corner which is at the closest distance to the first corner for gripping by the second gripper if more than one second corner is detected.

23. Method according to any of the previous clauses, comprising the step of: S2) shaking the flatwork item prior to step C2) such that a second corner which is at shortest distance from the first corner will hang free.

24. Method according to any of the previous clauses, wherein step H2) comprises the steps of:

H2a) lifting the second corner of the flatwork item via suction, and H2b) gripping the second corner via a enclosing at least part of the second corner by the second gripper, in particular by a clamping element of the second gripper.

25. Method according to any of the previous clauses, further comprising the steps of:

T2) suction of air from underneath the carrying surface such that the flatwork item is pressed against the carrying surface after step C2) and preferably before step D2), and

U2) stopping the suction of air after the second gripper has gripped the second corner.

26. Method according to clause 25, further comprising the step of blowing air through the carrying surface such that the flatwork item is pushed away from the carrying surface.

27. Method according to any of the previous clauses, further comprising the step of:

V2) feeding the flatwork item to a transport conveyor and/or a flatwork treating device. 28. System for detecting and gripping predetermined part of a flatwork item, such as a second corner of a flatwork item, preferably via a method according to any of clauses 16-27, comprising:

- at least one visual imaging device configured for capturing at least one second image of at least part of a flatwork item, a neural network trained to recognize at least one corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one corner,

- a displaceable first gripper configured for gripping a first corner of a flatwork item,

- a displaceable second gripper configured gripping a second corner of the flatwork item, and

- a carrying surface configured for carrying at least part of a flatwork item.

29. System according to clause 28, wherein the second gripper is configured for lifting at least part of the second corner before gripping said corner.

30. System according to any of clauses 28 or 29, wherein the carrying surface is at least partially permeable to air.

31. System according to any of clauses 28-30, comprising at least one clamp configured to hold a distal end of the flatwork item opposing the first corner.

32. System according to any of clauses 28-31 , wherein the first gripper and the second gripper are independently movable and controllable.

33. System according to any of clauses 28-32, wherein the second gripper comprises a suction element in combination with movable clamping element.

34. System according to any of clauses 28-33, wherein the visual imaging device comprises a camera. 36. Gripper configured for gripping at least part of a flatwork item, wherein the gripper is configured to provide a laminar airflow configured to temporarily lift at least part of the flatwork item.

37. Gripper according to clause 36, wherein the gripper comprises at least one air outlet for the outflow of air, in particular compressed air.

38. Gripper according to clause 36 or 37, comprising at least one boundary element configured to at least partially interrupt the airflow.

39. Gripper according to any of clauses 36-38, wherein the gripper comprises at least one internal and/or external structure configured to enable air distribution.

40. Gripper according to any of clauses 36-39, wherein the gripper comprises at least one turbulator.

The present invention will be explained further on the basis of the nonlimiting embodiment examples presented in the following figures. These show: figure 1 a schematic representation of a system according to the present invention; figure 2 a schematic representation of a second gripper according to the present invention and for use in a system according to the present invention; and figures 3a and 3b a schematic representation of a possible embodiment of a system according to the present invention.

Figure 1 shows a schematic representation of a system 100 according to the present invention. The system 100 is configured for feeding a flatwork item 101 to a transport conveyor and/or a flatwork treating device (not shown). The system 100 is configured for feeding a flatwork item 101 to a further device via the method according to the present invention. The method steps are explained hereinafter. Multiple flatwork items 101 are provided via a conveyor 120 and collected in a hopper 121. One flatwork item 101 is picked by a picker 122 comprising a claw.

The picker 122 positions the flatwork item 101 on a substantially horizontal surface 103. Said steps are exemplary steps. It is also conceivable that the flatwork item 101 is provided in another way. Preferably the flatwork item 101 is provided in a laying position. Once the flatwork item 101 is present, at least one first image is captured via the first visual imaging device 102. The system comprises at least one neural network(not shown) trained to recognize at least one corner of a flatwork item in an image and trained to indicate the position and/or orientation of said at least one corner. The position and/or orientation of at least one corner 101a of the flatwork item 101 is determined by the trained neural network. Once the position and/or orientation of the corner 101a is determined, the first gripper 104 is displaced towards the determined corner 101a of the flatwork item 101 and subsequently gripped. The flatwork item 101 is then displaced via the first gripper 104 holding the gripped corner 101a. This is typically done in a substantially horizontal orientation. The two adjacent guiding elements 105, in particular the guiding rollers 105, ensure the substantially horizontal orientation of the flatwork item 101 during the displacement. The guiding rollers 105 are positioned above another and the flatwork item 101 is displaced through the rollers 105. The distal end D opposing the first corner 101 of the flatwork item 101 is retained by a clamp 106. This is done such that at least one second corner 101 b of the flatwork item 101 substantially hangs fee. This can for example be enabled by shaking the flatwork item 101 at this point. The second corner 101 b is the corner at the shortest distance from the first corner 101a. The flatwork item 101 is then displaced over a carrying surface 107. This is typically done such that the second corner 101 b is positioned upon said carrying surface 107. At least one second image of at least part of the second corner 101 b of the flatwork item 101 is captured by the second visual imaging device 108. The neural network is used to determine the position and/or orientation of the second corner 101 b. Once this is done, the second gripper 109 is displaced towards the determined second corner 101b of the flatwork item 101 and subsequently gripped by said second gripper 109. The flatwork item 101 can then be fed to a further treating device such as a transport conveyor and/or a flatwork treating device. The arrows indicate the direction of movement of the flatwork item 101 within the system 100. In the shown embodiment, the system 100 comprises multiple first grippers 104. The primary first gripper 104a is configured to grip the first corner 101 a and displace the flatwork item 101 , possibly up to the guiding elements 105. The secondary first gripper 104b can take over the first corner from the primary first gripper 104a and further displace the flatwork item 101. The tertiary first gripper 104c can take over the first corner 101 a in order to displace the flatwork item 101 over the carrying surface 107. The use of described primary, secondary and tertiary first grippers 104a, 104b, 104c is merely illustrative, it is conceivable that another number of first grippers 104 is used. However, the use of multiple first grippers 104 may significantly speed up the overall process since more flatwork items 101 can be handled simultaneously. At least one first gripper 104 preferably comprises a gripping mouth configured to enclose at least part of a corner of the flatwork item 101 . The system 100 may comprise at least one drive unit (not shown) for displacing at least the first gripper 104, and possibly also the second gripper 109. The second gripper 109 is preferably configured for lifting at least part of the second corner 101b before said corner 101 b is gripped. A possible embodiment of the second gripper 109 is shown in figure 2. The carrying surface 107 shown is at least partially permeable to air. Once the flatwork item 101 is positioned upon the carrying surface 107 a vacuum may be applied. Hence, air is drawn though the carrying surface 107 such that the flatwork item is pressed against the carrying surface 107. The arrows underneath the carrying surface 107 indicate the suction of air. This step is typically discontinued after the second gripper 109 has gripped the second corner 101 b.

Figure 2 shows a schematic representation of a possible embodiment of a second gripper 209 according to the present invention and which is configured for use in a method and system according to the present invention. The gripper 209 is configured for gripping an edge and/or corner of a flatwork item (not shown). The gripper 209 comprising a suction element 210 configured for at least partially lifting an edge and/or a corner of the flatwork item, and a clamping element 211 for clampingly engaging said flatwork item. The clamping element 211 is configured to hold and/or enclose at least part of an edge and/or a corner of the flatwork item. The clamping element 211 and the suction element 210 are mutually displaceable. Typically, the clamping element 211 is displaceable with respect to the suction element 210. The arrows indication possible displacement directions of clamping element 211 and the suction element 210. The clamping element 211 is in the shown embodiment pivotably displaceable with respect to the suction element 210. The suction element 210 is also displaceable. The suction element 210 may for example comprise a telescopic arm. Due to the clamping element 211 being displaceable with respect to the suction element 210, the corner or edge can be firmly gripped. Typically, the suction element 210 draws air into the device thereby attracting the flatwork item. The gripper 209, and in particular the suction element 210, is then lifted such that the flatwork item is no longer positioned upon the carrying surface. Then the clamping element 211 is displaced such that it provides a clamping engagement with the corner and/or edge of the flatwork item. The air intake of the suction element 210 can be discontinued once a clamping engagement is established.

Figures 3a-3b shows a schematic representation of a system 300 according to the present invention. The system 300 is configured for detecting and gripping a second corner 301 b of a flatwork item 301 , preferably via a method according to the present invention. The system 300 comprises a neural network trained to recognize at least one corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one corner. The system 300 also comprises a visual imaging device 308 configured for capturing at least one second image of at least part of the flatwork item. With respect to the method, figure 3a shows that a flatwork item 301 is provided which is clamped at a first corner 301a by a first gripper 304, and which is, in the shown embodiment held at a distal end D opposing the first corner 301a by a clamp 306. This enables that at least one second corner 301 b substantially hangs fee. Subsequently the flatwork item 301 is displaced over a carrying surface 307. The arrow indicated the direction of displacement. The carrying surface 307 is at least partially permeable to air. The carrying surface 307 is substantially flat and comprises multiple air openings 307a.

Figure 3b shows the flatwork item 301 being positioned upon the carrying surface 307. The second corner 301b is positioned upon said carrying surface 307. Then at least one second image of at least part of the second corner 301b of the flatwork item 301 is to be captured by the visual imaging device 308. Subsequently the neural network provides a possible position and/or orientation of at least one second corner of the flatwork item. Then the second gripper 309 can be displaced towards the determined second corner 301 b which can then be gripped.

It will be clear that the invention is not limited to the embodiment examples presented and described here, but that numerous variants are possible within the scope of the appended claims or clauses, which will be obvious to a person skilled in the art. It is conceivable that various inventive concepts and/or technical measures of the embodiment variants described above may be combined completely or partially without departing from the inventive concepts described in the appended claims or clauses.

The verb "comprise" and conjugations thereof used in this patent specification mean not only "comprise", but also the expressions "include", "consist essentially of", "formed by", and conjugations thereof.

When the word ‘gripping’ is used, also the terms grasping, picking, holding, grabbing and/or clasping can be meant. When it is referred to a corner of a flatwork item, the point where two edges of the flatwork item meet is meant. It also possible that the corner is a shoulder part of for example a shirt or (bath)robe. The corner is typically gripped by double sided engagement of the textile. The position and/or orientation of the first image can be determined in both a 2D and/or 3D configuration. The first and second grippers are typically also configured for releasing the gripped part of the flatwork item after their step according to the invention is fulfilled.

Claims

1 . Method for feeding a flatwork item to a transport conveyor and/or a flatwork treating device, comprising the steps of: a) providing at least one neural network trained to recognize at least one corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one corner, b) providing at least one flatwork item, c) capturing at least one first image of at least part of said flatwork item, d) providing the first image to the neural network, e) receive a possible position and/or orientation of at least one corner of the flatwork item from the neural network; f) displacing a first gripper towards the determined corner of the flatwork item, g) gripping of the corner by said first gripper, h) displacing the flatwork item via the gripped corner preferably in a substantially horizontal orientation, i) retaining a distal end opposing the first corner by a clamp, such that at least one second corner of the flatwork item substantially hangs fee, j) displacing of the flatwork item over a carrying surface preferably such that the second corner is positioned upon said carrying surface, k) capturing at least one second image of at least part of the second corner of the flatwork item,

L) providing the second image to the neural network, m) receiving a possible position and/or orientation of at least one second corner of the flatwork item from the neural network, n) displacing a second gripper towards the determined second corner of said flatwork item, o) gripping of the determined second corner by said second gripper, and p) optionally feeding the flatwork item to a transport conveyor and/or a flatwork treating device.

2 Method according to claim 1 , further comprising the steps of: q) taking multiple images of at least one flatwork item, wherein each image shows at least one corner of the flatwork item, r) annotating each image with information about a position and/or orientation of at least one corner of said flatwork item, and s) saving the annotated images of flatwork items in a neural network.

3. Method according to claim 1 or 2, further comprising the steps of: t) determining if the gripping of the determined corner by the first gripper was successful, u) annotating at least one first image made at step b) with information of the position and/or orientation of at least one corner, v) adding of the annotated image to at least one neural network, and/or w) determining if the gripping of the second corner by the second gripper was successful, x) annotating at least one second image made at step k) with information of the position and/or orientation of the second corner, and y) adding of the annotated second image to at least one neural network.

4. Method according to any of the previous claims, comprising the step of: z) shaking the flatwork item prior to step j) such that a second corner which is at shortest distance from the first corner will hang free.

5. Method according to any of the previous claims, wherein step o) comprises the steps of: o1) lifting the second corner of the flatwork item via suction, and o2) gripping the second corner via a enclosing at least part of the second corner by the second gripper, in particular by a clamping element of the second gripper.

6. Method according to any of the previous claims, further comprising the steps of: aa) suction of air from underneath the carrying surface such that the flatwork item is pressed against the carrying surface after step j) and preferably before step k), and bb) stopping the suction of air after the second gripper has gripped the second corner.

7. Method according to claim 6, further comprising the step of blowing air through the carrying surface such that the flatwork item is pushed away from the carrying surface.

8. Method according to any of the previous claims, comprising the steps of: cc) providing multiple flatwork items, dd) picking at least one flatwork item by a picker, and ee) positioning of the flatwork item on a substantially horizontal surface.

9. Method according to any of the previous claims, further comprising the steps of: ff) rating each detected first corner of at least one flatwork item from the neural network, gg) selecting a corner to be gripped which has a score which is higher than a predetermined threshold score, and/or hh) discarding the flatwork item if the score is lower than the threshold score.

10. Method according to any of the previous claims, wherein the first gripper is controlled to grip the nearest corner when multiple corners are determined.

11. Method according to any of the previous claims, wherein a secondary first gripper takes over the first corner after step g).

12. Method according to any of the previous claims, wherein a tertiary first gripper takes over the first corner prior to step j).

13. System for feeding a flatwork item to a transport conveyor and/or a flatwork treating device, preferably via a method according to any of claims 1-12, comprising: at least one first visual imaging device configured for capturing at least one first image of at least part of a flatwork item, at least one second visual imaging device configured for capturing at least one second image of at least part of a flatwork item, a neural network trained to recognize at least one corner of a flatwork item in an image and to indicate the position and/or orientation of said at least one corner, at least one displaceable first gripper configured for gripping a first corner of a flatwork item a displaceable second gripper configured gripping a second corner of the flatwork item, and a carrying surface configured for carrying at least part of a flatwork item.

14. System according to claim 13, wherein the first gripper comprises a gripping mouth configured to enclose at least part of a corner of the flatwork item.

15. System according to any of claims 13 or 14, further comprising a drive unit for displacing the first gripper towards the location determined by the neural network.

16. System according to any of claims 13-15, wherein at least one first gripper, and in particular the gripping mouth, is rotatable over at least 180 degrees, preferably 360 degrees, over an axis of rotation.

17. System according to any of claims 13-16, wherein the second gripper is configured for lifting at least part of the second corner before gripping said corner.

18. System according to any of claims 13-17, wherein the carrying surface is at least partially permeable to air.

19. System according to any of claims 13-18, comprising at least one clamp configured to hold a distal end of the flatwork item opposing the first corner.

20. System according to any of claims 13-19, wherein the at least one first gripper and the second gripper are independently movable and controllable.

21. System according to any of claims 13-20, wherein the first visual imaging device and/or the second visual imaging device comprises a camera.

22. System according to any of claims 13-21 , comprising at least one air knife.

23. Use of a system according to any of claims 13-22.