Classifications

• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
  • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
  • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
  • B07C5/34—Sorting according to other particular properties
  • B07C5/342—Sorting according to other particular properties according to optical properties, e.g. colour
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
  • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
  • B07C2501/00—Sorting according to a characteristic or feature of the articles or material to be sorted
  • B07C2501/0063—Using robots

Definitions

• This disclosure relates to an apparatus for object classification.
• the disclosure is particularly useful, but not limited to classifying waste objects for sorting.
• the apparatus is capable of being retrofitted to an object sorting system.
• waste objects may be identified and classified using various object classification methods. Some of these methods may involve the use of conventional RGB cameras or other image-capturing devices involving the use of near-infrared (NIR) radiation-based or terahertz (THz) radiation-based imaging technology.
• NIR near-infrared
• THz terahertz
• implementation of some of the aforementioned technology may involve extensive modification(s) of existing conveyor belt systems and materials deployed in waste object sorting systems.
• modifications may need to be made to a production line, supporting frames, and/or adapt any conveyor system in the production lines of the MRF to incorporate THz imaging sensors or scanners.
• the apparatus is retrofittable to an existing conveyor system, the existing conveyor system may form part of a waste object sorting system, such as, but not limited, to a manual sorting system.
• the apparatus includes a retrofit flat belt conveyor, a modular sensor platform with Al-based object classification module, and a robotic sortation module.
• an apparatus as claimed in claim 1 is provided.
• a method for retrofitting the apparatus to an existing conveyor system is defined in claim 10.
• FIG. 1 shows a schematic perspective side-view setup of an apparatus for obtaining object images for object classification and sortation, the apparatus capable of being retrofitted to an existing conveyor belt system or production line system in a material recovery facility (MRF), according to some embodiments;
• MRF material recovery facility
• FIGS. 2 A to 2C show schematic views of the retrofittable conveyor platform module integrated with an object classification module of the apparatus of FIG. 1, the integrated module being configured to be fitted onto an existing conveyor system in an MRF;
• FIG. 3 A shows a system diagram of a particular implementation of the Al-based object classification module according to some embodiments
• FIG. 3B shows examples of predicted object classes based on obtained object images using the Al-based object classification module
• FIGS. 4 A to 4C are schematic views of various embodiments of the robotic sortation module
• FIGS. 5 A to 5E show additional embodiments of the retrofittable conveyor platform module
• FIG. 6 shows a flow chart of a method of retrofitting the apparatus to an existing conveyor belt system or production line system in a material recovery facility (MRF).
• MRF material recovery facility
• the articles “a”, “an” and “the” as used with regard to a feature or element include a reference to one or more of the features or elements.
• data may be understood to include information in any suitable analog or digital form, for example, provided as a file, a portion of a file, a set of files, an electronic signal or stream, a portion of a signal or stream, a set of signals or streams, and the like.
• data is not limited to the aforementioned examples and may take various forms and represent any information as understood in the art.
• the term “THz image sensor” refers broadly to THz-based systems for identifying an object, which usually comprises a transmitter or source and a receiver/scanner.
• the term “THz radiation” refers to electromagnetic waves/radiation within the frequency range of 0.1 to 10 THz. THz radiation may transmit through non-polar and non- conductive materials, particularly plastic objects, and may be used to identify different types of plastic objects.
• object includes any object, particularly recyclable or reusable object that may be identified according to type, class or categories. For example, object may be identified based on whether it is plastic or non-plastic (e.g. paper). Plastic objects may in turn be identified according to whether they are High-density polyethylene (HDPE), Polyethylene terephthalate (PET), Polypropylene (PP), Polystyrene (PS), Low-density polyethylene (LDPE), Polyvinyl chloride (PVC) plastic objects, post-consumer plastics (e.g. PP, PS, PVC), engineering plastics (e g.
• HDPE High-density polyethylene
• PET Polyethylene terephthalate
• PP Polypropylene
• PS Polystyrene
• LDPE Low-density polyethylene
• PVC Polyvinyl chloride
• Such objects may include bottles, jars, containers, plates, bowls etc. of various shapes, forms (distorted, flattened) and sizes.
• the object may be contaminated with food, grease (e.g. salad dressing), cooking oil, yogurt, liquid, shampoo, detergent, soy sauce, etc.
• module refers to, or forms part of, or includes an Application Specific Integrated Circuit (ASIC); an electronic circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor (shared, dedicated, or group) that executes code; other suitable hardware components such as one or more sensors, one or more robotic arms that provide the described functionality; or a combination of some or all of the above, such as in a system-on-chip.
• ASIC Application Specific Integrated Circuit
• FPGA field programmable gate array
• the term module may include memory (shared, dedicated, or group) that stores code executed by the processor.
• the present disclosure comprises an apparatus for object classification that can be easily implemented on or retrofitted to an existing conveyor system.
• the object sorting apparatus may be attached to an existing conveyor system of a material recovery facility (MRF) and may be configured as a plug and play system comprising three or four modules.
• MRF material recovery facility
• the object sorting apparatus can be added to an existing MRFs’ infrastructure regardless of the size of MRFs and the waste stream.
• an apparatus 100 for classifying one or more objects 102 comprises a retrofittable conveyor platform module 110 configured to receive the object 102 thereon, the conveyor platform module 110 comprises an interface portion 112, the interface portion 112 configured to be removably positioned on or attached to an existing conveyor system 160; an artificial intelligence-based (Al-based) object classification module 120 comprising at least one image sensor module 122 configured to obtain at least one object image 124, 126 (see FIG.
• the object 102 on the conveyor platform module 110 and classify the object 102 using the at least one object image 124, 126 as input to predict an object class 128 of the object 102; and a robotic sortation module 130 arranged in data connection with the Al-based object classification module 120 to receive the object class 128, the robotic sortation module 130 configured to sort the object 102 based on the object class 128.
• the coordinates of the object 102 as will be elaborated subsequently, based on the position on the existing conveyor system 160, may also be used as an input for sortation of the object 102.
• the one or more objects 102 may be waste objects positioned on an existing conveyor system 160 of a material recovery facility (MRF) for recycling.
• MRF material recovery facility
• the conveyor system 160 may include a feeding line 160a and a sorting line 160b.
• the feeding line 160a may be an inclined conveyor belt and the sorting line 160b may be a relatively long flat conveyor belt line, and one or more manual, such as human sorters may be positioned in the proximity of the sorting line 160b to retrieve objects for sorting.
• the conveyor platform module 110 is positioned between the inclined feeding line 160a and the sorting line 160b in a manner so as to receive the one or more objects 102 from the inclined feeding line 160a.
• the inclined feeding line 160a may be positioned lower relative to an upper sorting line 160b.
• the conveyor platform module 110 may be attached to a conveyor portion of the sorting line 160b and positioned on top/ above of the conveyor portion of the sorting line 160b as an overlapping member of the sorting line 160b.
• the conveyor platform module 110 may be positioned just after the end or edge of the inclined feeding line 160a and/or at the beginning region of the sorting line 160b.
• the conveyor platform module 110 may be positioned at a drop region between the sorting line 160b and the inclined feeding line 160a.
• the conveyor platform module 110 and the Al-based object classification module 120 may be integrated so that when the one or more objects 102 are on the conveyor platform module 110, the image sensor module 122 is operable to obtain object images 124, 126 of each object 102.
• the robotic sortation module 130 may be positioned downstream the conveyor platform module 110 and the Al-based object classification module 120 of a sorting line 160b.
• the robotic sortation module 130 may comprise at least one robotic arm 132 configured to retrieve objects 102 from the sorting line 160b or downstream of the sorting line 160b for sorting.
• An optional object tracking module 140 may be provided to track the position of the one or more objects 120 on the existing conveyor system 160.
• the object tracking module 140 may include one or more image-capturing devices 142 configured to track the position of the object(s) 120 on the existing conveyor platform 160 between the conveyor platform module 110 and the robotic sortation module 130.
• the object tracking module 140 may include one or more processors installed with computer vision and/or image processing software to facilitate the tracking function.
• FIGS. 2A to 2C show embodiments of the conveyor platform module 110 integrated with the object classification module 120.
• the interface portion 112 of the conveyor platform module 110 may be configured to be removably positioned on or attached to the existing conveyor system 160.
• the interface portion 112 may include one or more fasteners such as clamps to attach the conveyor platform module 110 to one or more sides of the existing conveyor system 160.
• the conveyor platform module 110 may comprise anti-slip stands formed of materials such as rubber, the anti-slip stands to be positioned on the existing conveyor system 160.
• the conveyor platform module 110 comprises a conveyor belt 111, and a slot 114 formed on an under-side or underneath the conveyor belt 111 to receive a portion of the image sensor module 122.
• the image sensor module 122 of the Al-based object classification module 120 may comprise a first image sensor 123 configured to obtain an RGB object image 124 of the object 102 on the conveyor platform module 110 and a second image sensor 125 configured to obtain a power intensity heatmap image 126 generated based on a THz electromagnetic radiation irradiated on the object 102 on the conveyor platform module 110.
• the first image sensor 123 may be a RGB camera or video recorder.
• the second image sensor 125 may comprise a THz source 125a and a scanner 125b.
• the THz source 125a may be configured to emit an electromagnetic radiation in a THz frequency range.
• the scanner 125b may be suitably receive in the slot 114 underneath the conveyor belt 111.
• the THz scanner 125b and the THz source 125a may be arranged in a manner such that THz radiation emitted from the THz source 125a passes the object 102 and the resultant radiation that passes the object 102 is incident on the scanner 125b.
• the captured radiation by the scanner 125b may then be processed and used to form the power intensity heatmap image 126.
• FIGS. 2B and 2C show suitable positions where the linear scanner 125b may be positioned, within a slot 114 formed underneath the conveyor belt 111.
• a frame 116 may comprise two extensions 116a, 116b extending vertically from the sides of the conveyor platform module 110, the two extensions 116a, and 116b joined by a horizontal portion 116c.
• the horizontal portion 116c may comprise respective mounts for mounting the first image sensor 123 and the THz source 125a.
• the frame 116 may also comprise a housing to contain the Al-based object classification module 120 therein.
• the frame 116 may be formed from or of metallic materials.
• one or more visible light source(s) may be positioned in the vicinity of the first image sensor 123 to provide visible lighting to facilitate RGB image capturing.
• FIG. 3A shows a processor 200 as an exemplary implementation of the Al-based object classification module 120.
• the processor 200 comprises an input module 202, an artificial intelligence (Al) module 204, and an output module 206.
• Al artificial intelligence
• the input module 202 is configured to receive the obtained and/or generated images from the first image sensor 123 and the second image sensor 125.
• the image sensors 123, 125 may be equipped with data communication capabilities to transmit the obtained object images 124, 126 as data to the input module 202.
• the input module 202 may be arranged in data communication or connection with the image sensors 123, 125 via wired or wireless data communication protocol.
• the Al module 204 is configured to receive the RGB image and THz image of the object 102 as inputs, and produce a predicted object class 128, the object class 128 forming part of the output dataset of the Al module 204.
• the Al module 204 may comprise Al algorithms such as deep learning convolutional neural networks.
• the Al module 204 may also include object or feature detection algorithms such as YOLOv4 algorithm, DeepSORT algorithm. Other known computer vision/ image processing techniques known to a skilled person may be combined/supplemented to form further embodiments to supplement and/or replace the ML/ Al algorithms.
• the Al module 204 may be trained to recognise contamination on the object images 124, 126.
• the Al module 204 may utilize a sensor fusion method that extract salient features from the RGB image 124 and THz heatmap image 126 for object classification and to determine detailed material content, for example, a degree of contamination and abnormal objects, such as unexpected items, that do not belong to general waste, e.g., toxic industrial waste (e.g. chemical bottles, drug), biohazardous waste (e.g. syringe).
• An example of feature extraction may involve steps of defining regions of interest (ROIs), such as bounding boxes around the object images, and utilizing image processing and computer vision techniques including steps of segmentation and labelling.
• ROIs regions of interest
• the output module 206 may be configured to store the object images 124, 126 in various data formats such as png, tiff, jpeg, bmp, etc.
• the output module 206 may be equipped with data communication devices to send the image data to the robotic sortation module 130.
• the output module 206 may be configured to send the object images 124, 126 together with the coordinate information of identified region(s) of interest (ROI) on the object images.
• ROI identified region(s) of interest
• Such coordinates information may be derived based on height and/or depth information from an RGB camera having depth scanning capabilities.
• FIG. 3B show some examples of predicted object classes 128 based on the object images 124, 126 as inputs to the Al-based object classification module 120.
• an object 102 passes an image-capturing region of the first image sensor 123, an RGB object image 124 is obtained.
• the electromagnetic radiation emitted by source 125a is incident on the object 102 and passes through the object 102.
• a modified (e.g. reflected, refracted, deflected) electromagnetic radiation is obtained by the THz scanner 125b.
• the spectral data obtained by the THz scanner 125b may be obtained using the THz scanner 125b configured to have a relatively wide spectral range of 0.05 GHz to 0.7 THz.
• the spectral data is then used to render a THz object image 126.
• the RGB object image 124 and the THz object image 126 may be rendered at the aforementioned processor of the Al-based object classification module 120.
• the Al module 204 may be utilised to differentiate post-consumer plastics (PET and HDPE bottle), the packaging materials (LDPE clear/colour, Cardboard, Paper, Paper bag (with plastic layer), PP/PET straps and Jumbo bag (PP)) and other rigid engineering plastics (HDPE/PP) by combining features extracted from the obtained THz object image 214 and RGB object image 212.
• Different THz features for example, shape, THz pattern intensity based on different THz transmittance, reflectance and absorbance according to material types
• RGB features e.g., shape, colour
• FIGS. 4 A to 4C show an embodiment of the robotic sortation module 130 arranged in data connection with the artificial intelligence-based object classification module 120 to receive the output object class 128 and to sort the object 102 based on the object class 128.
• the robotic sortation module 130 may comprise an industrial robotic arm 132 and a mounting gantry structure 134.
• the gantry structure 134 may be in the form of a frame, the gantry frame 134 configured to mount the robotic arm 132, the robotic arm 132 positioned to pick up objects 102 from the existing conveyor system 160. As shown in FIG.
• the gantry frame 134 may comprise two vertical supports 134a, 134b positioned such that a portion of the existing conveyor system 160 is positioned between the two supports 134a, 134b.
• the robotic arm 132 may be mounted at a horizontal beam 134c joining two ends of the vertical supports 134a, 134b.
• the robotic sortation module 130 may be equipped with a controller 136.
• the controller 136 may receive input data relating to the predicted object class 128 and corresponding object images 124, 126.
• the controller may also be configured to receive position data of the object 102 as the object 102 moves along the existing conveyor system 160.
• FIG. 4B shows an arrangement and a top perspective view of the conveyor belt system 160 and the robotic sortation module 130.
• FIG. 4C shows an arrangement and side perspective view of the conveyor belt system 160 and the robotic sortation module 130.
• the robotic arm 132 may be configured to pick and drop objects for sorting according to the type of objects, for example 4 types of objects into discharging bins 139.
• a plurality of robotic arms 132 arranged in series can be configured according to the existing processing & sorting lines and their waste input/output stream and volume in the MRFs.
• the object tracking module 140 may comprise at least one image-capturing device, which may be a RGB camera positioned to re-identify objects 102 and track the movement of objects 102 along the existing conveyor system 160. This is to account for situations where the position of some objects 102 may change after the object 102 is discharged from the conveyor platform module 110.
• the position information or data of the objects 102 can be re-identified using machine learning material identification and tracking algorithm (ROI detection).
• ROI detection machine learning material identification and tracking algorithm
• the respective coordinates of each of the objects 102 may be sent to the robotic controller 136 for the subsequent robotic sortation process.
• FIG. 5 A shows another embodiment of the conveyor platform module 110 with a retrofit flat belt conveyor system integrated with the Al-based object classification module 120.
• the Al-based object classification module 120 may be configured for the optical image capturing of objects using the first image sensor 123, deep learning of waste object detection and classification in an Al platform, and detailed material content, for example, contamination and anomaly identification module using the THz image sensor 125, and a conveyor belt system.
• the image sensor module 122 includes a lighting setup for obtaining clear and reliable images.
• a two-ply or three-ply Polyvinyl chloride (PVC) dark green belt may be used for the conveyor belt 111 material.
• PVC Polyvinyl chloride
• the conveyor speed may be configured to be in the range of 0.3 to 1.5 meters per second and may be controlled by a frequency inverter 113.
• the conveyor platform module 110 comprises at least one object guide, the at least one object guide 502 shaped and dimensioned to guide the object 102 to an image capturing region of the conveyor platform module 110.
• the object guide 502 may be a metallic side guide along the perimeter of the conveyor belt 111, and a part of the metallic side guide may extend towards a center region of the conveyor belt 111 corresponding to the image capturing regions. Such an arrangement will define a funnel -like profile for directing the object(s) 102 from the feeding line 160a in line with the image capturing region of THz image sensor 125.
• the metallic side guide may be formed from or of aluminum profile.
• the aluminum profile may be a 40x40 or 20x20 profile.
• FIGS. 5B and 5C show another embodiment of the conveyor platform module 110 with a retrofit flat belt conveyor system integrated with the Al-based object classification module 120.
• the embodiment of FIG. 5B comprises a slide plate 504 positioned at an entry side of the retrofit flat belt conveyor 111 for smooth feeding of object 102 which may be dropped from the inclined feeding belt 160a or an upper feeding belt line.
• the angle and slope length may be adjusted or adapted to minimize any mis-feeding of objects 102. For example, any rolling items on the retrofit flat conveyor belt 111, poor distribution with overlapping of objects 102 may be minimized.
• the angle of slope may be between 10 degrees (°) and 40° with respect to the horizontal and the slope length may be between 150 millimeters (mm) and 230 mm.
• FIG. 5C shows various embodiments of a retrofit flat belt conveyor with slide plates 504 inclined at angles of between 10° and 40°, and with varying slope length, to facilitate smooth feeding of waste stream input
• one or more air-jetting hole arrays 506 may be added to the slide plate 504 to vibrate the slide plate 504 so as to facilitate distribution of multiple objects 102 on the conveyor belt 111.
• the metallic sliding plate can be adapted for proper angle and slope length based on the aforementioned description, and the additional side panel(s) 508 may be added to facilitate smooth feeding of objects 102 onto the retrofit flat belt conveyor 111.
• the side panels 508 may be shaped and dimensioned to direct objects 102 around the edges of the slide plate 506 to the center portion of the conveyor belt 111.
• One side of the side panel 508 may be affixed to the slide plate 506 with fasteners such as screws, and the other end may be trimmed at a connection to the side belt.
• the thickness of this side panel may be 1.5mm, and its detailed design is depicted in FIG. 5E.
• roller brushes can be added to one or more sides underneath the flat belt conveyor 111 for cleaning any liquid and dust for clean and safe operation.
• the Al based object classification module 120 comprising image sensors to capture object images for object detection and tracking, and Terahertz radiation for material classification using deep learning method. Furthermore, it can detect contamination and add value to improve sorting grade.
• FIG. 6 shows a flow chart of a method 300 of retrofitting an existing conveyor system for classifying an object 102, the method 200 comprising the steps of [0057] Step 302: providing a conveyor platform module 110, the conveyor platform module 110 configured to receive the object 102 thereon, the conveyor platform module 110 comprises an interface portion 112, the interface portion 112 configured to be removably positioned on or attached to an existing conveyor system 160;
• Step 304 providing an artificial intelligence-based object classification module 120 comprising at least one image sensor module 122, the at least one image sensor module 122 configured to obtain at least one object image 124, 126 of the object 102 on the conveyor platform module 110 and classify the object 102 using the at least one object image 124 as input to predict a class of the object 102 and provide an object class 128 of the object 102; and [0059] Step 306: providing a robotic sortation module 130 arranged in data connection with the artificial intelligence-based object classification module 120 to receive the object class 128, the robotic sortation module 130 configured to sort the object 102 based on the object class 128.
• a computer program comprising instructions which, when executed by a processor, cause the processor to carry out a method of object classification within the Al-based object classification module 120.

Landscapes

• Sorting Of Articles (AREA)

Applications Claiming Priority (1)

Publications (1)

Family

ID=85224982

Family Applications (1)

Country Status (2)

Family Cites Families (2)

• 2023
  • 2023-02-09 EP EP23704748.5A patent/EP4662016A1/de active Pending
  • 2023-02-09 WO PCT/EP2023/053164 patent/WO2024165153A1/en not_active Ceased

Also Published As

Similar Documents

Legal Events