WO2023003670A1 - Système de manipulation de matériau - Google Patents

Système de manipulation de matériau Download PDF

Info

Publication number
WO2023003670A1
WO2023003670A1 PCT/US2022/035013 US2022035013W WO2023003670A1 WO 2023003670 A1 WO2023003670 A1 WO 2023003670A1 US 2022035013 W US2022035013 W US 2022035013W WO 2023003670 A1 WO2023003670 A1 WO 2023003670A1
Authority
WO
WIPO (PCT)
Prior art keywords
materials
classification
type
program product
computer program
Prior art date
Application number
PCT/US2022/035013
Other languages
English (en)
Inventor
Nalin Kumar
Manuel Gerardo GARCIA JR.
Kanishka Tyagi
Original Assignee
Sortera Alloys, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US17/380,928 external-priority patent/US20210346916A1/en
Priority claimed from US17/491,415 external-priority patent/US11278937B2/en
Application filed by Sortera Alloys, Inc. filed Critical Sortera Alloys, Inc.
Publication of WO2023003670A1 publication Critical patent/WO2023003670A1/fr

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/36—Sorting apparatus characterised by the means used for distribution
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/34—Sorting according to other particular properties
    • B07C5/342—Sorting according to other particular properties according to optical properties, e.g. colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition

Definitions

  • the present disclosure relates in general to the handling of materials, and in particular, to the classifying and/or sorting of materials.
  • Recycling is the process of collecting and processing materials that would otherwise be thrown away as trash, and turning them into new products. Recycling has benefits for communities and for the environment, since it reduces the amount of waste sent to landfills and incinerators, conserves natural resources, increases economic security by tapping a domestic source of materials, prevents pollution by reducing the need to collect new raw materials, and saves energy. After collection, recyclables are generally sent to a material recovery facility to be sorted, cleaned, and processed into materials that can be used in manufacturing. As a result, high throughput automated sorting platforms that economically sort highly mixed waste streams would be beneficial throughout various industries.
  • Scrap metals are often shredded, and thus require sorting to facilitate reuse of the metals. By sorting the scrap metals, metal is reused that may otherwise go to a landfill. Additionally, use of sorted scrap metal leads to reduced pollution and emissions in comparison to refining virgin feedstock from ore. Scrap metals may be used in place of virgin feedstock by manufacturers if the quality of the sorted metal meets certain standards.
  • the scrap metals may include types of ferrous and nonferrous metals, heavy metals, high value metals such as nickel or titanium, cast or wrought metals, and other various alloys.
  • FIG. 1 illustrates a schematic of a material handling system configured in accordance with embodiments of the present disclosure.
  • FIG. 2 illustrates an exemplary representation of a control set of material pieces used during a training stage in a machine learning system.
  • FIG. 3 illustrates a flowchart diagram configured in accordance with embodiments of the present disclosure.
  • FIG. 4 illustrates a flowchart diagram configured in accordance with embodiments of the present disclosure.
  • FIG. 5 illustrates a block diagram of a data processing system configured in accordance with embodiments of the present disclosure.
  • a “material” may include a chemical element, a compound or mixture of chemical elements, or a compound or mixture of a compound or mixture of chemical elements, wherein the complexity of a compound or mixture may range from being simple to complex.
  • “element” means a chemical element of the periodic table of elements, including elements that may be discovered after the filing date of this application.
  • materials may include any object, including but not limited to, metals (ferrous and nonferrous), metal alloys, novel alloys, super alloys (e.g., nickel super alloys), plastics (including, but not limited to PCB, HDPE, UHMWPE, and various colored plastics), rubber, foam, glass (including, but not limited to borosilicate or soda lime glass, and various colored glass), ceramics, paper, cardboard, Teflon, PE, bundled wires, insulation covered wires, rare earth elements, leaves, wood, plants, parts of plants, textiles, bio-waste, packaging, electronic waste (“e-waste”) such as electronic equipment and PCB boards, batteries and accumulators, end-of-life vehicle scrap pieces, mining, construction, and demolition waste, crop wastes, forest residues, purpose-grown grasses, woody energy crops, microalgae, food waste, hazardous chemical and biomedical wastes, construction debris, farm wastes, biogenic items, non-biogenic items, objects with a carbon content,
  • the terms “identify” and “classify,” the terms “identification” and “classification,” and any derivatives of the foregoing, may be utilized interchangeably.
  • to “classify” a piece of material is to determine (i.e., identify) a type or class of materials to which the piece of material belongs.
  • a sensor system may be configured to collect, or capture, as the case may be, any type of information (e.g., characteristics) for classifying materials, which classifications can be utilized within a sorting system to selectively sort material pieces as a function of a set of one or more physical and/or chemical characteristics (e.g., which may be user-defined), including but not limited to, color, texture, hue, shape, brightness, weight, density, composition, size, uniformity, manufacturing type, chemical signature, radioactive signature, transmissivity to light, sound, or other signals, and reaction to stimuli such as various fields, including emitted and or reflected electromagnetic radiation (“EM”) of the material pieces.
  • EM electromagnetic radiation
  • manufacturing type refers to the type of manufacturing process by which the material piece was manufactured, such as a metal part having been formed by a wrought process, having been cast (including, but not limited to, expendable mold casting, permanent mold casting, and powder metallurgy), having been forged, a material removal process, etc.
  • the types or classes (i.e., classification) of materials may be user-definable and not limited to any known classification of materials.
  • the granularity of the types or classes may range from very coarse to very fine.
  • the types or classes may include plastics, ceramics, glasses, metals, and other materials, where the granularity of such types or classes is relatively coarse; different metals and metal alloys such as, for example, zinc, copper, brass, chrome plate, and aluminum, where the granularity of such types or classes is finer; or between specific types of plastic, where the granularity of such types or classes is relatively fine.
  • the types or classes may be configured to distinguish between materials of significantly different compositions such as, for example, plastics and metal alloys, or to distinguish between materials of almost identical composition such as, for example, different types of plastics. It should be appreciated that the methods and systems discussed herein may be applied to accurately identify/classify pieces of material for which the composition is completely unknown before being classified.
  • a “conveyor system” may be any known piece of mechanical handling equipment that moves materials from one location to another, including, but not limited to, an aero- mechanical conveyor, automotive conveyor, belt conveyor, belt-driven live roller conveyor, bucket conveyor, chain conveyor, chain-driven live roller conveyor, drag conveyor, dust-proof conveyor, electric track vehicle system, flexible conveyor, gravity conveyor, gravity skatewheel conveyor, lineshaft roller conveyor, motorized-drive roller conveyor, overhead I-beam conveyor, overland conveyor, pharmaceutical conveyor, plastic belt conveyor, pneumatic conveyor, screw or auger conveyor, spiral conveyor, tubular gallery conveyor, vertical conveyor, vibrating conveyor, wire mesh conveyor, and robotic arm manipulators.
  • the systems and methods described herein receive a heterogeneous mixture of a plurality of material pieces, wherein at least one material piece within this heterogeneous mixture includes a composition of elements different from one or more other material pieces and/or at least one material piece within this heterogeneous mixture is physically distinguishable from other material pieces, and or at least one material piece within this heterogeneous mixture is of a class or type of material different from the other material pieces within the mixture, and the systems and methods are configured to identify/classify/sort this one material piece into a group separate from such other material pieces.
  • Embodiments of the present disclosure may be utilized to sort any types or classes of materials as defined herein. By way of contrast, a homogeneous set or group of materials all fall within an identifiable class or type of material.
  • Embodiments of the present disclosure may be described herein as sorting material pieces into such separate groups by physically depositing (e.g., diverting or ejecting) the material pieces into separate receptacles or bins as a function of user-defined groupings (e.g., material type classifications).
  • material pieces may be sorted into separate receptacles in order to separate material pieces classified as belonging to a certain class or type of material that are distinguishable from other material pieces (for example, which are classified as belonging to a different class or type of material).
  • the materials to be sorted may have irregular sizes and shapes.
  • such material may have been previously run through some sort of shredding mechanism that chops up the materials into such irregularly shaped and sized pieces (producing scrap pieces), which may then be fed or diverted onto a conveyor system.
  • FIG. 1 illustrates an example of a material handling system 100, which may be configured in accordance with various embodiments of the present disclosure to automatically classify/sort materials.
  • a conveyor system 103 may be implemented to convey individual (i.e., physically separable) material pieces 101 through the system 100 so that each of the individual material pieces 101 can be tracked, classified, and/or sorted into predetermined desired groups.
  • Such a conveyor system 103 may be implemented with one or more conveyor belts on which the material pieces 101 travel, typically at a predetermined constant speed.
  • certain embodiments of the present disclosure may be implemented with other types of conveyor systems as disclosed herein.
  • the conveyor system 103 may also be referred to as the conveyor belt 103.
  • some or all of the acts of conveying, stimulating, detecting, capturing, collecting, classifying, and or sorting may be performed automatically, i.e., without human intervention.
  • one or more sources of stimuli, one or more emissions detectors, a classification module, a sorting apparatus, and or other system components may be configured to perform these and other operations automatically.
  • FIG. 1 depicts a single stream of material pieces 101 on a conveyor belt 103
  • embodiments of the present disclosure may be implemented in which a plurality of such streams of material pieces are passing by the various components of the system 100 in parallel with each other, or a collection of material pieces deposited in a random manner onto a conveyor system (e.g., the conveyor belt 103) are passed by the various components of the system 100.
  • certain embodiments of the present disclosure are capable of simultaneously tracking, classifying, and/or sorting a plurality of such parallel travelling streams of material pieces, or material pieces randomly deposited onto a conveyor system (belt).
  • singulation of the material pieces 101 is not required to track, classify, and/or sort the material pieces.
  • the conveyor belt 103 may be a conventional endless belt conveyor employing a conventional drive motor 104 suitable to move the conveyor belt 103 at the predetermined speeds.
  • some sort of suitable feeder mechanism may be utilized to feed the material pieces 101 onto the conveyor belt 103, whereby the conveyor belt 103 conveys the material pieces 101 past various components within the system 100.
  • the conveyor belt 103 is operated to travel at a predetermined speed by a conveyor belt motor 104. This predetermined speed may be programmable and or adjustable by an operator in any well- known manner.
  • control of the conveyor belt motor 104 and or the position detector 105 may be performed by a well-known automation control system 108.
  • Such an automation control system 108 may be operated under the control of a computer system 107 and/or the functions for performing the automation control may be implemented in software within the computer system 107.
  • a position detector 105 may be operatively coupled to the conveyor belt 103 and the automation control system 108 to provide information corresponding to the movement (e.g., speed) of the conveyor belt 103.
  • the controls to the conveyor belt drive motor 104 and or the automation control system 108 and alternatively including the position detector 105
  • the automation control system 108 is able to track the location of each of the material pieces 101 while they travel along the conveyor belt 103.
  • a tumbler and/or a vibrator may be utilized to separate the individual material pieces from a collection of material pieces, and then they may be positioned into one or more singulated (i.e., single file) streams.
  • the material pieces may be positioned into one or more singulated (i.e., single file) streams, which may be performed by an active or passive singulator 106.
  • An example of a passive singulator is further described in U.S. Patent No. 10,207,296.
  • the conveyor system e.g., the conveyor belt 103 may simply convey a collection of material pieces, which have been deposited onto the conveyor belt 103, in a random manner.
  • certain embodiments of the present disclosure may utilize a vision, or optical recognition, system 110 and/or a distance measuring device 111 as a means to begin tracking each of the material pieces 101 as they travel on the conveyor belt 103.
  • the vision system 110 may utilize one or more still or live action cameras 109 to note the position (i.e., location and timing) of each of the material pieces 101 on the moving conveyor belt 103.
  • the vision system 110 may be further, or alternatively, configured to perform certain types of identification (e.g., classification) of all or a portion of the material pieces 101. For example, such a vision system 110 may be utilized to collect or capture information about each of the material pieces 101.
  • the vision system 110 may be configured (e.g., with a machine learning system) to collect or capture any type of information that can be utilized within the system 100 to selectively sort the material pieces 101 as a function of a set of one or more (user-defined) physical characteristics, including, but not limited to, color, hue, size, shape, texture, overall physical appearance, uniformity, composition, and or manufacturing type of the material pieces 101.
  • the vision system 110 captures images of each of the material pieces 101 (including one-dimensional, two-dimensional, three- dimensional, or holographic imaging), for example, by using an optical sensor as utilized in typical digital cameras and video equipment. Such images captured by the optical sensor are then stored in a memory device as image data.
  • such image data represents images captured within optical wavelengths of light (i.e., the wavelengths of light that are observable by a typical human eye).
  • optical wavelengths of light i.e., the wavelengths of light that are observable by a typical human eye
  • alternative embodiments of the present disclosure may utilize sensors that are capable of capturing an image of a material made up of wavelengths of light outside of the visual wavelengths of the typical human eye.
  • the system 100 may be implemented with one or more sensor systems 120, which may be utilized solely or in combination with the vision system 110 to classify/identify material pieces 101.
  • a sensor system 120 may be configured with any type of sensor technology, including sensors utilizing irradiated or reflected electromagnetic radiation (e.g., utilizing infrared (“IR”), Fourier Transform IR (“FTIR”), Forward-looking Infrared (“FLIR”), Very Near Infrared (“VNIR”), Near Infrared (“NIR”), Short Wavelength Infrared (“SWIR”), Long Wavelength Infrared (“LWIR”), Medium Wavelength Infrared (“MWIR”), X-Ray Transmission (“XRT”), Gamma Ray, Ultraviolet, X-Ray Fluorescence (“XRF”), Laser Induced Breakdown Spectroscopy (“LIBS”), Raman Spectroscopy, Anti-stokes Raman Spectroscopy, Gamma Spectroscopy, Hyper
  • FIG. 1 is illustrated with a combination of a vision system 110 and a sensor system 120
  • embodiments of the present disclosure may be implemented with any combination of sensor systems utilizing any of the sensor technologies disclosed herein, or any other sensor technologies currently available or developed in the future.
  • FIG. 1 is illustrated as including a sensor system 120 separate from the vision system 110, implementation of such a sensor system is optional within certain embodiments of the present disclosure.
  • a combination of both a vision system 110 and one or more sensor systems 120 may be used to classify the material pieces 101.
  • any combination of one or more of the different sensor technologies disclosed herein may be used to classify the material pieces 101 without utilization of a vision system 110.
  • embodiments of the present disclosure may include any combinations of one or more sensor systems and/or vision systems in which the outputs of such sensor and or vision systems are utilized by a machine learning system (as further disclosed herein) in order to classify/identify materials from a mixture of materials, which may then be sorted from each other.
  • a vision system 110 and/or sensor system(s) may be configured to identify which of the material pieces 101 are not of the kind to be sorted by the system 100, and send a signal to reject such material pieces.
  • the identified material pieces 101 may be diverted/ejected utilizing one of the mechanisms as described hereinafter for physically moving sorted material pieces into individual bins.
  • a distance measuring device 111 and accompanying control system 112 may be utilized and configured to measure the sizes and or shapes of each of the material pieces 101 as they pass within proximity of the distance measuring device 111, along with the position (i.e., location and timing) of each of the material pieces 101 on the moving conveyor belt 103.
  • An exemplary operation of such a distance measuring device 111 and control system 112 is further described in U.S. Patent No. 10,207,296.
  • the vision system 110 may be utilized to track the position (i.e., location and timing) of each of the material pieces 101 on the moving conveyor belt 103.
  • Such a distance measuring device 111 may be implemented with a well-known visible light (e.g., laser light) system, which continuously measures a distance the light travels before being reflected back into a detector of the laser light system. As such, as each of the material pieces 101 passes within proximity of the device 111, it outputs a signal to the control system 112 indicating such distance measurements. Therefore, such a signal may substantially represent an intermittent series of pulses whereby the baseline of the signal is produced as a result of a measurement of the distance between the distance measuring device 111 and the conveyor belt 103 during those moments when a material piece 101 is not in the proximity of the device 111, while each pulse provides a measurement of the distance between the distance measuring device 111 and a material piece 101 passing by on the conveyor belt 103.
  • a well-known visible light e.g., laser light
  • each pulse signal generated by the distance measuring device 111 provides the height of portions of each of the material pieces 101 as they pass by on the conveyor belt 103.
  • the length of each of such pulses also provides a measurement of a length of each of the material pieces 101 measured along a line substantially parallel to the direction of travel of the conveyor belt 103.
  • this length measurement (and alternatively the height measurements) that may be utilized within certain embodiments of the present disclosure to determine when to activate and deactivate the acquisition of detected fluorescence (i.e., the XRF spectrum) of each of the material pieces 101 by a sensor system 120 implementing an XRF system so that the detected fluorescence is obtained substantially only from each of the material pieces and not from any background surfaces, such as a conveyor belt 103.
  • detected fluorescence i.e., the XRF spectrum
  • a distance measuring device 111 may be utilized in combination with one or more sensor system(s) 120 even when an additional vision system 110 is not implemented/activated.
  • the sensor system(s) 120 may be configured to assist the vision system 110 to identify the composition, or relative compositions, and/or manufacturing types, of each of the material pieces 101 as they pass within proximity of the sensor system(s) 120.
  • the sensor system(s) 120 may include an energy emitting source 121, which may be powered by a power supply 122, for example, in order to stimulate a response from each of the material pieces 101.
  • the sensor system 120 may emit an appropriate stimulus (e.g., sensing signal) towards the material piece 101.
  • One or more detectors 124 may be positioned and configured to sense/detect one or more physical characteristics from the material piece 101 in a form appropriate for the type of utilized sensor technology.
  • the one or more detectors 124 and the associated detector electronics 125 capture these one or more received sensed characteristics to perform signal processing thereon and produce digitized information representing the sensed characteristics, which is then analyzed in accordance with certain embodiments of the present disclosure, which may be used in order to classify each of the material pieces 101.
  • This classification which may be performed within the computer system 107, may then be utilized by the automation control system 108 to activate one of the N (N>1) sorting devices 126...129 for sorting (e.g., diverting/ejecting) the material pieces 101 into one or more N (N>1) sorting receptacles 136...139 according to the determined classifications.
  • N (N>1) sorting devices 126...129 for sorting (e.g., diverting/ejecting) the material pieces 101 into one or more N (N>1) sorting receptacles 136...139 according to the determined classifications.
  • Four sorting devices 126...129 and four sorting receptacles 136...139 associated with the sorting devices are illustrated in FIG. 1 as merely a non limiting example.
  • the sorting devices may include any well-known mechanisms for redirecting selected material pieces 101 towards a desired location, including, but not limited to, diverting the material pieces 101 from the conveyor belt system into the plurality of sorting receptacles
  • a sorting device may utilize air jets, with each of the air jets assigned to one or more of the classifications.
  • one of the air jets e.g., 127) receives a signal from the automation control system 108, that air jet emits a stream of air that causes a material piece 101 to be diverted/ejected from the conveyor system 103 into a sorting receptacle (e.g., 137) corresponding to that air jet.
  • High speed air valves from Mac Industries may be used, for example, to supply the air jets with an appropriate air pressure configured to divert/eject the material pieces 101 from the conveyor system 103.
  • FIG. 1 uses air jets to divert/eject material pieces
  • other mechanisms may be used to divert/eject the material pieces, such as robotically removing the material pieces from the conveyor belt, pushing the material pieces from the conveyor belt (e.g., with paint brush type plungers), causing an opening (e.g., a trap door) in the conveyor system 103 from which a material piece may drop, or using air jets to separate the material pieces into separate receptacles as they fall from the edge of the conveyor belt.
  • a pusher device may refer to any form of device which may be activated to dynamically displace an object on or from a conveyor system/device, employing pneumatic, mechanical, or other means to do so, such as any appropriate type of mechanical pushing mechanism (e.g., an ACME screw drive), pneumatic pushing mechanism, or air jet pushing mechanism.
  • Some embodiments may include multiple pusher devices located at different locations and/or with different diversion path orientations along the path of the conveyor system. In various different implementations, these sorting systems describe herein may determine which pusher device to activate (if any) depending on characteristics of material pieces identified by the machine learning system.
  • the determination of which pusher device to activate may be based on the detected presence and or characteristics of other objects that may also be within the diversion path of a pusher device concurrently with a target item.
  • the disclosed sorting systems can recognize when multiple objects are not well singulated, and dynamically select from a plurality of pusher devices which should be activated based on which pusher device provides the best diversion path for potentially separating objects within close proximity.
  • objects identified as target objects may represent material that should be diverted off of the conveyor system.
  • objects identified as target objects represent material that should be allowed to remain on the conveyor system so that non-target materials are instead diverted.
  • the system 100 may also include a receptacle 140 that receives material pieces 101 not diverted/ejected from the conveyor system 103 into any of the aforementioned sorting receptacles 136...139.
  • a material piece 101 may not be diverted/ejected from the conveyor system 103 into one of the N sorting receptacles 136...139 when the classification of the material piece 101 is not determined (or simply because the sorting devices failed to adequately divert/eject a piece).
  • the receptacle 140 may serve as a default receptacle into which unclassified material pieces are dumped.
  • the receptacle 140 may be used to receive one or more classifications of material pieces that have deliberately not been assigned to any of the N sorting receptacles 136...139. These such material pieces may then be further sorted in accordance with other characteristics and/or by another sorting system.
  • multiple classifications may be mapped to a single sorting device and associated sorting receptacle.
  • the same sorting device may be activated to sort these into the same sorting receptacle.
  • Such combination sorting may be applied to produce any desired combination of sorted material pieces.
  • the mapping of classifications may be programmed by the user (e.g., using the algorithm(s) (e.g., see FIG. 6) operated by the computer system 107) to produce such desired combinations. Additionally, the classifications of material pieces are user-definable, and not limited to any particular known classifications of material pieces.
  • the conveyor system 103 may include a circular conveyor (not shown) so that unclassified material pieces are returned to the beginning of the system 100 and run through the system 100 again. Moreover, because the system 100 is able to specifically track each material piece 101 as it travels on the conveyor system 103, some sort of sorting device (e.g., the sorting device 129) may be implemented to direct/eject a material piece 101 that the system 100 has failed to classify after a predetermined number of cycles through the system 100 (or the material piece 101 is collected in receptacle 140).
  • some sort of sorting device e.g., the sorting device 129 may be implemented to direct/eject a material piece 101 that the system 100 has failed to classify after a predetermined number of cycles through the system 100 (or the material piece 101 is collected in receptacle 140).
  • the conveyor system 103 may be divided into multiple belts configured in series such as, for example, two belts, where a first belt conveys the material pieces past the vision system 110 and or an implemented sensor system 120, and a second belt conveys the material pieces from the vision system 110 and or an implemented sensor system 120 to the sorting devices. Moreover, such a second conveyor belt may be at a lower height than the first conveyor belt, such that the material pieces fall from the first belt onto the second belt.
  • the emitting source 121 may be located above the detection area (i.e., above the conveyor system 103); however, certain embodiments of the present disclosure may locate the emitting source 121 and or detectors 124 in other positions that still produce acceptable sensed/detected physical characteristics.
  • signals representing the detected XRF spectrum may be converted into a discrete energy histogram such as on a per-channel (i.e., element) basis, as further described herein.
  • a conversion process may be implemented within the control system 123 or the computer system 107.
  • a control system 123 or computer system 107 may include a commercially available spectrum acquisition module, such as the commercially available Amptech MCA 5000 acquisition card and software programmed to operate the card.
  • Such a spectrum acquisition module, or other software implemented within the system 100 may be configured to implement a plurality of channels for dispersing x-rays into a discrete energy spectrum (i.e., histogram) with such a plurality of energy levels, whereby each energy level corresponds to an element that the system 100 has been configured to detect.
  • the system 100 may be configured so that there are sufficient channels corresponding to certain elements within the chemical periodic table, which are important for distinguishing between different materials.
  • the energy counts for each energy level may be stored in a separate collection storage register. The computer system 107 then reads each collection register to determine the number of counts for each energy level during the collection interval, and build the energy histogram.
  • a sorting algorithm configured in accordance with certain embodiments of the present disclosure may then utilize this collected histogram of energy levels to classify at least certain ones of the material pieces 101 and/or assist the vision system 110 in classifying the material pieces 101.
  • the source 121 may include an in-line x-ray fluorescence (“IL-XRF”) tube, such as further described within U.S. Patent No. 10,207,296.
  • IL-XRF in-line x-ray fluorescence
  • Such an IL-XRF tube may include a separate x- ray source each dedicated for one or more streams (e.g., singulated) of conveyed material pieces.
  • the one or more detectors 124 may be implemented as XRF detectors to detect fluoresced x-rays from material pieces 101 within each of the singulated streams. Examples of such XRF detectors are further described within U.S. Patent No. 10,207,296.
  • the systems and methods described herein may be applied to classify and or sort individual material pieces having any of a variety of sizes. Even though the systems and methods described herein are described primarily in relation to sorting individual material pieces of a stream one at a time, the systems and methods described herein are not limited thereto. Such systems and methods may be used to stimulate and/or detect emissions from a plurality of materials concurrently. For example, as opposed to a singulated stream of materials being conveyed along one or more conveyor belts in series, multiple singulated streams may be conveyed in parallel. Each stream may be on a same belt or on different belts arranged in parallel. Further, pieces may be randomly distributed on (e.g., across and along) one or more conveyor systems.
  • the systems and methods described herein may be used to collect characteristics from a plurality of these small pieces at the same time.
  • a plurality of small pieces may be treated as a single piece as opposed to each small piece being considered individually.
  • the plurality of small pieces of material may be classified and sorted (e.g., diverted/ejected from the conveyor system) together. It should be appreciated that a plurality of larger material pieces also may be treated as a single material piece.
  • certain embodiments of the present disclosure may implement one or more vision systems (e.g., vision system 110) in order to identify, track, and/or classify material pieces.
  • vision system(s) may operate alone to identify and/or classify and sort material pieces, or may operate in combination with a sensor system (e.g., sensor system 120) to identify and or classify and sort material pieces.
  • a material handling system e.g., system 100
  • the sensor system 120 may be omitted from the system 100 (or simply deactivated).
  • Such a vision system may be configured with one or more devices for capturing or acquiring images of the material pieces as they pass by on a conveyor system.
  • the devices may be configured to capture or acquire any desired range of wavelengths irradiated or reflected by the material pieces, including, but not limited to, visible, infrared (“IR”), ultraviolet (“UV”) light.
  • the vision system may be configured with one or more cameras (still and or video, either of which may be configured to capture two-dimensional, three-dimensional, and/or holographical images) positioned in proximity (e.g., above) the conveyor system so that images of the material pieces are captured (e.g., as image data) as they pass by the sensor system(s).
  • material characteristics captured by a sensor system 120 may be processed (converted) into data to be utilized (either solely or in combination with the image data captured by the vision system 110) for classifying/sorting of the material pieces.
  • Such an implementation may be in lieu of, or in combination with, utilizing the sensor system 120 for classifying material pieces.
  • the information may then be sent to a computer system (e.g., computer system 107) to be processed (e.g., by a machine learning system) in order to identify and or classify each of the material pieces.
  • a computer system e.g., computer system 107
  • process e.g., by a machine learning system
  • a machine learning system may implement any well-known machine learning technique or technology, including one that implements a neural network (e.g., artificial neural network, deep neural network, convolutional neural network, recurrent neural network, autoencoders, reinforcement learning, etc.), supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, self learning, feature learning, sparse dictionary learning, anomaly detection, robot learning, association rule learning, fuzzy logic, artificial intelligence (“AI”), deep learning algorithms, deep structured learning hierarchical learning algorithms, support vector machine (“SVM”) (e.g., linear SVM, nonlinear SVM, SVM regression, etc.), decision tree learning (e.g., classification and regression tree (“CART”), ensemble methods (e.g., ensemble learning, Random Forests, Bagging and Pasting, Patches and Subspaces, Boosting, Stacking, etc.), dimensionality reduction (e.g., Projection, Manifold Learning, Principal Components Analysis, etc.) and or deep machine learning algorithms, such as those described in and publicly available
  • Non-limiting examples of publicly available machine learning software and libraries that could be utilized within embodiments of the present disclosure include Python, OpenCV, Inception, Theano, Torch, PyTorch, Pylearn2, Numpy, Blocks, TensorFlow, MXNet, Caffe, Lasagne, Keras, Chainer, Matlab Deep Learning, CNTK, MatConvNet (a MATLAB toolbox implementing convolutional neural networks for computer vision applications), DeepLearnToolbox (a Matlab toolbox for Deep Learning (from Rasmus Berg Palm)), BigDL, Cuda- Convnet (a fast C++/CUDA implementation of convolutional (or more generally, feed-forward) neural networks), Deep Belief Networks, RNNLM, RNNLIB-RNNLIB, matrbm, deeplearning 4j, Eblearn.lsh, deepmat, MShadow, Matplotlib, SciPy, CXXNET, Nengo-Nengo, Eblearn, cudamat, Gnumpy, 3-way factor
  • Machine learning often occurs in two stages. For example, first, training occurs, which may be performed offline in that the system 100 is not being utilized to perform actual classifying/sorting of material pieces.
  • the system 100 may be utilized to train the machine learning system in that one or more examples or sets (which may also be referred to herein as control samples) of material pieces (i.e., having the same types or classes of materials) are passed through the system 100 (e.g., by a conveyor system 103); and all such material pieces may not be sorted, but may be collected in a common receptacle (e.g., receptacle 140).
  • the training may be performed at another location remote from the system 100, including using some other mechanism for collecting sensed information (characteristics) of sets of material pieces.
  • one or more algorithms within the machine learning system extract features from the captured information (e.g., using image processing techniques well known in the art).
  • training algorithms include, but are not limited to, linear regression, gradient descent, feed forward, polynomial regression, learning curves, regularized learning models, and logistic regression. It is during this training stage that the one or more algorithms within the machine learning system learn the relationships between different types of materials and their features/characteristics (e.g., as captured by the vision system and/or sensor system(s)), generating a knowledge base for later classification of a heterogeneous mixture of material pieces received by the system 100.
  • Such a previously generated knowledge base may include one or more libraries, wherein each library includes parameters (e.g., “neural network parameters”) for utilization by the machine learning system in classifying material pieces.
  • each library includes parameters (e.g., “neural network parameters”) for utilization by the machine learning system in classifying material pieces.
  • one particular library may include parameters configured by the training stage to recognize and classify a particular type or class of material.
  • such libraries may be inputted into the machine learning system and then the user of the system 100 may be able to adjust certain ones of the parameters in order to adjust an operation of the system 100 (for example, adjusting the threshold effectiveness of how well the machine learning system identifies/classifies a particular material from a mixture of materials).
  • a machine learning system configured in accordance with certain embodiments of the present disclosure may be configured to sort between material pieces as a function of their respective material/chemical compositions. For example, such a machine learning system may be configured so that material pieces containing a particular element can be sorted as a function of the percentage (e.g., weight or volume percentage) of that element contained within the material pieces.
  • examples of one or more material pieces 201 of a particular class or type of material may be delivered past the vision system and/or one or more sensor system(s) (e.g., by a conveyor system 203) so that the one or more algorithms within the machine learning system detect, extract, and learn what characteristics or features represent such a type or class of material.
  • the material pieces 201 may be any of the “materials” disclosed herein.
  • each of the material pieces 201 may represent one or more particular type or class of plastic, which are passed through such a training stage so that the one or more algorithms within the machine learning system “learn” how to detect, recognize, and classify such classes or types of plastic.
  • This creates a library of parameters particular to those classes or types of plastic.
  • the same process may be performed with respect to a certain class, or type, of metal alloy, creating a library of parameters particular to that class, or type, of metal alloy, and so on.
  • any number of exemplary material pieces of that class or type of material may be passed by the vision system and or one or more sensor system(s).
  • the machine learning algorithm(s) may use N classifiers, each of which test for one of N different material classes or types.
  • the libraries of parameters for the different materials may be then implemented into a material classifying and or sorting system (e.g., system 100) to be used for identifying and/or classifying material pieces from a mixture of material pieces, and then possibly sorting such classified material pieces if sorting is to be performed.
  • a material classifying and or sorting system e.g., system 100
  • characteristics captured by a sensor and/or vision system with respect to a particular material piece may be processed as an array of data values.
  • the data may be image data captured by a digital camera or other type of imaging sensor with respect to a particular material piece and processed as an array of pixel values.
  • Each data value may be represented by a single number, or as a series of numbers representing values.
  • These values are multiplied by the neuron weight parameters, and may possibly have a bias added. This is fed into a neuron nonlinearity.
  • the resulting number output by the neuron can be treated much as the values were, with this output multiplied by subsequent neuron weight values, a bias optionally added, and once again fed into a neuron nonlinearity.
  • Each such iteration of the process is known as a “layer” of the neural network.
  • the final outputs of the final layer may be interpreted as probabilities that a material is present or absent in the captured data pertaining to the material piece. Examples of such a process are described in detail in both of the previously noted “ImageNet Classification with Deep Convolutional Networks ” and “Gradient-Based Learning Applied to Document Recognition ” references.
  • the final set of neurons’ outputs is trained to represent the likelihood a material piece is associated with the captured data.
  • the likelihood that a material piece is associated with the captured data is over a user-specified threshold, then it is determined that the particular material piece is indeed associated with the captured data.
  • a sensor system may utilize optical spectrometric techniques using multi- or hyper-spectral cameras to provide a signal that may indicate the presence or absence of a type of material (e.g., containing one or more particular elements) by examining the spectral emissions of the material.
  • Photographs of a material piece may also be used in a template-matching algorithm, wherein a database of images is compared against an acquired image to find the presence or absence of certain types of materials from that database.
  • a histogram of the captured image may also be compared against a database of histograms.
  • a bag of words model may be used with a feature extraction technique, such as scale-invariant feature transform (“SIFT”), to compare extracted features between a captured image and those in a database.
  • SIFT scale-invariant feature transform
  • training of the machine learning system may be performed utilizing a labeling/annotation technique (or any other supervised learning technique) whereby as data/information of material pieces (e.g., containing one or more particular types of contaminant) are captured by a vision/sensor system, a user inputs a label or annotation that identifies each material piece, which is then used to create the library for use by the machine learning system when classifying material pieces within a heterogenous mixture of material pieces.
  • a previously generated knowledge base of characteristics captured from one or more samples of a class of materials may be accomplished by any of the techniques disclosed herein, whereby such a knowledge base is then utilized to automatically classify
  • certain embodiments of the present disclosure provide for the identification/classification of one or more different materials in order to determine which material pieces should be diverted from a conveyor system or device.
  • machine learning techniques may be utilized to train (i.e., configure) a neural network to identify a variety of one or more different classes or types of materials. Images, or other types of sensed information, may be captured of materials (e.g., traveling on a conveyor system), and based on the identification/classification of such materials, the systems described herein can decide which material piece should be allowed to remain on the conveyor system, and which should be diverted/removed from the conveyor system (for example, either into a collection receptacle, or diverted onto another conveyor system).
  • a machine learning system for an existing installation may be dynamically reconfigured to detect and recognize characteristics of a new class or type of material by replacing a current set of neural network parameters with a new set of neural network parameters.
  • the collected/captured/detected/extracted features/characteristics of the material pieces may not be necessarily simply particularly identifiable physical characteristics; they can be abstract formulations that can only be expressed mathematically, or not mathematically at all; nevertheless, the machine learning system may be configured to parse all of the data to look for patterns that allow the control samples to be classified during the training stage. Furthermore, the machine learning system may take subsections of captured information of a material piece and attempt to find correlations between the pre-defined classifications.
  • any sensed characteristics captured by any of the sensor systems 120 disclosed herein may be input into a machine learning system in order to classify and/or sort materials.
  • sensor system 120 outputs that uniquely characterize a particular type or class of material may be used to train the machine learning system.
  • an electronic machine vision apparatus is commonly employed in conjunction with an automatic machining, assembly and inspection apparatus, particularly of the robotics type.
  • Television cameras are commonly employed to observe the object being machined, assembled, read, viewed, or inspected, and the signal received and transmitted by the camera can be compared to a standard signal or database to determine if the imaged article is properly machined, finished, oriented, assembled, determined, etc.
  • a machine vision apparatus is widely used in inspection and flaw detection applications whereby inconsistencies and imperfections in both hard and soft goods can be rapidly ascertained and adjustments or rejections instantaneously effected.
  • a machine vision apparatus detects abnormalities by comparing the signal generated by the camera with a predetermined signal indicating proper dimensions, appearance, orientation, or the like. See International Published Patent Application WO 99/2248, which is hereby incorporated by reference herein. Nevertheless, machine vision systems do not perform any sort of further data processing (e.g., image processing) that would include further processing of the captured information through an algorithm. See definition of Machine Vision in Wikipedia, which is hereby incorporated by reference herein. Therefore, it can be readily appreciated that a machine vision apparatus or system does not further include any sort of algorithm, such as a machine learning algorithm. Instead, a machine vision system essentially compares images of parts to templates of images.
  • FIG. 3 illustrates a flowchart diagram depicting exemplary embodiments of a process 3500 of classifying/sorting material pieces utilizing a vision system and or sensor system in accordance with certain embodiments of the present disclosure.
  • the process 3500 may be configured to operate within any of the embodiments of the present disclosure described herein, including the system 100 of FIG. 1. Operation of the process 3500 may be performed by hardware and/or software, including within a computer system (e.g., computer system 3400 of FIG. 5) controlling the sorting system (e.g., the computer system 107, the vision system 110, and or the sensor system(s) 120 of FIG. 1).
  • the material pieces may be deposited onto a conveyor system.
  • the location on the conveyor system of each material piece is detected for tracking of each material piece as it travels through the sorting system. This may be performed by the vision system 110 (for example, by distinguishing a material piece from the underlying conveyor system material while in communication with a conveyor system position detector (e.g., the position detector 105)). Alternatively, a linear sheet laser beam can be used to locate the pieces. Or, any system that can create a light source (including, but not limited to, visual light, UV, and IR) and have a detector that can be used to locate the pieces. In the process block 3503, when a material piece has traveled in proximity to one or more of the vision system and or the sensor system(s), sensed information/characteristics of the material piece is captured/acquired.
  • the vision system 110 for example, by distinguishing a material piece from the underlying conveyor system material while in communication with a conveyor system position detector (e.g., the position detector 105)).
  • a linear sheet laser beam can be used to locate the pieces.
  • a vision system may perform pre-processing of the captured information, which may be utilized to detect (extract) each of the material pieces (e.g., from the background (e.g., the conveyor belt); in other words, the pre-processing may be utilized to identify the difference between the material piece and the background).
  • Well-known image processing techniques such as dilation, thresholding, and contouring may be utilized to identify the material piece as being distinct from the background.
  • segmentation may be performed.
  • the captured information may include information pertaining to one or more material pieces. Additionally, a particular material piece may be located on a seam of the conveyor belt when its image is captured.
  • a first step is to apply a high contrast of the image; in this fashion, background pixels are reduced to substantially all black pixels, and at least some of the pixels pertaining to the material piece are brightened to substantially all white pixels.
  • the image pixels of the material piece that are white are then dilated to cover the entire size of the material piece.
  • the location of the material piece is a high contrast image of all white pixels on a black background.
  • a contouring algorithm can be utilized to detect boundaries of the material piece. The boundary information is saved, and the boundary locations are then transferred to the original image. Segmentation is then performed on the original image on an area greater than the boundary that was earlier defined. In this fashion, the material piece is identified and separated from the background.
  • the material pieces may be conveyed along the conveyor system within proximity of a distance measuring device and/or a sensor system in order to determine a size and/or shape of the material pieces, which may be useful if an XRF system or some other spectroscopy sensor is also implemented within the sorting system.
  • post processing may be performed. Post processing may involve resizing the captured information/data to prepare it for use in the neural networks. This may also include modifying certain properties (e.g., enhancing image contrast, changing the image background, or applying filters) in a manner that will yield an enhancement to the capability of the machine learning system to classify the material pieces.
  • the data may be resized.
  • Data resizing may be desired under certain circumstances to match the data input requirements for certain machine learning systems, such as neural networks.
  • neural networks may require much smaller image sizes (e.g., 225 x 255 pixels or 299 x 299 pixels) than the sizes of the images captured by typical digital cameras.
  • image sizes e.g., 225 x 255 pixels or 299 x 299 pixels
  • the smaller the input data size the less processing time is needed to perform the classification.
  • smaller data sizes can ultimately increase the throughput of the system 100 and increase its value.
  • the process block 3510 may be configured with a neural network employing one or more machine learning algorithms, which compare the extracted features with those stored in the knowledge base generated during the training stage, and assigns the classification with the highest match to each of the material pieces based on such a comparison.
  • the algorithms of the machine learning system may process the captured information/data in a hierarchical manner by using automatically trained filters. The filter responses are then successfully combined in the next levels of the algorithms until a probability is obtained in the final step.
  • these probabilities may be used for each of the N classifications to decide into which of the N sorting receptacles the respective material pieces should be sorted.
  • each of the N classifications may be assigned to one sorting receptacle, and the material piece under consideration is sorted into that receptacle that corresponds to the classification returning the highest probability larger than a predefined threshold.
  • predefined thresholds may be preset by the user.
  • a particular material piece may be sorted into an outlier receptacle (e.g., sorting receptacle 140) if none of the probabilities is larger than the predetermined threshold.
  • a sorting device corresponding to the classification, or classifications, of the material piece is activated.
  • the material piece has moved from the proximity of the vision system and/or sensor system(s) to a location downstream on the conveyor system (e.g., at the rate of conveying of a conveyor system).
  • the activation of the sorting device is timed such that as the material piece passes the sorting device mapped to the classification of the material piece, the sorting device is activated, and the material piece is diverted/ejected from the conveyor system into its associated sorting receptacle.
  • the activation of a sorting device may be timed by a respective position detector that detects when a material piece is passing before the sorting device and sends a signal to enable the activation of the sorting device.
  • the sorting receptacle corresponding to the sorting device that was activated receives the diverted/ejected material piece.
  • FIG. 4 illustrates a flowchart diagram depicting exemplary embodiments of a process 400 of sorting material pieces in accordance with certain embodiments of the present disclosure.
  • the process 400 may be configured to operate within any of the embodiments of the present disclosure described herein, including the system 100 of FIG. 1.
  • the process 400 may be configured to operate in conjunction with the process 3500.
  • the process blocks 403 and 404 may be incorporated in the process 3500 (e.g., operating in series or in parallel with the process blocks 3503-3510) in order to combine the efforts of a vision system 110 that is implemented in conjunction with a machine learning system with a sensor system (e.g., the sensor system 120) that is not implemented in conjunction with a machine learning system in order to classify and or sort material pieces.
  • a vision system 110 that is implemented in conjunction with a machine learning system with a sensor system (e.g., the sensor system 120) that is not implemented in conjunction with a machine learning system in order to classify and or sort material pieces.
  • a sensor system e.g., the sensor system 120
  • Operation of the process 400 may be performed by hardware and or software, including within a computer system (e.g., computer system 3400 of FIG. 5) controlling the sorting system (e.g., the computer system 107 of FIG. 1).
  • the material pieces may be deposited onto a conveyor system.
  • the material pieces may be conveyed along the conveyor system within proximity of a distance measuring device and or an optical imaging system in order to determine a size and or shape of the material pieces.
  • the material piece may be interrogated, or stimulated, with EM energy (waves) or some other type of stimulus appropriate for the particular type of sensor technology utilized by the sensor system.
  • process block 404 physical characteristics of the material piece are sensed/detected and captured by the sensor system.
  • the type of material is identified/classified based (at least in part) on the captured characteristics, which may be combined with the classification by the machine learning system in conjunction with the vision system 110.
  • a sorting device corresponding to the classification, or classifications, of the material piece is activated. Between the time at which the material piece was sensed and the time at which the sorting device is activated, the material piece has moved from the proximity of the sensor system to a location downstream on the conveyor system, at the rate of conveying of the conveyor system.
  • the activation of the sorting device is timed such that as the material piece passes the sorting device mapped to the classification of the material piece, the sorting device is activated, and the material piece is diverted/ejected from the conveyor system into its associated sorting receptacle.
  • the activation of a sorting device may be timed by a respective position detector that detects when a material piece is passing before the sorting device and sends a signal to enable the activation of the sorting device.
  • the sorting receptacle corresponding to the sorting device that was activated receives the diverted/ejected material piece.
  • a plurality of at least a portion of the system 100 may be linked together in succession in order to perform multiple iterations or layers of sorting.
  • a conveyor system may be implemented with a single conveyor belt, or multiple conveyor belts, conveying the material pieces past a first vision system (and, in accordance with certain embodiments, a sensor system) configured for sorting material pieces of a first set of a heterogeneous mixture of materials by a sorter (e.g., the first automation control system 108 and associated one or more sorting devices 126...129) into a first set of one or more receptacles (e.g., sorting receptacles 136...139), and then conveying the material pieces past a second vision system (and, in accordance with certain embodiments, another sensor system ) configured for sorting material pieces of a second set of a heterogeneous mixture of materials by a second sorter into a
  • each successive vision system may be configured to sort out a different classified or type of material than previous vision system(s).
  • different types or classes of materials may be classified by different types of sensors each for use with a machine learning system, and combined to classify material pieces in a stream of scrap or waste.
  • data from two or more sensors can be combined using a single or multiple machine learning systems to perform classifications of material pieces.
  • multiple sensor systems can be mounted onto a single conveyor system, with each sensor system utilizing a different machine learning system.
  • multiple sensor systems can be mounted onto different conveyor systems, with each sensor system utilizing a different machine learning system.
  • Certain embodiments of the present disclosure may be configured to produce a mass of materials having a content of less than a predetermined weight or volume percentage of a certain element or material after sorting.
  • FIG. 5 a block diagram illustrating a data processing (“computer”) system 3400 is depicted in which aspects of embodiments of the disclosure may be implemented.
  • the computer system 107, the automation control system 108, aspects of the sensor system(s) 120, and/or the vision system 110 may be configured similarly as the computer system 3400.
  • the computer system 3400 may employ a local bus 3405 (e.g., a peripheral component interconnect (“PCI”) local bus architecture). Any suitable bus architecture may be utilized such as Accelerated Graphics Port (“AGP”) and Industry Standard Architecture (“ISA”), among others.
  • AGP Accelerated Graphics Port
  • ISA Industry Standard Architecture
  • One or more processors 3415, volatile memory 3420, and non-volatile memory 3435 may be connected to the local bus 3405 (e.g., through a PCI Bridge (not shown)).
  • An integrated memory controller and cache memory may be coupled to the one or more processors 3415.
  • the one or more processors 3415 may include one or more central processor units and/or one or more graphics processor units and/or one or more tensor processing units. Additional connections to the local bus 3405 may be made through direct component interconnection or through add-in boards.
  • a communication (e.g., network (LAN)) adapter 3425, an I/O (e.g., small computer system interface (“SCSI”) host bus) adapter 3430, and expansion bus interface (not shown) may be connected to the local bus 3405 by direct component connection.
  • An audio adapter (not shown), a graphics adapter (not shown), and display adapter 3416 (coupled to a display 3440) may be connected to the local bus 3405 (e.g., by add-in boards inserted into expansion slots).
  • the user interface adapter 3412 may provide a connection for a keyboard 3413 and a mouse 3414, modem/router (not shown), and additional memory (not shown).
  • the I/O adapter 3430 may provide a connection for a hard disk drive 3431, a tape drive 3432, and a CD-ROM drive (not shown).
  • One or more operating systems may be run on the one or more processors 3415 and used to coordinate and provide control of various components within the computer system 3400.
  • the operating system(s) may be a commercially available operating system.
  • An object-oriented programming system e.g., Java, Python, etc.
  • Java, Python, etc. may run in conjunction with the operating system and provide calls to the operating system from programs or programs (e.g., Java, Python, etc.) executing on the system 3400.
  • Instructions for the operating system, the object-oriented operating system, and programs may be located on non-volatile memory 3435 storage devices, such as a hard disk drive 3431, and may be loaded into volatile memory 3420 for execution by the processor 3415.
  • FIG. 5 may vary depending on the implementation.
  • Other internal hardware or peripheral devices such as flash ROM (or equivalent nonvolatile memory) or optical disk drives and the like, may be used in addition to or in place of the hardware depicted in FIG. 5.
  • any of the processes of the present disclosure may be applied to a multiprocessor computer system, or performed by a plurality of such systems 3400. For example, training of the vision system 110 may be performed by a first computer system 3400, while operation of the vision system 110 for classifying may be performed by a second computer system 3400.
  • the computer system 3400 may be a stand-alone system configured to be bootable without relying on some type of network communication interface, whether or not the computer system 3400 includes some type of network communication interface.
  • the computer system 3400 may be an embedded controller, which is configured with ROM and/or flash ROM providing non-volatile memory storing operating system files or user-generated data.
  • FIG. 5 The depicted example in FIG. 5 and above-described examples are not meant to imply architectural limitations. Further, a computer program form of aspects of the present disclosure may reside on any computer readable storage medium (i.e., floppy disk, compact disk, hard disk, tape, ROM, RAM, etc.) used by a computer system.
  • any computer readable storage medium i.e., floppy disk, compact disk, hard disk, tape, ROM, RAM, etc.
  • embodiments of the present disclosure may be implemented to perform the various functions described for identifying, tracking, classifying, and/or sorting material pieces.
  • Such functionalities may be implemented within hardware and or software, such as within one or more data processing systems (e.g., the data processing system 3400 of FIG. 5), such as the previously noted computer system 107, the vision system 110, aspects of the sensor system(s) 120, and or the automation control system 108. Nevertheless, the functionalities described herein are not to be limited for implementation into any particular hardware/software platform.
  • aspects of the present disclosure may be embodied as a system, process, method, and or program product. Accordingly, various aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.), or embodiments combining software and hardware aspects, which may generally be referred to herein as a “circuit,” “circuitry,” “module,” or “system.” Furthermore, aspects of the present disclosure may take the form of a program product embodied in one or more computer readable storage medium(s) having computer readable program code embodied thereon. (However, any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium.)
  • a computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, biologic, atomic, or semiconductor system, apparatus, controller, or device, or any suitable combination of the foregoing, wherein the computer readable storage medium is not a transitory signal per se. More specific examples (a non-exhaustive list) of the computer readable storage medium may include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (“RAM”) (e.g., RAM 3420 of FIG. 5), a read-only memory (“ROM”) (e.g., ROM 3435 of FIG.
  • RAM random access memory
  • ROM read-only memory
  • a computer readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, controller, or device.
  • Program code embodied on a computer readable signal medium may be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
  • a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof.
  • a computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, controller, or device.
  • each block in the flowcharts or block diagrams may represent a module, segment, or portion of code, which includes one or more executable program instructions for implementing the specified logical function! s).
  • the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • Modules implemented in software for execution by various types of processors may, for instance, include one or more physical or logical blocks of computer instructions, which may, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together, but may include disparate instructions stored in different locations which, when joined logically together, include the module and achieve the stated purpose for the module. Indeed, a module of executable code may be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices.
  • operational data e.g., material classification libraries described herein
  • modules may be identified and illustrated herein within modules, and may be embodied in any suitable form and organized within any suitable type of data structure.
  • the operational data may be collected as a single data set, or may be distributed over different locations including over different storage devices.
  • the data may provide electronic signals on a system or network.
  • program instructions may be provided to one or more processors and/or controller(s) of a general purpose computer, special purpose computer, or other programmable data processing apparatus (e.g., controller) to produce a machine, such that the instructions, which execute via the processor(s) (e.g., GPU 3401, CPU 3415) of the computer or other programmable data processing apparatus, create circuitry or means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
  • processors e.g., GPU 3401, CPU 3415
  • each block of the block diagrams and or flowchart illustrations, and combinations of blocks in the block diagrams and or flowchart illustrations can be implemented by special purpose hardware -based systems (e.g., which may include one or more graphics processing units (e.g., GPU 3401)) that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
  • a module may be implemented as a hardware circuit including custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, controllers, or other discrete components.
  • a module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, or the like.
  • a flow-charted technique may be described in a series of sequential actions.
  • the sequence of the actions, and the element performing the actions may be freely changed without departing from the scope of the teachings.
  • Actions may be added, deleted, or altered in several ways.
  • the actions may be re-ordered or looped.
  • processes, methods, algorithms, or the like may be described in a sequential order, such processes, methods, algorithms, or any combination thereof may be operable to be performed in alternative orders.
  • some actions within a process, method, or algorithm may be performed simultaneously during at least a point in time (e.g., actions performed in parallel), and can also be performed in whole, in part, or any combination thereof.
  • Computer program code i.e., instructions, for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, Python, C++, or the like, conventional procedural programming languages, such as the “C” programming language or similar programming languages, programming languages such as MATLAB or Lab VIEW, or any of the machine learning software disclosed herein.
  • the program code may execute entirely on the user’s computer system, partly on the user’s computer system, as a stand-alone software package, partly on the user’s computer system (e.g., the computer system utilized for sorting) and partly on a remote computer system (e.g., the computer system utilized to train the machine learning system), or entirely on the remote computer system or server.
  • the remote computer system may be connected to the user’s computer system through any type of network, including a local area network (“LAN”) or a wide area network (“WAN”), or the connection may be made to an external computer system (for example, through the Internet using an Internet Service Provider).
  • LAN local area network
  • WAN wide area network
  • various aspects of the present disclosure may be configured to execute on one or more of the computer system 107, automation control system 108, the vision system 110, and aspects of the sensor system(s) 120.
  • program instructions may also be stored in a computer readable storage medium that can direct a computer system, other programmable data processing apparatus, controller, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
  • the program instructions may also be loaded onto a computer, other programmable data processing apparatus, controller, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and or block diagram block or blocks.
  • One or more databases may be included in a host for storing and providing access to data for the various implementations.
  • any databases, systems, or components of the present disclosure may include any combination of databases or components at a single location or at multiple locations, wherein each database or system may include any of various suitable security features, such as firewalls, access codes, encryption, de-encryption and the like.
  • the database may be any type of database, such as relational, hierarchical, object-oriented, and or the like. Common database products that may be used to implement the databases include DB2 by IBM, any of the database products available from Oracle Corporation, Microsoft Access by Microsoft Corporation, or any other database product.
  • the database may be organized in any suitable manner, including as data tables or lookup tables.
  • Association of certain data may be accomplished through any data association technique known and practiced in the art.
  • the association may be accomplished either manually or automatically.
  • Automatic association techniques may include, for example, a database search, a database merge, GREP, AGREP, SQL, and/or the like.
  • the association step may be accomplished by a database merge function, for example, using a key field in each of the manufacturer and retailer data tables.
  • a key field partitions the database according to the high-level class of objects defined by the key field.
  • a certain class may be designated as a key field in both the first data table and the second data table, and the two data tables may then be merged on the basis of the class data in the key field.
  • the data corresponding to the key field in each of the merged data tables is preferably the same.
  • data tables having similar, though not identical, data in the key fields may also be merged by using AGREP, for example.
  • aspects of the present disclosure provide a computer program product stored on a computer readable storage medium, which when executed performs a method for classifying materials for sorting, wherein the method includes receiving image data of each piece of a heterogeneous mixture of materials moving in a stream past a vision system camera, wherein the materials in the heterogeneous mixture of materials are a collection of different classes or types of materials; classifying the materials that have a specified class or type of material into a first classification; classifying the materials that do not have the specified class or type of material into a second classification, wherein the classifying of the materials is performed by a neural network employing one or more algorithms that compare features detected in the image data captured from each piece of the heterogeneous mixture of materials with those stored in a knowledge base generated during a training stage; and sending to an automated sorting device information regarding the first and second classifications so that the automated sorting device can sort the materials classified into the first classification from the materials classified into the second classification.
  • the one or more algorithms learn relationships between the one or more specified classes or types of materials and their features extracted from the captured image data that creates the knowledge base.
  • control samples are delivered past the camera so that the one or more algorithms detect, extract, and learn what features visually represent the one or more specified classes or types of materials to be classified into the first classification.
  • the materials classified into the first classification may be metal scrap pieces of a first alloy, and wherein the materials classified into the second classification may be metal scrap pieces of a second alloy that is different than the first alloy.
  • aspects of the present disclosure provide a computer program product stored on a computer readable storage medium, which when executed performs a method for handling a heterogeneous mixture of separable materials comprising at least one of a first type of materials and at least one of a second type of materials, wherein the method includes receiving a characteristic of each of the heterogeneous mixture of materials captured with a sensor; and assigning with a machine learning system a classification to certain ones of the heterogeneous mixture of materials as belonging to the first type of materials based on the captured characteristics of each of the heterogeneous mixture of materials, wherein the classification is based on a knowledge base produced from a previously generated classification of one or more examples of the first type of materials.
  • the knowledge base may contain a library of observed characteristics captured by a camera configured to capture images of the one or more examples of the first type of materials as they were conveyed past the camera.
  • the sensor may be a camera configured to capture visual images of the heterogeneous mixture of materials to produce image data, and wherein the captured characteristic is a visually observed characteristic.
  • the method executed by the computer program product may further include sending to an automated sorting device information regarding the classifications so that the automated sorting device can sort the certain ones of the heterogeneous mixture of materials from the heterogeneous mixture as a function of the classification.
  • the machine learning system may be an artificial intelligence neural network.
  • the first type of materials may be organic waste materials.
  • the first type of materials may be high end composite materials.
  • the first type of materials may be electronic equipment or e- waste.
  • the first type of materials may be agriculture materials.
  • the first type of materials may be one or more specified metal alloys.
  • substantially refers to a degree of deviation that is sufficiently small so as to not measurably detract from the identified property or circumstance.
  • the exact degree of deviation allowable may in some cases depend on the specific context.
  • the term “about,” when referring to a value or to an amount of mass, weight, time, volume, concentration or percentage is meant to encompass variations of in some embodiments ⁇ 20%, in some embodiments ⁇ 10%, in some embodiments ⁇ 5%, in some embodiments ⁇ 1%, in some embodiments ⁇ 0.5%, and in some embodiments ⁇ 0.1% from the specified amount, as such variations are appropriate to perform the disclosed method.
  • Coupled is not intended to be limited to a direct coupling or a mechanical coupling. Unless stated otherwise, terms such as “first” and “second” are used to arbitrarily distinguish between the elements such terms describe. Thus, these terms are not necessarily intended to indicate temporal or other prioritization of such elements.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Sorting Of Articles (AREA)

Abstract

Systèmes et procédés de classification de matériaux à l'aide d'un ou de plusieurs systèmes de capteurs, qui peuvent mettre en œuvre un système d'apprentissage automatique afin d'identifier ou de classer chacun des matériaux, qui peuvent ensuite être triés en groupes séparés sur la base d'une telle identification ou classification. Le système d'apprentissage automatique peut utiliser un réseau de neurones artificiels, et être préalablement formé pour reconnaître et classifier certains types de matériaux. Des modes de réalisation de la présente divulgation peuvent être utilisés pour trier n'importe quels types ou classes de matériaux tels que définis dans la description. Par contraste, un ensemble homogène ou un groupe de matériaux appartiennent tous à une classe ou un type de matériau identifiable.
PCT/US2022/035013 2021-07-20 2022-06-24 Système de manipulation de matériau WO2023003670A1 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US17/380,928 US20210346916A1 (en) 2015-07-16 2021-07-20 Material handling using machine learning system
US17/380,928 2021-07-20
US17/491,415 US11278937B2 (en) 2015-07-16 2021-09-30 Multiple stage sorting
US17/491,415 2021-09-30
US17/495,291 2021-10-06
US17/495,291 US11975365B2 (en) 2015-07-16 2021-10-06 Computer program product for classifying materials

Publications (1)

Publication Number Publication Date
WO2023003670A1 true WO2023003670A1 (fr) 2023-01-26

Family

ID=84979524

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2022/035013 WO2023003670A1 (fr) 2021-07-20 2022-06-24 Système de manipulation de matériau

Country Status (1)

Country Link
WO (1) WO2023003670A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180243800A1 (en) * 2016-07-18 2018-08-30 UHV Technologies, Inc. Material sorting using a vision system
US20200050922A1 (en) * 2018-08-13 2020-02-13 National Chiao Tung University Recycling system and method based on deep-learning and computer vision technology
US20200084966A1 (en) * 2018-09-18 2020-03-19 Deere & Company Grain quality control system and method
US20200368786A1 (en) * 2015-07-16 2020-11-26 UHV Technologies, Inc. Metal sorter
US20210094075A1 (en) * 2017-07-28 2021-04-01 AMP Robotics Corporation Systems and methods for sorting recyclable items and other materials

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200368786A1 (en) * 2015-07-16 2020-11-26 UHV Technologies, Inc. Metal sorter
US20180243800A1 (en) * 2016-07-18 2018-08-30 UHV Technologies, Inc. Material sorting using a vision system
US20210094075A1 (en) * 2017-07-28 2021-04-01 AMP Robotics Corporation Systems and methods for sorting recyclable items and other materials
US20200050922A1 (en) * 2018-08-13 2020-02-13 National Chiao Tung University Recycling system and method based on deep-learning and computer vision technology
US20200084966A1 (en) * 2018-09-18 2020-03-19 Deere & Company Grain quality control system and method

Similar Documents

Publication Publication Date Title
US20210346916A1 (en) Material handling using machine learning system
US11975365B2 (en) Computer program product for classifying materials
US11964304B2 (en) Sorting between metal alloys
US20220355342A1 (en) Sorting of contaminants
US20230053268A1 (en) Classification and sorting with single-board computers
WO2023003669A1 (fr) Système de classification de matériaux
US12017255B2 (en) Sorting based on chemical composition
KR20240090253A (ko) 다단계 선별
EP4422807A1 (fr) Séparation de métaux dans un parc à ferraille
TWI829131B (zh) 用於淘選材料之方法和系統以及儲存在電腦可讀儲存媒體上的電腦程式產品
WO2023003670A1 (fr) Système de manipulation de matériau
US20220371057A1 (en) Removing airbag modules from automotive scrap
US20240132297A1 (en) Thin strip classification
US20230173543A1 (en) Mobile sorter
EP4267318A1 (fr) Tri basé sur une composition chimique
EP4267319A1 (fr) Extraction de modules d'airbag de déchets d'automobiles
CN116917055A (zh) 基于化学组合物的分拣
WO2023137423A1 (fr) Analyse de données de déchets
WO2022251373A1 (fr) Tri de contaminants
KR20240137600A (ko) 스크랩 데이터 분석

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22846391

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22846391

Country of ref document: EP

Kind code of ref document: A1