CN117529372A

CN117529372A - Sorting dark and black plastics

Info

Publication number: CN117529372A
Application number: CN202280023776.8A
Authority: CN
Inventors: N·库马; 小曼纽尔·G·加西亚; I·K·玛乌恩; J·A·雷西; L·V·蒙托托
Original assignee: Sotera Technology Co ltd; Battelle Energy Alliance LLC
Current assignee: Sotera Technology Co ltd; Battelle Energy Alliance LLC
Priority date: 2021-02-08
Filing date: 2022-02-08
Publication date: 2024-02-06
Also published as: CN117241897A

Abstract

Systems and methods for classifying and sorting dark and/or black plastic materials using a vision system or one or more sensor systems implemented using one or more mid-wave infrared cameras whereby captured image data is processed within a machine learning system to identify or sort each of the materials, which may then be sorted into individual groups based on such identification or sorting.

Description

Sorting dark and black plastics

Priority is claimed in U.S. provisional patent application Ser. No. 63/146,892 and U.S. provisional patent application Ser. No. 63/173,301. This application is a partial continuation of U.S. patent application Ser. No. 17/495,291, a partial continuation of U.S. patent application Ser. No. 17/380,928, a partial continuation of U.S. patent application Ser. No. 17/227,245, a partial continuation of U.S. patent application Ser. No. 16/939,011, a U.S. patent application Ser. No. 16/375,675 (as an issue of U.S. patent No. 10,722,922), a partial continuation of U.S. patent application Ser. No. 16/939,011, a U.S. patent application Ser. No. 15/963,755 (as an issue of U.S. 10,710,119), a priority of U.S. patent application Ser. No. 15/963,755 to be claimed to U.S. No. 62/490,219, and a partial U.S. patent application Ser. No. 15/213,129 (as an issue of U.S. patent No. 10,207,29), and a full priority of U.S. 15/213,332, all of which are hereby incorporated by reference. The present application is also a continuation-in-part of U.S. patent application Ser. No. 17/491,415, a continuation-in-part of U.S. patent application Ser. No. 16/852,514, a division of U.S. patent application Ser. No. 16/358,374 (advertised as U.S. patent No. 10,625,304), and a continuation-in-part of U.S. patent application Ser. No. 16/358,374 (advertised as U.S. patent No. 10,710,119).

Government licensing rights

The present disclosure is made with U.S. government support under DE-AR0000422 awarded by the U.S. department of energy. The united states government may have certain rights in this disclosure.

Technical Field

The present disclosure relates generally to sorting of plastics, and more particularly, to sorting dark and black plastics from municipal or industrial solid waste.

Background

This section is intended to introduce various aspects of the art, which may be associated with exemplary embodiments of the present disclosure. This discussion is believed to be helpful in providing a framework to facilitate a better understanding of the particular aspects of the present disclosure. Accordingly, it should be understood that this section should be read from this perspective and not necessarily as an admission of prior art.

Reclamation is the process of collecting and disposing of materials (e.g., from a waste stream) that would otherwise be discarded as waste and converting them into new products (or at least enabling more appropriate disposal). Since recycling reduces the amount of waste to be sent to a landfill, protects natural resources such as wood, water and minerals, improves economic safety by using domestic material sources, prevents pollution by reducing the need for collecting new raw materials, and saves energy, recycling has benefits to communities as well as to the environment. After collection, the recyclables may be sent to a material recovery facility ("MRF") for sorting, cleaning, and processing into materials that may be used for manufacturing. Thus, a high throughput automated sorting platform that economically sorts highly mixed waste streams would be beneficial to various industries. Thus, there is a need for a cost-effective sorting platform that can identify, analyze, and separate mixed industrial or municipal fixed waste streams at high throughput to economically generate higher quality feedstocks (which may also include lower levels of trace contaminants) for subsequent processing. Often, MRFs cannot distinguish between multiple materials, which limits the sorted materials to markets of lower quality and value, or to speeds that are too slow, labor intensive, and inefficient, which limits the amount of material that can be economically recovered or recovered.

Municipal solid waste ("MSW") is a broad term representing waste streams covering domestic, commercial and industrial sources. Within each of these categories, there are thousands of different materials and products. The U.S. national Environmental Protection Agency (EPA) reports that 2.678 million tons of MSW were produced in 2017. 3537 ten thousand tons, or 13.2% of the total weight of the MSW consists of plastic. Of these 3537 ten thousand tons of plastic, 296 ten thousand tons (8.4%) of plastic was recovered, 559 ten thousand tons (15.8%) of plastic was energy recovered by combustion, and 2682 ten thousand tons (75.8%) of plastic was buried. Obviously, more recycling of the plastic is required.

Plastic recycling is the reprocessing of plastic waste into new useful products. Recycling is necessary because almost all plastics are non-biodegradable and therefore can accumulate in the environment. At present, almost all recycling is carried out by remelting and reforming waste plastics into new articles; so-called mechanical recovery. This can lead to degradation of the polymer at a chemical level and also requires sorting of the plastic waste before reprocessing, both by colour and by polymer type, which is complex and expensive. Failure in this regard may result in inconsistent material properties, which is not attractive to the industry. In an alternative process, known as raw material recovery, the plastic waste is converted back to its starting chemicals, which can then be reprocessed back into fresh plastic. This provides the hope of more recovery, but also brings higher energy and capital costs. As part of the energy recovery, plastic waste may also be burned instead of fossil fuels.

Currently, only some plastics are recyclable. When the plastics are recovered, they are typically sorted into different types of plastics. The recovery rate varies from plastic to plastic. Several types are commonly used, each with different chemical and physical properties. This results in a difference in the ease of sorting and reprocessing thereof, thereby affecting the value and market size of the recycled material. Plastic packages and products made from a single material (e.g., polyethylene terephthalate ("PET"), high density polyethylene ("HDPE"), and polypropylene ("PP")) can be more easily recycled. Sometimes or almost never recyclable plastics include polyvinyl chloride ("PVC"), low density polyethylene ("LDPE"), linear low density polyethylene ("LLDPE"), and polystyrene ("PS"). Additionally, the plastic can be recycled only a limited number of times.

In modern single-stream MRF and plastic reclaimers, a large number of feeds require processing equipment capable of moving and sorting materials at high speeds. At the same time, the highest value is obtained from the purest, least contaminated stream. To achieve these somewhat contradictory goals, single stream MRFs and reclaimers today employ automated equipment that sorts plastic packages by near infrared ("NIR") feature (transmission or reflection). These sensors rely on reflection of light from an external source and can only observe the surface of the material. Furthermore, only polymer information is captured from the sensor. For example, NIR spectra can be identified for type #1 plastics, which are clear and bluish PET, and #2HDPE, while rejecting #1 color PET, #3PVC, #4LDPE, #5PP, #6PS, and #7 other plastics (such as multi-layer polymers, composite polymers, acrylic, and nylon). Furthermore, NIR spectra cannot accurately identify black or dark (dark colored) plastics, as well as composite materials such as plastic coated papers and multi-layer packaging (made of polymeric multi-layer films), which can give misleading readings. Most black plastics are colored with carbon. Black plastics are widely used in the automotive industry, electronic devices, food packaging, plastic bags, and the like. But in addition to absorbing visible light, black plastics also absorb the near infrared part of the spectrum, which has an unfortunate side effect, namely, making it invisible to the NIR spectrum. Thus, the "invisible" black plastic enters "miscellaneous" bins at the end of the conveyor, where they are burned to harvest energy or dump to a landfill, if not detected.

In a closed loop or primary recovery, the waste plastic is recovered as a new item of similar quality or kind (e.g., changing the beverage bottle back to a beverage bottle). However, continuous mechanical recycling of plastics without degrading the quality is very challenging due to the risk of cumulative degradation of the polymer and accumulation of contaminants. Although many closed loop recovery of polymers has been studied, the only commercial success to date has been in PET bottle recovery.

In open loop or secondary recovery (also known as degraded recovery), the quality of the plastic is reduced with each recovery, so the material cannot be recovered indefinitely and will eventually become waste. Recycling PET bottles into wool-like fabrics or other fibers is a common example and accounts for the majority of PET recycling. The degradation of the polymer quality can be counteracted by mixing the recycled plastic with the original material or compatible plastic when manufacturing new products.

Although thermosetting polymers do not melt, techniques for their mechanical recovery have been developed. This typically involves breaking down the material into pieces, which can then be mixed with some binder to form a new composite.

In the raw materials or three recycles (also referred to as chemical recycles), the polymers are reduced to their chemical components (monomers) which can then be polymerized back to fresh plastic. Thermal depolymerization and chemical depolymerization are two types of feedstock recovery.

Energy recovery (also known as energy recovery or four times recovery) involves burning plastic waste instead of fossil fuel for energy production.

A process has been developed in which certain types of plastics can be used as a carbon source (instead of coke) in the recovery of scrap steel. In some applications, the ground plastic may be used as a construction aggregate or filler material.

The plastic waste may simply burn as refuse derived fuel ("RDF") during the conversion of the waste into energy, or may first be chemically converted into synthetic fuel. In either method, PVC must be eliminated or compensated for by the installation of a dinitration technique, as PVC generates large amounts of Hydrogen Chloride (HCL) upon combustion, which can corrode equipment and lead to undesirable chlorination of fuel products.

The mixed plastic waste may be depolymerized to produce synthetic fuel. Synthetic fuels have a higher heating value than the starting plastic and can burn more efficiently, although their efficiency is still lower than fossil fuels. Various conversion techniques have been studied, with pyrolysis being the most common. The use of a catalyst in pyrolysis may result in a more definite product with higher value. Plastic converted fuel technology has been economically unfeasible due to the relatively low cost of collecting and sorting plastics and the value of the fuel produced, as compared to the widespread use of incineration.

Drawings

Fig. 1 illustrates a schematic diagram of a sorting system configured in accordance with certain embodiments of the present disclosure.

FIG. 2 illustrates an exemplary representation of a control set of pieces of material used during a training phase of a machine learning system.

Fig. 3 illustrates a flow chart configured in accordance with certain embodiments of the present disclosure.

Fig. 4 illustrates a simplified schematic diagram of a configuration in accordance with certain embodiments of the present disclosure.

Fig. 5 and 6 show examples of chemical features.

Fig. 7 illustrates a flow chart configured in accordance with certain embodiments of the present disclosure.

Fig. 8 illustrates a flow chart configured in accordance with certain embodiments of the present disclosure.

FIG. 9 illustrates a block diagram of a data processing system configured in accordance with certain embodiments of the present disclosure.

Detailed Description

Various detailed embodiments of the present disclosure are disclosed herein. However, it is to be understood that the disclosed embodiments are merely exemplary of the disclosure that may be embodied in various and alternative forms. The figures are not necessarily to scale; some features may be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ the present disclosure.

As used herein, "material" may include any item or object including, but not limited to: metals (black and colored), metal alloys, plastics (including but not limited to any of the plastics disclosed herein, known in the art, or newly created in the future), rubber, foam, glass (including but not limited to borosilicate or soda lime glass, and various colored glasses), ceramics, paper, cardboard, polytetrafluoroethylene, polyethylene, bundled wires, insulated coated wires, rare earth elements, leaves, wood, plants, plant parts, textiles, biowaste, packaging, electronic waste, batteries and accumulators, scrap vehicles, mining, construction, and demolition waste, crop waste, forest residues, specialty grass, wood energy crops, microalgae, municipal food waste, hazardous chemicals and biomedical waste, construction waste, farm waste, biological items, non-biological items, objects having a specific carbon content, any other objects, items or materials that may be found within municipal solid waste, and any other objects, items or materials disclosed herein, including any of the foregoing types of technologies that may be distinguished from one another by one or more sensor systems (including but not limited to any of the sensor technologies disclosed herein).

"Material" may include any article or object composed of a chemical element, a compound or mixture of chemical elements, or a compound or mixture of chemical elements, where the complexity of a compound or mixture may vary from simple to complex. As used herein, "chemical element" means a chemical element in the periodic table of elements, including elements that may be found later in the filing of the present application. Within this disclosure, the terms "scrap," "scrap piece," "material," and "material piece" may be used interchangeably.

As is well known in the art, a "polymer" is a substance or material composed of very large molecules or macromolecules, which is composed of many repeating subunits. The polymer may be a natural polymer or a synthetic polymer found in nature.

"multilayer polymeric film" is composed of two or more different compositions and may have up to about 7.5 ^-8 ×10 ^-4 m thickness. The layers are at least partially continuous and preferably but optionally coextensive.

As used herein, the terms "plastic", "plastic part" and "plastic material part" (all of which may be used interchangeably) refer to a polymeric composition comprising or consisting of one or more polymers and/or multi-layer polymeric films.

As used herein, a "dark plastic" is any type of plastic material with dark pigmentation that is indistinguishable from other light plastics by NIR spectroscopy.

As used herein, a "black plastic" is any type of plastic material having a substantially black pigmentation.

As used herein, the term "chemical feature" refers to a unique pattern (e.g., fingerprint spectrum) that would be produced by one or more analytical instruments, which indicates the presence of one or more specific elements or molecules (including polymers) in a sample. The element or molecule may be organic and/or inorganic. Such analytical instrumentation includes any of the sensor systems disclosed herein. According to embodiments of the present disclosure, one or more sensor systems disclosed herein may be configured for producing chemical characteristics of a piece of material (e.g., a piece of plastic).

As used herein, "fraction" refers to any particular combination of: organic and/or inorganic elements or molecules, polymer types, plastic types, polymer compositions, chemical characteristics of the plastic, physical characteristics of the plastic part (e.g., color, transparency, strength, melting point, density, shape, size, type of manufacture, uniformity, response to stimuli, etc.), etc., including any and all of the various classifications and types of plastics disclosed herein. A non-limiting example of a score is one or more different types of plastic pieces comprising: LDPE plus a relatively high percentage of aluminum; LDPE and PP plus a relatively low percentage of iron; adding zinc into PP; a combination of PE, PET and HDPE; any type of red LDPE plastic; any combination of plastic parts other than PVC; dark plastic pieces and/or black plastic pieces; a plastic part at least partially composed of a dark and/or black plastic material; a combination of type #3- #7 plastics comprising a specific combination of organic and inorganic molecules; a combination of one or more different types of multilayer polymeric films; combinations of specific plastics that do not contain specific contaminants or additives; any type of plastic having a melting point greater than a particular threshold; any of a number of specific types of thermoset; specific plastics that do not contain chlorine; combinations of plastics with similar densities; combinations of plastics with similar polarity; plastic bottles without attached caps or attached caps without plastic bottles.

"catalytic pyrolysis" involves the degradation of a polymeric material by heating the polymeric material in the absence of oxygen and a catalyst.

The term "predetermined" refers to a predetermined or decided thing.

"spectral imaging" refers to imaging using multiple bands across the electromagnetic spectrum. Although a common camera captures light across three bands in the visible spectrum (red, green, and blue (RGB)), spectral imaging encompasses a variety of technologies including but exceeding RGB. For example, spectral imaging may use infrared (e.g., MWIR), visible, ultraviolet, and/or x-ray spectra, or some combination of the above. The spectral data or spectral image data is a digital data representation of the spectral image. Spectral imaging may include simultaneous acquisition of spectral data in the visible and invisible bands, illumination from outside the visible range, or use of optical filters for capturing a particular spectral range. It is also possible to capture hundreds of bands for each pixel in the spectral image.

As used herein, the term "image data packet" refers to a digital data packet associated with a captured spectral image of each piece of material.

As used herein, the terms "identify …" and "classify …," and the terms "identify" and "classify" may be used interchangeably with any of the derivatives of the foregoing. As used herein, "classifying" a piece of material is to determine (i.e., identify) the type or class of material to which the piece of material belongs. For example, according to certain embodiments of the present invention, a sensor system (as further described herein) may be configured to collect and analyze any type of information for classifying materials, which classifications may be utilized within a sorting system to selectively sort pieces of material according to one or more sets of physical and/or chemical characteristics (which may be user-defined, for example), including, but not limited to: color; texture; color tone; shape; brightness; a weight; a density; a composition; size of the material; uniformity; the type of manufacture; chemical characteristics; a predetermined score; a radioactive feature; transmittance of light, sound, or other signals; and responses to stimuli such as various fields including emitted and/or reflected electromagnetic radiation ("EM") of the piece of material. As used herein, "manufacturing type" refers to the type of manufacturing process by which a piece of material is manufactured, such as a cast (including but not limited to consumable mold casting, permanent mold casting, and powder metallurgy), forged metal part formed by a forging process; material removal processes, and the like.

The type or class (i.e., classification) of material may be user-definable and is not limited to any known classification of material. The granularity of a type or class may vary from very coarse to very fine. For example, the type or category may include: plastics, ceramics, glass, metals, and other materials, wherein the particle size of such types or classes is relatively coarse; different metals and metal alloys such as, for example, zinc, copper, brass, chrome plating and aluminum, wherein the particle size of such types or classes is finer; or between certain types or colors of plastics, where the granularity of such types or categories is relatively fine. Thus, the type or class may be configured to distinguish between materials of significantly different compositions such as, for example, different types of plastics (e.g., distinguishing between any of the #1 to #7 types of plastics), or to distinguish between materials of nearly identical compositions such as, for example, different sub-types of plastics that fall within a particular plastic type (e.g., plastics that can be identified/distinguished by MWIR spectra and plastics that cannot be identified/distinguished by MWIR spectra). It should be appreciated that the methods and systems discussed herein may be applied to accurately identify/classify pieces of material for which the composition is not known at all prior to being classified.

Certain embodiments of the present disclosure improve plastic sortation capability by fusing a variety of sensor technologies and machine learning systems. Limitations of sensor-based sorter technology stem from the use of a single sensor, as each sensor can only detect a narrow range of signals. The most common types of sorter sensors are eddy current, visible camera, x-ray transmission, near infrared and x-ray fluorescence ("XRF"), and the following table summarizes these sensors.

Sensor type	Associated physics	Detected signal
			Vortex flow	Magnetic field	Metal material
Visible camera	Visible light reflection	Materials of different colours
			X-ray transmission	Intensity of X-ray	Transmission and density
Near infrared	IR spectrum	Polymer type
			XRF	XRF spectrum	Composition of inorganic elements

However, the plastic in MSW may be composed of one or more organic polymers, one or more inorganic elements, and have many different colors, shapes, and sizes. Examples of such plastics include potato chip bags, squeezable juice boxes, beverage choice containers, and electromagnetic sensitive packaging for electronics. Certain embodiments of the present disclosure utilize sensor-based techniques that enable classification of these different types of plastics into unique classifications that can take into account their organic polymer composition and/or their inorganic elemental composition to generate new fractions from the waste stream. For example, a conversion chemist highly focusing on the relative composition of the polymer and inorganic elements will be able to select one or more novel fractions so that a particular product can be produced from recycled plastic sorted into such fractions. Thus, sorting systems configured in accordance with embodiments of the present disclosure may produce scores that exceed those that are possible with existing most advanced sorting techniques.

As has been described herein, embodiments of the present disclosure may utilize a combination of vision system 110 and sensor system 120 utilizing one or more MWIR cameras to sort/sort one or more different types of dark and/or black plastics from each other and/or from other plastics or other materials. For example, some shapes (i.e., as a predetermined fraction) of dark and/or black plastic may be classified and sorted using such combinations.

As also already described herein, embodiments of the present disclosure may utilize a sensor system 120, the sensor system 120 utilizing one or more MWIR cameras by: the output of the MWIR camera(s) (i.e., spectral image data) is used to train a machine learning system to classify one or more different types of dark and/or black plastics, which is then used to classify and/or sort such dark and/or black plastics from a material stream such as MSW.

For example, certain embodiments of the present disclosure may be configured to sort and/or sort predetermined scores from #3- #7 type plastic packages to create new products (e.g., by recycling methods) and/or fuels. Exemplary end uses for such fractions may include, but are not limited to, gases (e.g., C1-C4), fuels (e.g., gasoline, diesel), and vacuum gas oils. However, sorting of plastics of the type #3- #7 based on organic and inorganic elemental composition of the plastics has never been successfully completed.

Embodiments of the present disclosure may be configured for sorting pieces of plastic material according to various different predetermined scores or combinations of characteristics or types, as disclosed below and elsewhere in the present disclosure.

Plastics can be classified into three types according to their chemical characteristics, their polarities and their applications, depending on the characteristics of the plastics.

Plastics can be classified into thermoplastics, thermosets and elastomers according to their chemical structure and temperature behavior.

Regarding polarity, the presence of atoms of different nature can cause electrons to move to the most electronegative atoms of the covalent bonds, creating dipoles. Polymers containing these very electronegative atoms (such as Cl, O, N, F etc.) will become polar compounds, which have an effect on the properties of the material. If the polarity is increased, the mechanical resistance, hardness and rigidity are improvedDegree, heat resistance, water absorption and hygroscopicity, and chemical resistance, as well as permeability to polar compounds such as water vapor and adhesion to metals and adhesiveness are also increased. At the same time, the increase in polarity reduces thermal expansion, electrical insulation capability, tendency to accumulate static charge, and the potential for polar molecules (O ₂ 、N ₂ ) Is a permeability of (a). In this way, it is possible to distinguish between different families, such as polyolefins, polyesters, acetals, halogenated polymers and others.

The third classification (depending on their application) applies to thermoplastic materials. Within this third category are four types of plastics.

Standard plastics or commodity: plastics are manufactured and used in large quantities due to their price and good properties in many respects. Some examples are polyethylene ("PE"), polypropylene ("PP"), polystyrene ("PS"), polyvinylchloride ("PVC") or acrylonitrile butadiene styrene ("ABS").

Engineering plastics: used when good structural, transparency, self-lubricating and thermal properties are required. Some examples are polyamide ("PA"), polyacetal ("POM"), polycarbonate ("PC"), polyethylene terephthalate ("PET"), polyphenylene ether ("PPE"), and polybutylene terephthalate ("PBT").

Special plastics: they have extraordinary specific properties such as polymethyl methacrylate ("PMMA") with high transparency and light stability, or Polytetrafluoroethylene (PTFE) with good temperature and chemical resistance.

High-performance plastic: mainly thermoplastics with high heat resistance. In other words, they have good mechanical high temperature resistance, in particular high temperatures up to 150 ℃. Polyimide ("PI"), polysulfone ("PSU"), polyethersulfone ("PES"), polyarylsulfone ("PAS"), polyphenylene sulfide ("PPS"), and liquid crystal polymers ("LCP") are high performance plastics.

Many plastic articles carry symbols identifying the type of polymer from which they are made. These resin identification codes are commonly abbreviated as RIC and are used internationally. There are seven total codes, six for the most common commodity plastic types and another for all other types. These types are also referred to herein as polymer types #1- #7. Polymer type #1 refers to polyethylene terephthalate ("PET"), #2 refers to high density polyethylene ("HDPE"), #3 refers to polyvinyl chloride ("PVC"), #4 refers to low density polyethylene ("LDPE"), #5 refers to polypropylene ("PP"), #6 refers to polystyrene ("PS"), and #7 refers to other polymers (e.g., acrylic, polycarbonate ("PC"), polylactic acid fibers, polylactide, nylon, fiberglass, ABS) that are not part of polymer types #1- # 6. The european union maintains a similar nine code list, which also includes ABS and polyamide.

PET plastics are used to make many common household items such as beverage bottles, cans, ropes, clothing and carpet fibers. HDPE plastics are commonly used to make containers for milk, motor oil, shampoos and conditioners, soap bottles, detergents and bleach. PVC is used in various pipes and tiles, most commonly in water pipes. LDPE products include plastic wrap, sandwich bags, squeezable bottles, and plastic grocery bags. PP is used in the manufacture of lunch boxes, margarine containers, yoghurt pots, syrup bottles, prescription bottles and plastic bottle caps. Polystyrene articles include disposable coffee cups, plastic food boxes, plastic tableware and packaging foam. Polycarbonates are used in baby bottles, compact discs and medical storage containers. Thus, according to certain embodiments of the present disclosure, a vision system implemented with a machine learning system may be trained to distinguish and sort between these different types of plastics based on the type of product they are made of.

Plastic parts may be classified according to the type of additives they may contain. Additives are compounds that are mixed into plastics to enhance performance, including stabilizers, fillers, and dyes. Transparent plastics have the highest value because they may not have been dyed, whereas black or dark plastics have a much lower value because their inclusion may lead to discoloration of the product. Thus, plastics may need to be sorted according to both polymer type and color to obtain materials suitable for recycling.

Plastics may also be sorted and sorted based on density. Some polymers have similar density ranges (e.g., PP and PE, or PET, PS and PVC). If the plastic part contains a high proportion of filler, this may affect its density.

Plastic waste can also be broadly divided into two categories: industrial waste (sometimes referred to as post-industrial resins) and post-consumer waste.

The plastic parts can also be sorted/sorted according to their way of recycling. During mechanical recycling, depending on the polymer type, the plastic may be reprocessed at any temperature between 150 ℃ and 320 ℃, which may cause unwanted chemical reactions leading to polymer degradation. This may reduce the physical properties and overall quality of the plastic and may produce volatile low molecular weight compounds which may produce undesirable tastes or odors and cause thermochromic colour changes. Accordingly, embodiments of the present disclosure may be configured for sorting and sorting plastic parts such that such unwanted chemical reactions are avoided. The presence of additives within the plastic may accelerate this degradation. For example, oxygen-containing biodegradation additives that aim to improve the biodegradability of plastics can increase the extent of thermal degradation. Similarly, flame retardants may have undesirable effects. Accordingly, embodiments of the present disclosure may be configured for sorting and sorting plastic parts such that plastic parts with certain such additives are discarded.

The quality of the product may also depend to a large extent on how well the plastic is sorted. Many polymers are immiscible with each other when melted and phase separation (e.g., oil and water) occurs during reprocessing. Products made from such blends contain many boundaries between different polymer types and the cohesion across these boundaries is weak, resulting in poor mechanical properties. Thus, embodiments of the present disclosure may be configured for sorting and sorting plastic parts such that certain immiscible plastic parts are not sorted together into the same group.

Systems and methods described herein according to certain embodiments of the present disclosure receive a heterogeneous mixture of a plurality of pieces of material (e.g., any combination of the various plastics disclosed herein), wherein at least one piece of material within the heterogeneous mixture comprises a composition of elements that are different from one or more other pieces of material (e.g., produce dark and/or black plastic material) and/or at least one piece of material within the heterogeneous mixture is distinguishable from other pieces of material (e.g., visually distinguishable characteristics or features, different chemical features, etc.), and the systems and methods are configured for identifying/classifying/sorting the piece of material into a group separate from such other pieces of material. Embodiments of the present disclosure may be used to sort any type or class of material or score defined herein.

Embodiments of the present disclosure will be described herein as sorting pieces of material into separate groups by physically depositing (e.g., diverting or discharging) the pieces of material into separate receptacles or bins according to user-defined groupings (e.g., material type classifications or scores). As an example, within certain embodiments of the present disclosure, pieces of material may be sorted into separate bins in order to separate pieces of material having physical characteristics (e.g., visually distinguishable characteristics or features, different chemical features, etc.) that are distinguishable from the physical characteristics of other pieces of material.

Fig. 1 illustrates an example of a system 100 configured in accordance with various embodiments of the invention. The conveyor system 103 may be implemented to convey one or more streams of individual pieces of material 101 through the system 100 such that each of the individual pieces of material 101 may be tracked, sorted, and sorted into a predetermined desired group. Such conveyor systems 103 may be implemented using one or more conveyors on which the pieces of material 101 travel at a generally predetermined constant speed. However, certain embodiments of the present disclosure may be implemented with other types of conveyor systems, including systems in which pieces of material fall freely past the various components of the system 100 (or any other type of vertical sorter), or vibrating conveyor systems. Hereinafter, where applicable, the conveyor system 103 may also be referred to as a conveyor belt 103. In one or more embodiments, some or all of the acts of conveying, stimulating, detecting, classifying, and sorting may be performed automatically, i.e., without human intervention. For example, in system 100, one or more stimulus sources, one or more emission detectors, classification modules, sorting devices, and/or other system components may be configured to automatically perform these and other operations.

Furthermore, while fig. 1 shows a single stream of pieces of material 101 on conveyor system 103, embodiments of the present disclosure may be implemented with multiple such streams of pieces of material passing parallel to each other through various components of system 100. For example, as further described in U.S. patent No. 10,207,296, pieces of material may be distributed into two or more parallel separate streams or sets of parallel conveyors traveling on a single conveyor. Thus, certain embodiments of the present disclosure are capable of simultaneously tracking, sorting, and sorting a plurality of such parallel traveling streams of material pieces. According to certain embodiments of the present disclosure, no separator is required to be incorporated or used. Instead, a conveyor system (e.g., conveyor system 103) may simply convey a collection of pieces of material that may have been placed onto conveyor system 103 in a random manner.

According to certain embodiments of the present disclosure, some suitable feeding mechanism (e.g., another conveyor system or hopper 102) may be utilized to feed the pieces of material 101 onto the conveyor system 103, whereby the conveyor system 103 conveys the pieces of material 101 through various components within the system 100. An optional drum/vibrator/separator 106 may be used to separate individual pieces of material from the collection of pieces of material after the pieces of material 101 are received by the conveyor system 103. In certain embodiments of the present disclosure, the conveyor system 103 is operated by the conveyor system motor 104 to travel at a predetermined speed. The predetermined speed may be programmable and/or adjustable by an operator in any known manner. The monitoring of the predetermined speed of the conveyor system 103 may alternatively be performed with the position detector 105. Within certain embodiments of the present disclosure, control of the conveyor system motor 104 and/or the position detector 105 may be performed by the automated control system 108. Such an automation control system 108 may be operated under control of the computer system 107 and/or the functions for performing the automation control may be implemented in software within the computer system 107.

The conveyor system 103 may be a conventional endless belt conveyor employing a conventional drive motor 104, the conventional drive motor 104 being adapted to move the belt conveyor at a predetermined speed. The position detector 105, which may be a conventional encoder, may be operably coupled to the conveyor system 103 and the automation control system 108 to provide information (e.g., speed) corresponding to the movement of the conveyor belt. Thus, as will be further described herein, by utilizing control of the conveyor system drive motor 104 and/or the automated control system 108 (and, alternatively, the position detector 105), as each of the pieces of material 101 traveling on the conveyor system 103 is identified, they may be tracked by position and time (relative to the various components of the system 100) such that the various components of the system 100 may be activated/deactivated as each piece of material 101 passes in proximity to the various components of the system 100. As a result, the automated control system 108 is able to track the position of each of the pieces 101 as each of the pieces 101 travels along the conveyor system 103.

Referring again to fig. 1, certain embodiments of the present disclosure may utilize a visual or optical recognition system 110 and/or a piece tracking device 111 as a means for tracking each of the pieces 101 as each of the pieces 101 moves on the conveyor system 103. The vision system 110 may utilize one or more stationary or real-time motion cameras 109 to record the position (i.e., position and timing) of each of the pieces of material 101 on the moving conveyor system 103. The vision system 110 may be further or alternatively configured to perform certain types of identification (e.g., classification) of all or a portion of the piece of material 101, as will be further described herein. For example, such vision systems 110 may be used to capture or gather information about each of the pieces of material 101. For example, the vision system 110 may be configured (e.g., using a machine learning system) to capture or collect any type of information from the pieces of material that may be utilized within the system 100 to classify the pieces of material 101 and/or selectively sort according to one or more sets of characteristics (e.g., physical and/or chemical and/or radioactivity, etc.), as described herein. According to certain embodiments of the present disclosure, the vision system 110 may be configured to capture visual images (including one-dimensional, two-dimensional, three-dimensional, or holographic imaging) of each of the pieces of material 101, for example, by using optical sensors utilized in typical digital cameras and video equipment. Such visual images captured by the optical sensor are then stored as image data in a memory device (e.g., formatted as image data packets). Such image data may represent images captured within the optical wavelength of light (i.e., the wavelength of light that is observable by a typical human eye), according to certain embodiments of the present disclosure. However, alternative embodiments of the present disclosure may utilize a vision system configured to capture images of materials composed of wavelengths of light other than the human eye's visual wavelength (e.g., MWIR spectrum).

According to certain embodiments of the present disclosure, the system 100 may be implemented using one or more sensor systems 120, which one or more sensor systems 120 may be utilized alone or in combination with the vision system 110 to classify/identify the piece of material 101. The sensor system 120 may be configured with any type of sensor technology for determining chemical characteristics of and/or classifying plastic parts for sorting, the sensor system 120 including a sensor system that utilizes irradiated or reflected electromagnetic radiation (e.g., utilizing infrared ("IR"), fourier transform IR ("FTIR"), forward-looking infrared ("FLIR"), very near infrared ("VNIR"), near infrared ("NIR"), short wave infrared ("SWIR"), long wave infrared ("LWIR"), mid wave infrared ("MWIR" or "MIR"), X-ray transmission ("XRT"), gamma rays, ultraviolet ("UV"), X-ray fluorescence ("XRF"), laser induced breakdown spectroscopy ("LIBS"), raman spectroscopy, anti-stokes raman spectroscopy, gamma spectroscopy, hyperspectral spectroscopy (e.g., any range beyond visible wavelengths), acoustic spectroscopy, NMR spectroscopy, microwave spectroscopy, terahertz spectroscopy, and including one-dimensional, two-dimensional or three-dimensional imaging with any of the foregoing), or by any other type of sensor technology including, but not limited to, chemistry or radioactivity. An implementation of an XRF system (e.g., for use as sensor system 120 herein) is further described in U.S. patent No. 10,207,296. XRF may be used in embodiments of the present disclosure to identify inorganic materials within a plastic part (e.g., for inclusion in a chemical feature).

The following sensor systems may also be used in certain embodiments of the present disclosure to determine chemical characteristics of plastic parts and/or to sort plastic parts for sorting.

The various forms of infrared spectra previously disclosed can be used to obtain chemical characteristics specific to each plastic piece that provide information about the base polymer of any plastic material as well as other components present in the material (mineral fillers, copolymers, polymer blends, etc.).

Differential scanning calorimetry ("DSC") is a thermal analysis technique that can obtain the thermal transitions generated during the heating of each material-specific analytical material.

Thermogravimetric analysis ("TGA") is another thermal analysis technique that can obtain quantitative information about the composition of plastic materials, including polymer percentages, other organic components, mineral fillers, carbon black, and the like.

Capillary and rotational rheometry can determine the rheological properties of polymeric materials by measuring their creep and deformation resistance.

Optical and scanning electron microscopy ("SEM") can provide information about the structure of the analyzed material, including the number and thickness of layers in a multilayer material (e.g., a multilayer polymer film), the dispersion size of pigment or filler particles in a polymer matrix, coating defects, interface morphology between components, and the like.

Chromatography (e.g., LC-PDA, LC-MS, LC-LS, GC-MS, GC-FID, HS-GS) can quantify minor components of plastic materials such as UV stabilizers, antioxidants, plasticizers, anti-slip agents, etc., as well as residual monomers, ink or adhesive agent residual solvents, degradation substances, etc.

According to embodiments of the present disclosure, vision system 110 may be configured with one or more cameras configured to capture MWIR images (hyperspectral data in the MWIR area) of the piece of material located near (e.g., above) the conveyor system such that the MWIR images of the piece of material are captured as the piece of material passes through vision system 110. Such camera(s) operate in the MWIR Spectral range (2.7-5.3 μm), such as a specm FX50 camera (available from specm, spectra Imaging limited) that is capable of detecting PS, PE, PP, ABS and PVC black plastic, rubber and specific additives in the MWIR region, and other dark plastics.

It should be noted that although fig. 1 is illustrated with a combination of vision system 110 and one or more sensor systems 120, embodiments of the present disclosure may be implemented using any combination of sensor systems that utilize any of the sensor technologies disclosed herein or any other sensor technology currently available or developed in the future. Although fig. 1 is illustrated as including one or more sensor systems 120, implementations of such sensor system(s) are optional within certain embodiments of the present disclosure. Within certain embodiments of the present disclosure, a combination of both the vision system 110 and the one or more sensor systems 120 may be used to categorize the piece of material 101. Within certain embodiments of the present disclosure, any combination of one or more of the different sensor technologies disclosed herein may be used to classify the piece of material 101 without utilizing the vision system 110. Further, embodiments of the present disclosure may include any combination of one or more sensor systems and/or vision systems, wherein the output of such sensor/vision systems is processed within a machine learning system (as further disclosed herein) in order to classify/identify materials from a heterogeneous mixture of materials, which may then be sorted from one another.

According to alternative embodiments of the present disclosure, the vision system 110 and/or sensor system(s) may be configured to identify which pieces of material 101 are not of the type to be sorted by the system 100 (e.g., plastic pieces containing specific contaminants, additives, or undesirable physical characteristics (e.g., attached container caps formed from different types of plastic than containers)), and send signals to reject such pieces of material. In such a configuration, the identified pieces of material 101 may be transferred/ejected using one of the mechanisms for physically transferring the sorted pieces of material into separate bins as described below.

Within certain embodiments of the present disclosure, the piece tracking device 111 and accompanying control system 112 may be utilized and configured to measure the size and/or shape of each of the pieces 101 as the pieces 101 pass in proximity to the piece tracking device 111 along with the position (i.e., location and timing) of each of the pieces 101 on the moving conveyor system 103. Exemplary operation of such a material piece tracking device 111 and control system 112 is further described in U.S. patent No. 10,207,296. Alternatively, as previously disclosed, the vision system 110 may be used to track the position (i.e., position and timing) of each of the pieces of material 101 as the pieces of material 101 are transported by the conveyor system 103. Thus, certain embodiments of the present disclosure may be implemented without a material piece tracking device (e.g., material piece tracking device 111) for tracking a material piece.

Within certain embodiments of the present disclosure implementing one or more sensor systems 120, the sensor system(s) 120 may be configured to aid the vision system 110 in identifying the chemical composition, relative chemical composition, and/or type of manufacture of each of the pieces of material 101 as the piece of material 101 passes in proximity to the sensor system(s) 120. The sensor system(s) 120 may include an energy emitting source 121, which energy emitting source 121 may be powered, for example, by a power source 122 to excite a response from each of the pieces 101.

According to certain embodiments of the present disclosure implementing an XRF system as sensor system 120, source 121 may comprise an inline X-ray fluorescence ("IL-XRF") tube, such as further described in U.S. Pat. No. 10,207,296. Such IL-XRF tubes may include separate X-ray sources, each dedicated to one or more (e.g., separate) streams of conveyed material. In such cases, one or more detectors 124 may be implemented as XRF detectors for detecting fluorescent X-rays from material pieces 101 within each of the separate streams. Examples of such XRF detectors are further described in U.S. patent No. 10,207,296.

In certain embodiments of the present disclosure, the sensor system 120 may emit an appropriate sensing signal toward the pieces of material 101 as each piece of material 101 passes near the emission source 121. The one or more detectors 124 may be positioned and configured to sense/detect one or more characteristics from the piece of material 101 in a form suitable for the type of sensor technology utilized. One or more detectors 124 and associated detector electronics 125 capture these received sensed characteristics to perform signal processing thereon and generate digitized information (e.g., spectral data) representative of the sensed characteristics, which are then analyzed in accordance with certain embodiments of the present disclosure so as to be usable to aid the vision system 110 in classifying each of the pieces of material 101. This sorting may be performed within the computer system 107 and then may be utilized by the automated control system 108 to activate one of the N (N.gtoreq.1) sorting devices 126 … 129 of the sorting apparatus for sorting (e.g., transferring/discharging) the pieces of material 101 into one or more N (N.gtoreq.1) sorting bins 136 … 139 according to the determined sorting. Four sorting apparatuses 126 … 129 and four sorting bins 136 … 139 associated with the sorting apparatuses are illustrated in fig. 1 by way of non-limiting example only.

Existing plastic sorters are designed to sort materials in a binary fashion, with an air nozzle at the end of the conveyor discharging the identified categories of plastic into one of the two bins. For example, if four categories of plastic need to be separated, the entire stream would need to be conveyed four times through such a binary sorter, which would take four times as much time to attempt to remove a single object in the stream. According to certain embodiments of the present disclosure, the system 100 allows sorting of multiple sorted plastics in one pass.

The sorting apparatus may include any known sorting mechanism for redirecting selected pieces of material 101 toward a desired location, including but not limited to transferring pieces of material 101 from a conveyor system into a plurality of sorting bins. For example, the sorting device may utilize air injectors, wherein each of the air injectors is assigned to one or more of the classifications. When one of the air injectors (e.g., 127) receives a signal from the automation control system 108, that air injector emits an air flow that causes the pieces of material 101 to be diverted/discharged from the conveyor system 103 to a sorting bin (e.g., 137) corresponding to that air injector.

Other mechanisms may be used to transfer/eject the pieces of material, such as robotically removing the pieces from the conveyor belt, pushing the pieces of material from the conveyor belt (e.g., using a paint brush type plunger), thereby creating an opening (e.g., a trapdoor) in the conveyor system 103 from which the pieces of material may fall or using an air jet to transfer the pieces of material into a separate bin as they fall from the edge of the conveyor belt. As the term is used herein, a pusher device may refer to any form of device that may be activated to dynamically transfer objects on or from a conveyor system/device, employing a pneumatic, mechanical, or other means such as any suitable type of mechanical pushing mechanism (e.g., ACME screw drive), pneumatic pushing mechanism, or air ejector pushing mechanism. Some embodiments may include multiple pusher devices located at different locations and/or having different transfer path orientations along the path of the conveyor system. In various implementations, the sorting systems described herein may determine which pusher device (if any) to activate depending on the classification of the pieces of material performed by the machine learning system. Further, determining which pusher device to activate may be based on the detected presence and/or characteristics of other objects that may also be in the transfer path of the pusher device concurrently with the target article. Further, even for facilities in which separation along the conveyor system is imperfect, the disclosed sorting system can identify when the plurality of objects are not well separated and dynamically select a pusher device from the plurality of pusher devices that should be activated based on which pusher device provides the best transfer path for potentially separating objects within the immediate vicinity. In some embodiments, the object identified as the target object may represent material that should be transferred away from the conveyor system. In other embodiments, the object identified as the target object represents material that should be allowed to remain on the conveyor system such that non-target material is transferred instead.

In addition to the N sorting bins 136 … 139 into which the pieces 101 are transferred/discharged, the system 100 may also include a container or bin 140 that receives pieces 101 that are not transferred/discharged from the conveyor system 103 into any of the sorting containers in the foregoing sorting bins 136 … 139. For example, when the sorting of the pieces of material 101 is not determined (or simply because the sorting equipment fails to adequately transfer/discharge the pieces), the pieces of material 101 may not be transferred/discharged from the conveyor system 103 into one of the N sorting containers 136, …. Thus, the bin 140 may serve as a default container into which unclassified pieces of material are poured. Alternatively, the bins 140 may be used to receive one or more sorted pieces of material that are not intentionally assigned to any of the N sorting containers 136..139. These pieces of material may then be further sorted according to other characteristics and/or by another sorting system.

Depending on the multiple classifications of desired pieces of material, multiple classifications may be mapped to a single sorting apparatus and associated sorting bins. In other words, there need not be a one-to-one correlation between sorting and sorting bins. For example, a user may desire to sort certain categories of material into the same sorting bin (e.g., different types of plastic falling within a score). To achieve such sorting, when pieces of material 101 are sorted to fall into predetermined sorted groupings (e.g., fractions), the same sorting equipment may be activated to sort the pieces of material 101 into the same sorting bin. Such combination sorting may be applied to produce any desired combination of sorted pieces of material. The mapping of classifications may be programmed by a user (e.g., using a sorting algorithm operated by computer system 107 (see, e.g., fig. 7)) to produce such desired combinations. Additionally, the classification of the pieces of material is user-definable and is not limited to any particular known classification of pieces of material (e.g., scores as disclosed herein).

Conveyor system 103 may include a circular conveyor (not shown) such that unclassified material pieces are returned to the beginning of system 100 and again pass through system 100. Further, because the system 100 is capable of specifically tracking each piece of material 101 as the pieces of material 101 travel on the conveyor system 103, some sort of sorting apparatus (e.g., sorting apparatus 129) may be implemented to direct/discharge pieces of material 101 that fail to sort after a predetermined number of cycles through the system 100 (or pieces of material 101 are collected in bins 140).

Within certain embodiments of the present disclosure, the conveyor system 103 may be divided into multiple belts (such as, for example, two belts) configured in series, with a first belt conveying pieces of material through the vision system 110 and a second belt conveying certain sorted pieces of material through the implemented sensor system 120 for secondary sorting. Further, such a second conveyor belt may be at a lower height than the first conveyor belt, such that the pieces of material fall from the first belt onto the second belt.

Within certain embodiments of the present disclosure implementing sensor system 120, emission source 121 may be located above the detection zone (i.e., above conveyor system 103); however, certain embodiments of the present disclosure may position the emission source 121 and/or the detector 124 in other locations that still produce acceptable sensed/detected physical characteristics.

The systems and methods described herein may be applied to sort and/or sort individual pieces of material having various sizes and shapes. Although the systems and methods described herein are primarily described with respect to sorting individual pieces of material, the systems and methods described herein are not limited thereto. Such systems and methods may be used to simultaneously excite and/or detect emissions of multiple materials. For example, multiple separate streams may be conveyed in parallel as opposed to separate streams of material conveyed along one or more series-connected conveyors. Each stream may be on the same belt, or on different belts arranged in parallel. Further, the pieces may be randomly distributed on one or more conveyor belts (e.g., across and along). Accordingly, the systems and methods described herein may be used to simultaneously excite and/or detect emissions from multiple pieces of material. In other words, rather than considering each material piece individually, a plurality of material pieces may be considered as a single piece. Accordingly, multiple pieces of material may be sorted and sorted together (e.g., transferred/discharged from a conveyor system).

Although the systems and methods described herein are described primarily with respect to sorting pieces of material, such systems and methods are not limited to this use. They may be used in other applications, for example, to identify elements (e.g., contaminants) within a piece of material or to determine the composition of a piece of material.

As previously described, certain embodiments of the present disclosure may implement one or more vision systems (e.g., vision system 110) to identify, track, and/or classify pieces of material. According to embodiments of the present disclosure, such vision system(s) may operate alone to identify and/or sort and sort pieces of material, or may operate in combination with one or more sensor systems (e.g., sensor system(s) 120) to identify and/or sort and sort pieces of material. The sensor system(s) 120 may be omitted from the system 100 (or simply disabled) if the sorting system (e.g., system 100) is configured to operate with only such vision system(s) 110.

Regardless of the type(s) of sensed characteristics/information of the captured pieces of material, the information (e.g., image data packets) may then be sent to a computer system (e.g., computer system 107) for processing by a machine learning system to identify and/or classify each of the pieces of material. Such machine learning systems may implement any known machine learning system, including machine learning systems that implement: neural networks (e.g., artificial neural networks, deep neural networks, convolutional neural networks, recurrent neural networks, auto encoders, reinforcement learning, etc.), fuzzy logic, artificial intelligence ("AI"), deep learning algorithms, deep structure learning hierarchical learning algorithms, support vector machines ("SVMs") (e.g., linear SVMs, nonlinear SVMs, SVM regression, etc.), decision tree learning (e.g., classification and regression trees ("CART"), integration methods (e.g., ensemble learning, random forests, bagging (Bagging) and Pasting (packing), patches and subspaces, boosting (Boosting), stacking (Stacking), etc.), dimension reduction (e.g., projection, manifold learning, principal component analysis, etc.), and/or deep machine learning algorithms (such as deep machine learning algorithms described and publicly available in the deeplening. Net website (including hyperlinks of all software, publications, and available software referenced within the website), which are incorporated by reference herein; python, openCV, inception, theano library, torch library, pyTorch library, pylearn2 library, numpy library, blocks library, tensorFlow library, MXNet library, caffe library, lasagne library, keras library, chainer library, matlab deep learning, CNTK, matConvNet (MATLAB toolbox implementing convolutional neural networks for computer vision applications), deep Learn toolbox (MATLAB toolbox for deep learning (from Rasmus Berg Palm)), bigDL (large DL), takara Shuzo, cuda-Convnet (convolutional (or more generally, fast c++/Cuda implementation of feed forward) neural networks), deep belief networks, RNNLM, RNNLIB-RNNLIB, matrbm, deeplearning j (deep learning4 j), eblow.1sh, deep mat library, MShadow library, matplotlib library, sciPy library, CXXNET, nengo-Nengo library, eblearn, CUDAMat, gnumpy, three-way factors RBM and mcRBM, mPoT (Python code using CUDAMat and Gnumpy to train natural image models), convNet, elektronn, openNN library, neuroaldesigner (nerve designer), theano generalized hebbn learning, apache Singa, lightnet and SimpleDNN (simple DNN).

According to certain embodiments of the present disclosure, machine learning may be performed in two phases. For example, first, training occurs, which may be performed offline, as the system 100 is not utilized to perform actual sorting/sorting of the pieces of material. The system 100 may be used to train a machine learning system in that a homogeneous set (also referred to herein as a control sample) of pieces of material (i.e., of the same type or class of material (e.g., dark and/or black plastic), or falling within the same predetermined fraction) is conveyed through the system 100 (e.g., by the conveyor system 103); and all such pieces of material may not be sorted, but may be collected in a common bin (e.g., bin 140). Alternatively, training may be performed at another location remote from the system 100, including using some other mechanism for collecting sensed information (characteristics) of the control set of pieces of material. During this training phase, algorithms within the machine learning system extract features from the captured information (e.g., using image processing techniques well known in the art). Non-limiting examples of training algorithms include, but are not limited to: linear regression, gradient descent, feed forward, polynomial regression, learning curve, canonical learning model, and logistic regression. It is during this training phase that algorithms within the machine learning system learn the relationships between the materials and their features/characteristics (e.g., captured by the vision system and/or sensor system(s) (e.g., capturing features/characteristics using MWIR spectra)) to create a knowledge base for later classification of heterogeneous mixtures of material pieces received by the system 100, which can then be sorted by desired classification. Such knowledge bases may include one or more libraries, where each library includes parameters (e.g., neural network parameters) for use by the machine learning system in classifying the piece of material. For example, one particular library may include parameters configured by the training phase for identifying (classifying) a particular type or class of material (e.g., dark and/or black plastic), or one or more materials that fall within a predetermined score. According to certain embodiments of the present disclosure, such libraries may be input into a machine learning system, and then a user of the system 100 may be able to adjust certain of the parameters in order to adjust the operation of the system 100 (e.g., adjust how well the machine learning system identifies the threshold effectiveness of a particular piece of material (e.g., dark and/or black plastic) from a heterogeneous mixture of materials).

As depicted in fig. 2, during the training phase, multiple pieces of material 201 as a fraction of one or more particular types, classifications, or material(s) of control samples may be delivered (e.g., by the conveyor system 203) through the vision system and/or one or more sensor systems such that algorithms within the machine learning system detect, extract, and learn what features represent such types or classifications of material. For example, each of the pieces of material 201 may be one or more pieces of plastic, wherein at least a portion of the pieces of plastic are composed of a dark plastic material and/or a black plastic material, which are passed through such training phases such that an algorithm within the machine learning system "learns" (is trained) how to detect, identify, and classify such pieces of plastic accordingly. In the case of training a vision system (e.g., vision system 110) implemented with one or more MWIR cameras, the machine learning system is trained to distinguish between dark and/or black pieces of material and non-dark and/or black pieces of material. This will create a library of parameters specific to pieces of material consisting at least of dark plastic material and/or black plastic material. Given the captured sensed information as input data, an algorithm within the machine learning system may use N classifiers, each of which tests for one of N different dark colors and/or melanin colors. Note that the machine learning system may be "taught" (trained) to detect any type, class, or fraction of material, including any of any type, class, or fraction of material found within Municipal Solid Waste (MSW), or any other material disclosed herein.

After the algorithm has been established and the machine learning system has sufficiently learned (trained) the differences (e.g., visually discernable differences) in material classifications (e.g., within a user-defined statistical confidence level), a library for different material classifications is then implemented into a material classification/sorting system (e.g., system 100) for identifying and/or sorting material pieces (e.g., dark and/or black plastics) from heterogeneous mixtures of material pieces (e.g., as contained within the MSW), and then possibly sorting such sorted material pieces if sorting is to be performed. For example, such a system 100 may be configured with a vision system 110 implemented with one or more MWIR cameras.

As found in the relevant literature, techniques for constructing, optimizing, and utilizing machine learning systems are known to those of ordinary skill in the art. Examples of such documents include publications: "ImageNet Classification with Deep Convolutional Networks (ImageNet Classification using deep convolutional networks)", "25 th International conference treatise on neuro-information handling systems", 2012, 12 months 3-6 days, taihao lake, nevada; and LeCun et al, "Gradient-Based Learning Applied to Document Recognition (applied to Gradient-based learning of document recognition)", institute of Electrical and Electronics Engineers (IEEE), month 11 in 1998, both of which are hereby incorporated by reference in their entirety.

In one example technique, data captured by a vision or sensor system about a particular piece of material may be processed into an array of data values within a data processing system (e.g., the data processing system 3400 of fig. 9 (configured with a machine learning system)). For example, the data may be spectral data captured by a digital camera, other types of imaging sensors (e.g., MWIR cameras, such as Specim FX50 or the same type of MWIR camera), or other types of sensor systems with respect to a particular piece of material and processed into an array of data values (e.g., image data packets). Each data value may be represented by a single number, or by a series of numbers representing the value. These values may be multiplied by a neuron weight parameter (e.g., using a neural network), and may have added bias. This can be fed into the neuron nonlinearity. The resulting number of outputs from the neurons may be processed as the original value, with the outputs multiplied by subsequent neuron weight values, optionally with the addition of bias, and again fed into the neuron nonlinearity. Each such iteration of the process is referred to as a "layer" of the neural network. The final output of the final layer may be interpreted as the probability that a material (e.g., dark and/or black plastic) is present or absent in the captured data relating to the piece of material. Examples of such processes are described in detail in both the previously mentioned "ImageNet Classification with Deep Convolutional Networks (ImageNet classification using deep convolutional networks)" and "Gradient-Based Learning Applied to Document Recognition (Gradient-based learning applied to document recognition)" references.

According to certain embodiments of the present disclosure in which the neural network is implemented as a final layer ("classification layer"), a final set of outputs of neurons is trained to represent the likelihood that a piece of material (e.g., dark and/or black plastic) is associated with the captured data. During operation, if the likelihood that the piece of material is associated with the captured data exceeds a user-specified threshold, it is determined that the piece of material is indeed associated with the captured data. These techniques may be extended to determine not only the presence of a material type associated with a particular captured data, but also whether a sub-region of the particular captured data belongs to one type of material or another type of material. This process is known as segmentation and there are techniques in the literature that use neural networks (such as what is known as "fully-convoluted" neural networks), or networks that additionally include convolved portions if not fully-convoluted (i.e., partially-convoluted). This allows the location and size of the material to be determined.

It should be understood that the present disclosure is not limited exclusively to machine learning techniques. Other common techniques for material classification/identification may also be used. For example, the sensor system may provide a signal that may indicate the presence or absence of a certain type, class, or fraction of material by examining the spectral emissions (i.e., spectral imaging) of the material using optical spectrometry techniques using a multispectral or hyperspectral camera. The spectral images of the piece of material may also be used in a template matching algorithm, wherein a database of spectral images is compared to the acquired spectral images to find the presence or absence of certain types of material from the database. The histogram of the captured spectral image may also be compared to a histogram database. Similarly, the bag of words model may be used with feature extraction techniques such as scale invariant feature transform ("SIFT") to compare extracted features between captured spectral images and spectral images in a database.

Thus, as disclosed herein, certain embodiments of the present disclosure provide for the identification/classification of one or more different types, categories, or fractions of materials in order to determine which pieces of material should be transferred from a conveyor system in a defined set. According to some embodiments, machine learning techniques are utilized to train (i.e., configure) a neural network to identify various different types, categories, or scores of materials (e.g., dark and/or black plastic). A spectral image or other type of sensed information (e.g., MWIR spectral data) of the material (e.g., traveling on the conveyor system) is captured, and based on the identity/classification of such material, the systems described herein can decide which pieces of material should be allowed to remain on the conveyor system, and which pieces of material should be transferred/removed from the conveyor system (e.g., either into a collection bin, or onto another conveyor system).

According to certain embodiments of the present disclosure, a machine learning system (e.g., system 100) for an existing device may be dynamically reconfigured to identify/classify a new type, category, or fraction of material by replacing a current set of neural network parameters with a new set of neural network parameters.

It is to be mentioned herein that the collected/captured/detected/features/characteristics (e.g., spectral images) of the piece of material are not necessarily simple particularly identifiable or discernable physical characteristics, according to certain embodiments of the present disclosure; they may be abstract formulas that can only be expressed mathematically, or not at all; however, the machine learning system may be configured to parse the spectral data for patterns that allow classification of the control samples during the training phase. Further, the machine learning system may acquire sub-portions of the captured information (e.g., spectral images) of the piece of material and attempt to find correlations between the predefined classifications.

According to certain embodiments of the present disclosure, instead of a training phase in which control samples of material pieces are communicated by a vision system and/or sensor system(s), training of a machine learning system may be performed using a marking/annotation technique whereby a user enters a marking or annotation identifying each material piece as the data/information of the material piece is captured by the vision/sensor system and then used to create a library for use by the machine learning system when classifying material pieces within a heterogeneous mixture of material pieces.

Referring to fig. 3-6, certain embodiments of the present disclosure may be configured to combine or fuse multiple sensor technologies (e.g., any combination of visual ("VIS"), XRF, NIR, and MWIR) in a manner that uniquely identifies various types, classes, or fractions of plastics, so that they may be classified according to their organic and inorganic chemical compositions. However, since the plastic pieces within the MSW are of many different sizes and shapes, the signals generated from the different sensors may have a large degree of variance between them. Thus, even with such large differences, the fusion of machine learning with various sensor technologies improves the classification accuracy of these signals. Because implementing multiple different sensors in a system may increase the cost of the system and also reduce sorting speed, certain embodiments of the present disclosure may implement a system (e.g., system 100) with a fewer number of sensor systems (and thus lower capital and operating costs) to increase economic viability, but still be able to sort materials adequately.

Fig. 4 shows a simplified schematic of a system (e.g., system 100) in which pieces of material (e.g., plastic pieces) 401 are conveyed by a conveyor system 408 past a sensor system(s) that captures spectral data from each piece of material 401. In this non-limiting example, the sensor system(s) are a camera 410 (e.g., vision system 110), XRF system 411, NIR system 412, and MWIR system 413 that capture visible image data for each material piece 401. Note, however, that any of the other sensor systems disclosed herein may be utilized in any combination.

Referring to fig. 3 and 4, in process block 301, chemical characteristics of a material are determined using one or more sensor systems. For each piece of material, the sensing/detection/capture signals from the sensor system(s) are combined (e.g., in a multi-dimensional data array) to create a chemical signature. Recall that the XRF sensor system is capable of determining the presence of inorganic elements or molecules within the plastic, while a combination of one or more other sensor systems (such as NIR and MWIR) is capable of determining the presence of organic elements or molecules within the plastic. In process block 302, a visual image of each piece of material is captured. In process block 303, the captured visible image (i.e., its associated image data) of each piece of material is associated with its determined chemical characteristics (i.e., spectral image data). Figures 5 and 6 show non-limiting exemplary representations of chemical characteristics and associated image data of two different types of plastic materials (chip bags and electronics packages). It can be readily seen that different types or classes of plastic parts will have different (unique) chemical characteristics that are utilized in embodiments of the present disclosure to produce fractions and/or classifications of plastic waste (which may be user-defined). According to certain embodiments of the present disclosure, a control group of specific types or categories of plastic parts may be run through the system shown in fig. 4 in order to train the machine learning system to associate specific chemical features with specific types or categories of chemical parts.

For example, with respect to the example shown in fig. 5, images captured from multiple chip bags (which may include such bags having different physical conditions or orientations, or even bags associated with different brands of chips and/or manufacturers) may be processed to train a machine learning system.

Process block 304 may involve separating the plastic part into one or more scores. There are many ways to create these scores. One approach is to create a first layer based on the primary element and then a second and even third layer based on the secondary element. For example, the fraction may be determined first according to the polymer type and then branched into inorganic elements such as aluminum and zinc. Other exemplary fractions can then be created for the blend of polymers, but with branching into their inorganic elemental composition. Still other computing methods may perform this type of clustering to determine scores, such as principal component analysis, K-means clustering, and unsupervised and semi-supervised learning. The score is further defined herein.

In process block 305, after the scores have been determined, the plastic parts associated with the scores may be sorted (e.g., manually) to create a control group for each score. Since each score is measured using the sensor system(s), each control group contains chemical information about the piece. A vision system (e.g., vision system 110) may be used to train a machine learning model to identify those scores. Using this approach, chemical data in the plastic is converted into visual features that the machine learning system can learn to classify. And when the system 100 is used to perform classification based on visual images, it also separates the plastic by chemical composition. This approach works well when two objects look different and have different chemical compositions. Two or more sensor systems may be used to perform classification (e.g., VIS plus XRF, etc.) when two objects look the same or very similar and have different chemical compositions.

Since the determined scores may constitute any desired class of particular organic and/or inorganic elements or molecules, the process 300 may be used to train a machine learning system implemented within a sorting system such that it is configured to sort heterogeneous mixtures of different plastic pieces to produce at least one score containing one or more different types or classes of plastic pieces. For example, if the machine learning system has been trained to identify any plastic parts that contain a particular combination of organic and/or inorganic elements or molecules, when sorting is completed, the sorted scores may contain non-identical plastic parts (i.e., plastic chip bags belonging to different brands of chips, as each plastic chip bag is composed of organic and/or inorganic elements or molecules defined by a predetermined score).

Fig. 7 illustrates a flow chart depicting an exemplary embodiment of a process 3500 for sorting/sorting pieces of material using a vision system and/or one or more sensor systems, in accordance with certain embodiments of the present disclosure. Note that various aspects of process 3500 may be modified to account for the fact that the MWIR spectrum is utilized. Process 3500 may be performed to classify heterogeneous mixtures of plastic pieces into predetermined types, categories, and/or combinations of scores (e.g., classifying and/or sorting dark and/or black plastic with other plastic or waste). Process 3500 may be configured for operation within any of the embodiments of the present disclosure described herein, including system 100 of fig. 1. The operations of process 3500 may be performed by hardware and/or software included within a computer system (e.g., computer system 3400 of fig. 9) of a control system (e.g., computer system 107, vision system 110, and/or sensor system(s) 120 of fig. 1). In process block 3501, a piece of material may be placed onto a conveyor system. In process block 3502, the position of each piece of material on the conveyor system is detected for tracking each piece of material as it travels through the system 100. This may be performed by vision system 110 (e.g., by distinguishing pieces of material from underlying conveyor system material when communicating with a conveyor system position detector (e.g., position detector 105). Alternatively, the material tracking device 111 may be used to track the pieces. Alternatively, light sources (including but not limited to visible light, UV, and IR) may be created and any system with detectors that can be used to locate the pieces may be used to track the pieces. In process block 3503, when the piece of material has traveled into proximity with one or more of the vision system and/or sensor system(s), the sensed information/characteristics of the piece of material are captured/acquired (e.g., by the vision system or sensor system implemented with one or more MWIR cameras). In process block 3504, a vision system, such as previously disclosed (e.g., implemented within computer system 107), may perform preprocessing on the captured information (e.g., reflected MWIR from each of the pieces of material), which may be used to detect (extract) information (e.g., from the background (e.g., conveyor belt)) for each of the pieces of material, in other words, the preprocessing may be used to identify differences between the pieces of material and the background. Well-known image processing techniques such as dilation, thresholding and contouring may be used to identify pieces of material as being different from the background. In process block 3505, segmentation may be performed. For example, the captured information may include information related to one or more pieces of material. Additionally, a particular piece of material may be located on a seam of the conveyor belt when an image of the particular piece of material is captured. Thus, in such instances, it may be desirable to isolate the image of each piece of material from the background of the image. In an exemplary technique for process block 3505, the first step is to apply a high contrast of the image; in this way, the background pixels are reduced to substantially all black pixels, and at least some of the pixels associated with the piece of material are brightened to substantially all white pixels. The white image pixels of the piece of material are then expanded to cover the entire size of the piece of material. After this step the position of the piece of material is a high contrast image of all white pixels on a black background. A contouring algorithm may then be utilized to detect the boundary of the piece of material. The boundary information is saved and then the boundary position is transferred to the original image. Then, segmentation is performed on the original image over an area larger than the earlier defined boundary. In this way, the piece of material is identified and separated from the background.

In optional process block 3506, the pieces of material may be conveyed along a conveyor system within a vicinity of the piece tracking device and/or the sensor system to track each of the pieces of material and/or to determine a size and/or shape of the piece of material; this may be useful if the XRF system or some other spectral sensor is also implemented within the sorting system. In process block 3507, post-processing may be performed. Post-processing may involve resizing the captured information/data to prepare it for use in a machine learning system. This may also include modifying certain properties (e.g., enhancing image contrast, changing image background, or applying filters) in a manner that will result in an enhancement of the ability of the machine learning system to classify the piece of material. In process block 3509, the size of the data may be adjusted. In some cases, it may be desirable to resize the data to match the data input requirements for some machine learning systems (such as neural networks). For example, a neural network may require an image size (e.g., 225x 255 pixels or 299x 299 pixels) that is much smaller than the size of an image captured by a typical digital camera. In addition, the smaller the input data size, the less processing time is required to perform classification. Thus, smaller data sizes may ultimately increase the throughput of the system 100 and increase its value.

In process blocks 3510 and 3511, each piece of material is identified/categorized by one or more MWIR cameras based on the sensed/detected features. For example, process block 3510 may be configured with a neural network employing one or more machine learning algorithms that compare extracted features to features stored in a knowledge base that was previously generated (e.g., generated during a training phase), and assign a classification with the highest match to each of the pieces of material based on such comparison. The algorithms of the machine learning system can process the captured information/data in a hierarchical manner by using automatically trained filters. The filter responses are then successfully combined in the next stage algorithm until probabilities are obtained in the final step. In process block 3511, these probabilities may be used for each of the N classifications to determine into which of the N sorting containers the respective piece of material should be sorted. For example, each of the N classifications may be assigned to one sorting container, and the considered piece of material is sorted into the container corresponding to the highest probability of returning greater than a predetermined threshold. Within embodiments of the present disclosure, such predefined thresholds may be preset by a user. If none of the probabilities is greater than a predetermined threshold, then the particular piece of material may be sorted into an abnormal container (e.g., sorting container 140).

Next, in process block 3512, sorting equipment corresponding to one or more classifications of pieces of material is activated. Between the time that the image of the piece of material is captured and the time that the sorting apparatus is activated, the piece of material has moved (e.g., at the conveyance rate of the conveyor system) from near the vision system and/or sensor system(s) to a position downstream of the conveyor system. In embodiments of the present disclosure, activation of the sorting apparatus is timed such that as a piece of material passes through the sorting apparatus mapped to a sorting of the piece of material, the sorting apparatus is activated and the piece of material is transferred/discharged from the conveyor system into its associated sorting container. Within embodiments of the present disclosure, activation of the sorting apparatus may be timed by a respective position detector that detects when a piece of material passes before the sorting apparatus and sends a signal to enable activation of the sorting apparatus. In process block 3513, a sorting container corresponding to the activated sorting apparatus receives the transferred/ejected piece of material.

Fig. 8 illustrates a flow chart depicting an exemplary embodiment of a process 800 of sorting pieces of material according to certain embodiments of the present disclosure. Note that various aspects of process 800 may be modified to account for the fact that the MWIR spectrum is utilized. Process 800 may be configured for operation within any of the embodiments of the present disclosure described herein, including system 100 of fig. 1. Process 800 may be configured to operate in conjunction with process 3500. For example, according to certain embodiments of the present disclosure, process blocks 803 and 804 may be incorporated in process 3500 (e.g., operate in series or in parallel with process blocks 3503-3510) to combine the operation of vision system 110 implemented with a machine learning system with a sensor system (e.g., sensor system 120) not implemented with a machine learning system to sort and/or sort pieces of material.

The operations of process 800 may be performed by hardware and/or software included within a computer system (e.g., computer system 3400 of fig. 9) of a control system (e.g., computer system 107 of fig. 1). In process block 801, a piece of material may be placed onto a conveyor system. Next, in optional process block 802, the pieces of material may be conveyed along a conveyor system in the vicinity of the piece tracking device and/or the optical imaging system to track each piece of material and/or determine the size and/or shape of the piece of material. In process block 803, as the material piece travels into proximity of the sensor system, the material piece may be interrogated or stimulated with EM energy (e.g., from one or more MWIR cameras) or some other type of stimulus suitable for the particular type of sensor technology utilized by the sensor system. In process block 804, a physical property of the piece of material is sensed/detected and captured by a sensor system. In process block 805, for at least some of the pieces of material, the type of material is identified/classified based (at least in part) on the captured characteristics, which may be combined with the classification by the machine learning system in combination with the vision system 110.

Next, if sorting of the pieces of material is to be performed, sorting equipment corresponding to one or more classifications of the pieces of material is activated in process block 806. Between the time the piece of material is sensed and the time the sorting apparatus is activated, the piece of material has moved from the vicinity of the sensor system to a position downstream of the conveyor system at the conveying rate of the conveyor system. In certain embodiments of the present disclosure, activation of the sorting apparatus is timed such that as a piece of material passes through the sorting apparatus mapped to a sorting of the piece of material, the sorting apparatus is activated and the piece of material is transferred/discharged from the conveyor system into its associated sorting container. Within certain embodiments of the present disclosure, activation of the sorting apparatus may be timed by a respective position detector that detects when a piece of material passes before the sorting apparatus and sends a signal to enable activation of the sorting apparatus. In process block 807, a sorting receptacle corresponding to the activated sorting apparatus receives the transferred/ejected piece of material.

According to certain embodiments of the present disclosure, at least a portion of the plurality of systems 100 may be linked together serially in order to perform a plurality of iterations or sorting layers. For example, when two or more systems 100 are linked in such a manner, the conveyor system may be implemented with a single conveyor belt or multiple conveyor belts to convey the pieces of material through a first vision system (and, according to certain embodiments, a sensor system) configured to sort pieces of material in a heterogeneous mixture of a first material into a first one or more bin sets (e.g., sorting containers 136..139) by a sorter (e.g., first automated control system 108 and associated one or more sorting devices 126..129), and then convey the pieces of material through a second vision system (and, according to certain embodiments, another sensor system) configured to sort pieces of material in a heterogeneous mixture of a second material into a second one or more sorting bin sets by a second sorter. For further discussion of such multi-stage sorting see U.S. published patent application No. 2022/0016675, which is incorporated herein by reference.

Such continuous system 100 may comprise any number of such systems linked together in such a manner. According to certain embodiments of the present disclosure, each vision system may be configured to sort out different classified materials or different types of materials than the previous system(s).

According to various embodiments of the present disclosure, different types, classes, or fractions of materials may be classified by different types of sensors, each for use with a machine learning system, and combined to classify pieces of material in a waste or waste stream.

According to various embodiments of the present disclosure, data (e.g., spectral data) from two or more sensors may be combined using a single or multiple machine learning systems to perform classification of pieces of material.

According to various embodiments of the present disclosure, multiple sensor systems may be mounted on a single conveyor system, with each sensor system utilizing a different machine learning system. According to various embodiments of the present disclosure, multiple sensor systems may be mounted on different conveyor systems, with each sensor system utilizing a different machine learning system.

Certain embodiments of the present disclosure may be configured for producing a quantity of material having a content less than a predetermined weight or volume percent of a particular element or material after sorting.

According to various embodiments of the present disclosure, any combination of different types of sensor systems may be utilized to identify/classify possible sortation materials as disclosed herein. For example, imaging or spectroscopic sensors as disclosed herein may be used to generate data from sensed information/characteristics of a piece of material for processing by a machine learning system specific to the sensor system. Alternatively, any sensor system may be used without machine learning system processing, or with machine learning system processing, or in a combination of both.

According to various embodiments of the present disclosure, different types, categories, and/or fractions of materials may be classified by different types of sensors, each for use with a machine learning system, and combined to classify pieces of material in a waste stream.

Referring now to FIG. 9, a block diagram is depicted showing a data processing ("computer") system 3400 in which aspects of embodiments of the present disclosure may be implemented. (the terms "computer," "system," "computer system," and "data processing system" may be used interchangeably herein.) computer system 107, automation control system 108, aspects of sensor system(s) 120, and/or vision system 110 may be configured similarly to computer system 3400. The computer system 3400 may employ a local bus 3405 (e.g., a peripheral component interconnect ("PCI") local bus architecture). Any suitable bus architecture may be utilized, such as an accelerated graphics port ("AGP"), industry standard architecture ("ISA"), and the like. One or more processors 3415, volatile memory 3420, and non-volatile memory 3435 may be connected to the local bus 3405 (e.g., through a PCI bridge (not shown)). The integrated memory controller and cache memory may be coupled to one or more processors 3415. The one or more processors 3415 may include one or more central processor units and/or one or more graphics processor units 3401 and/or one or more tensor processing units. Additional connections to the local bus 3405 may be made through direct component interconnection or through a card. In the depicted example, communications (e.g., network (LAN)) adapter 3425, I/O (e.g., small computer system interface ("SCSI") host bus) adapter 3430, and expansion bus interface (not shown) may be connected to local bus 3405 by direct component connection. An audio adapter (not shown), a graphics adapter (not shown), and a display adapter 3416 (coupled to the display 3440) may be connected to the local bus 3405 (e.g., through a card plugged into an expansion slot).

The user interface adapter 3412 may provide a connection for a keyboard 3413 and a mouse 3414, a modem (not shown), and additional memory (not shown). The I/O adapter 3430 may provide connections for a hard disk drive 3431, a tape drive 3432, and a CD-ROM drive (not shown).

An operating system may run on the one or more processors 3415 and be used to coordinate and provide control of various components within the computer system 3400. In fig. 9, the operating system may be a commercially available operating system. An object oriented programming system (e.g., java, python, etc.) may run in conjunction with the operating system and provide calls to the operating system from one or more programs (e.g., java, python, etc.) executing on system 3400. Instructions for the operating system, the object-oriented operating system, and programs may be located on non-volatile memory 3435 storage devices, such as hard disk drive 3431, and may be loaded into volatile memory 3420 for execution by processor 3415.

Those of ordinary skill in the art will appreciate that the hardware in FIG. 9 may vary depending on the implementation. Other internal hardware or peripheral devices, such as flash ROM (or equivalent nonvolatile memory) or optical disk drives and the like, may be used in addition to or in place of the hardware depicted in FIG. 9. Further, any of the processes of the present disclosure may be applied to a multiprocessor computer system, or performed by a plurality of such systems 3400. For example, training of the machine learning system may be performed by a first computer system 3400, while operations of the system 100 for sorting may be performed by a second computer system 3400.

As another example, computer system 3400 may be a stand-alone system configured to be bootable without relying on some type of network communication interface, whether or not computer system 3400 comprises some type of network communication interface. As a further example, the computer system 3400 may be an embedded controller configured with ROM and/or flash ROM that provides non-volatile memory that stores operating system files or user-generated data.

The depicted example in FIG. 9 and above-described examples are not meant to imply architectural limitations. Further, the computer program forms of aspects of the disclosure may reside on any computer readable storage medium (i.e., floppy disk, compact disk, hard disk, magnetic tape, ROM, RAM, etc.) used by a computer system.

As has been described herein, embodiments of the present disclosure may be implemented to perform the various functions described for identifying, tracking, sorting, and/or sorting pieces of material. Such functionality may be implemented within hardware and/or software, such as within one or more data processing systems (e.g., data processing system 3400 of fig. 9), such as the previously mentioned computer system 107, vision system 110, aspects of sensor system(s) 120, and/or automation control system 108. However, the functionality described herein is not limited to implementation in any particular hardware/software platform.

As will be appreciated by one of skill in the art, aspects of the present disclosure may be embodied as systems, processes, methods, and/or computer program products. Thus, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit," circuitry, "" module "or" system. Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer-readable storage media, the computer program product having computer-readable program code embodied on the one or more computer-readable storage media. ( However, any combination of one or more computer readable media may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. )

The computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, biological, atomic, or semiconductor system, apparatus, controller, or device, or any suitable combination of the foregoing, where the computer readable storage medium itself is not a transitory signal. More specific examples (a non-exhaustive list) of the computer-readable storage medium could include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory ("RAM") (e.g., RAM 3420 of fig. 9), a read-only memory ("ROM") (e.g., ROM 3435 of fig. 9), an erasable programmable read-only memory ("EPROM" or flash memory), an optical fiber, a portable compact disc read-only memory ("CD-ROM"), an optical storage device, a magnetic storage device (e.g., hard drive 3431 of fig. 9), or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, controller, or device. Program code embodied on a computer readable signal medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, radio frequency, etc., or any suitable combination of the foregoing.

The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein (e.g., in baseband or as part of a carrier wave). Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, controller, or device.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, processes and program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable program instructions for implementing the specified logical function(s). It should also be noted that, in some implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

In the description herein, the techniques of the flowcharts may be described in terms of a series of sequential actions. The order of the acts and the manner in which the acts are performed may be varied freely without departing from the scope of the present teachings. Actions may be added, deleted, or altered in several ways. Similarly, actions may be reordered or looped. Further, although processes, methods, algorithms, etc. may be described in a sequential order, such processes, methods, algorithms, or any combination thereof, may be operable to be executed in alternate orders. Further, some acts within a process, method, or algorithm may be performed concurrently (e.g., acts are performed in parallel) during at least one point in time, and may also be performed in whole, in part, or any combination thereof.

Modules implemented in software for execution by various types of processors (e.g., GPU 3401, CPU 3415) may, for example, comprise physical or logical blocks of one or more computer instructions, which may, for example, be organized as objects, procedures, or functions. However, the executables of an identified module need not be physically located together, but may comprise disparate instructions stored in different locations which, when joined logically together, comprise the module and achieve the stated purpose for the module. Indeed, a module of executable code may be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices. Similarly, operational data (e.g., the materials taxonomy library described herein) may be identified and illustrated herein within modules, and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations including over different storage devices. The data may provide an electronic signal on a system or network.

These program instructions may be provided to one or more processors and/or controller(s) of a general purpose computer, special purpose computer, or other programmable data processing apparatus (e.g., controller) to produce a machine, such that the instructions, which execute via the processor(s) of the computer or other programmable data processing apparatus (e.g., GPU 3401, CPU 3415), create means or circuitry for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, which can include, for example, one or more graphics processing units (e.g., GPU 3401), or combinations of special purpose hardware and computer instructions. For example, a module may be implemented as a hardware circuit comprising custom Very Large Scale Integration (VLSI) circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, controllers, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices or the like.

Computer program code (i.e., instructions) for carrying out operations of aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, python, C ++ or the like, conventional procedural programming languages, such as the "C" programming language or similar programming languages, or any of the machine learning software disclosed herein. The program code may execute entirely on the user's computer system, partly on the user's computer system (e.g., the computer system used for sorting) and partly on a remote computer system (e.g., the computer system used for training the sensor system), or entirely on the remote computer system or server as a stand-alone software package. In the latter scenario, the remote computer system may be connected to the user's computer system through any type of network, including a local area network ("LAN") or a wide area network ("WAN"), or the connection may be made to an external computer system (for example, through the Internet using an Internet service provider).

These program instructions may also be stored in a machine-readable storage medium that can direct a computer system, other programmable data processing apparatus, controller, or other device to function in a particular manner, such that the instructions stored in the machine-readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The program instructions may also be loaded onto a computer, other programmable data processing apparatus, controller, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

One or more databases can be included in the host for storing data and providing access to data for various implementations. Those skilled in the art will also appreciate that any database, system, or component of the present disclosure may include any combination of databases or components in a single location or multiple locations for security reasons, wherein each database or system may include any of a variety of suitable security features (such as firewalls, access codes, encryption, decryption, compression, and the like). The database may be any type of database, such as a relational database, a hierarchical database, an object-oriented database, and the like. Common database products that may be used to implement a database include IBM's DB2, any of the database products available from Oracle corporation, microsoft Access from Microsoft corporation, or any other database product. The database may be organized in any suitable manner, including as a data table or a look-up table.

The association of certain data (e.g., for each of the pieces of material processed by the sorting system described herein) may be accomplished by any data association technique known and practiced in the art. For example, the association may be achieved either manually or automatically. Automatic association techniques may include, for example, database searches, database merging, GREP, AGREP, SQL, and the like. The association step may be implemented by a database merge function, for example, using key fields in each of the manufacturer and retailer data tables. The key field partitions the database according to the high-level class of objects defined by the key field. For example, a certain category may be specified as a key field in both the first data table and the second data table, and then the two data tables may be merged based on category data in the key field. In these embodiments, the data corresponding to the key fields in each of the merged data tables is preferably the same. However, for example, data tables with similar but not identical data in the key fields may also be consolidated by using AGREP.

Reference herein is made to a "configuration" device or a "device configured to" perform certain functions. It should be appreciated that this may include selecting predefined logic blocks and logically associating them so that they provide specific logic functions, including monitoring or control functions. It may also include programming computer software-based logic to retrofit control devices, wiring separate hardware components, or a combination of any or all of the foregoing.

In the description herein, numerous specific details are provided, such as examples of programming, software modules, user selections, network transactions, database queries, database structures, hardware modules, hardware circuits, hardware chips, controllers, etc., to provide a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the disclosure may be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.

Those skilled in the art will appreciate that various settings and parameters of the components of the system 100, including neural network parameters, may be customized, optimized, and reconfigured over time based on the type of materials being classified and sorted, the desired classification and sorting results, the type of equipment used, the empirical results of previous classifications, available data, and other factors.

Reference throughout this specification to "one embodiment" or "an embodiment" or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, appearances of the phrases "in one embodiment," "in an embodiment," "embodiments," "certain embodiments," "various embodiments," and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment. Furthermore, the described features, structures, aspects, and/or characteristics of the disclosure may be combined in any suitable manner in one or more embodiments. Accordingly, even though features initially claimed are functional in certain combinations, one or more features from a claimed combination may in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Benefits, other advantages, and solutions to problems have been described herein with regard to specific embodiments. The benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced, however, are not to be construed as a critical, required, or essential feature or element of any or all the claims. Further, unless explicitly described as being necessary or critical, the components described herein are not required for the practice of the present disclosure.

Although this description contains many specific details, these should not be construed as limitations on the scope of the disclosure or of what may be claimed, but rather as descriptions of features specific to particular implementations of the disclosure. Headings herein are not intended to limit the disclosure, embodiments of the disclosure, or other content disclosed under the headings.

Herein, the term "or" may be intended to be included, wherein "a or B" includes a or B and also includes both a and B. As used herein, the term "and/or" when used in the context of a list of entities refers to entities that exist alone or in combination. Thus, for example, the phrase "A, B, C and/or D" includes A, B, C and D alone, but also includes any and all combinations and subcombinations of A, B, C and D.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below may be intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed.

As used herein, terms such as "controller," "processor," "memory," "neural network," "interface," "sorter device," "sorting equipment," "pushing mechanism," "pusher equipment," "imaging sensor," "cartridge," "container," "system," and "circuitry" each refer to a non-generic equipment element that would be recognized and understood by one of skill in the art, and are not used herein as random number words or random number terms for the purpose of reference to 35u.s.c.112 (f).

As used herein, "substantially" with respect to an identified property or condition refers to a degree of deviation that is sufficiently small so as not to visually deviate from the identified property or condition. In some cases, the exact degree of allowable deviation may depend on the particular context.

As used herein, a plurality of items, structural elements, constituent elements, exemplary scores, and/or materials may be presented in a common list for convenience. However, these lists should be understood as though each member of the list is individually identified as a separate and unique member. Thus, any individual member of such list should not be construed as a de facto equivalent of any other member of the same list solely based on their presentation in a common group without indications to the contrary.

Unless otherwise defined, all technical and scientific terms used herein, such as abbreviations for polymers or chemical elements in the periodic table, have the same meaning as commonly understood by one of ordinary skill in the art to which the presently disclosed subject matter pertains. All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety unless a particular paragraph is cited. In case of conflict, the present specification, including definitions, will control. In addition, the materials, methods, and examples (e.g., scores, plastics listed) are illustrative only and are not intended to be limiting.

To the extent not described herein, many details of the processing acts and circuits are conventional with respect to the specific materials and may be found in textbooks and other sources within the computing, electronic and software arts.

Unless otherwise indicated, all numbers expressing quantities of compositions, reaction conditions, and so forth used in the specification and claims are to be understood as being modified in all instances by the term "about". Accordingly, unless indicated to the contrary, the numerical parameters set forth in this specification and attached claims are approximations that may vary depending upon the desired properties sought to be obtained by the presently disclosed subject matter.

Claims

1. A system for processing a mixture of materials, the system comprising:

a mid-wave infrared ("MWIR") camera configured to generate MWIR image data for the mixture of materials; and

a data processing system comprising a machine learning system configured with a knowledge base to classify certain materials in a mixture of the materials as including dark plastic pieces by the machine learning system processing the MWIR image data of the mixture of materials, wherein the knowledge base includes a previously generated parameter library trained to identify dark plastic pieces.

2. The system of claim 1, wherein the dark plastic piece is a black plastic piece.

3. The system of claim 2, further comprising:

a conveyor system configured for conveying the mixture of materials past the MWIR camera; and

sorting means configured for sorting some of the sorted materials from the mixture of materials according to the sorting of some of the mixture of materials.

4. The system of claim 2, wherein the previously generated parameter library is trained with MWIR images captured from a homogeneous sample set of black plastic pieces as the homogeneous sample set passes through an MWIR camera.

5. The system of claim 1, wherein the previously generated parameter library is trained with MWIR images captured from a homogeneous sample set of dark plastic pieces as the homogeneous sample set passes through a MWIR camera.

6. The system of claim 1, wherein the machine learning system comprises an artificial intelligence neural network, and wherein the parameter is a neural network parameter.

7. The system of claim 2, wherein the MWIR image data is generated as a result of MWIR spectral imaging of the mixture of materials.

8. The system of claim 1, wherein the dark plastic piece cannot be identified using NIR spectroscopy.

9. A method for processing a mixture of materials comprising a dark plastic piece, the method comprising:

generating MWIR image data for the mixture of materials;

processing the MWIR image data by a machine learning system configured with a knowledge base containing a previously generated library of MWIR spectral characteristics relating to dark plastic parts; and

based on the processing of the MWIR image of the mixture of materials, certain materials in the mixture of materials are classified as including dark plastic pieces with the machine learning system.

10. The method as recited in claim 9, further comprising:

passing the mixture of materials through an MWIR camera configured for generating the MWIR image data; and

sorting certain materials of the mixture of materials from the mixture according to the sorting.

11. The method of claim 9, wherein the previously generated library of MWIR spectral characteristics is captured by an MWIR camera configured to capture MWIR images of a homogenous sample set of a material piece comprising a dark plastic piece as the homogenous sample set of a material piece comprising a dark plastic piece is conveyed past the MWIR camera.

12. The method of claim 10, wherein the dark plastic piece is a black plastic piece.

13. The method of claim 12, wherein the previously generated library of MWIR spectral characteristics is captured by an MWIR camera configured to capture MWIR images of a homogenous sample set of a material piece comprising a black plastic piece as the homogenous sample set of a material piece comprising a black plastic piece is conveyed past the MWIR camera.

14. The method of claim 13, wherein the machine learning system comprises an artificial intelligence neural network, and wherein the parameter is a neural network parameter.

15. The method of claim 12, wherein the MWIR image data is generated as a result of MWIR spectral imaging of the mixture of materials.

16. The method of claim 12, wherein the black plastic piece cannot be identified using NIR spectroscopy.

17. A computer program product stored on a computer readable storage medium, which when executed performs a method for processing a heterogeneous mixture of separable materials comprising dark plastic pieces, the method comprising:

Receiving MWIR image data for each of the heterogeneous mixtures of materials captured with an MWIR camera; and

based on the processing of the MWIR image data by a machine learning system, certain materials in the heterogeneous mixture of materials are classified with the machine learning system as including dark plastic pieces, wherein the classification is based on a knowledge base generated from previously generated classifications of one or more samples of dark plastic pieces.

18. The computer program product of claim 17, wherein the knowledge base comprises a library of observed MWIR spectral characteristics captured by an MWIR camera configured to capture MWIR images of the one or more samples of the dark plastic piece.

19. The computer program product of claim 18, further comprising sending information about the classification to an automated sorting device, such that the automated sorting device is capable of sorting certain materials of the heterogeneous mixture of materials from the heterogeneous mixture according to the classification.

20. The computer program product of claim 19, wherein the dark plastic piece is a black plastic piece.