WO2025201972A1

WO2025201972A1 - A sensor module for material identification, a system, a method, and a computer program

Info

Publication number: WO2025201972A1
Application number: PCT/EP2025/057428
Authority: WO
Inventors: Simon Stahl; Silvia Rosselli; Anthony Roberts; Gabriele Barbara NELLES; Mustafa Sarpasan; Sebastian EMMERLING
Original assignee: Sony Europe Bv; Sony Semiconductor Solutions Corp
Current assignee: Sony Europe Bv; Sony Semiconductor Solutions Corp
Priority date: 2024-03-27
Filing date: 2025-03-18
Publication date: 2025-10-02
Anticipated expiration: 2026-09-27

Abstract

A sensor module for material identification, the sensor module comprising: an array of pixels comprising at least two triboelectric sensors, wherein each of the at least two triboelectric sensors is made of a different triboelectric material; and circuitry configured to: acquire data from the array of pixels; perform pre-processing on data from the array of pixels, the preprocessing including converting the data from the array of pixels to a frequency domain; and perform material identification based on the pre-processed data from the array of pixels, the material identification including the use of a trained model.

Description

A SENSOR MODULE FOR MATERIAL IDENTIFICATION, A SYSTEM, A METHOD, AND A COMPUTER PROGRAM

BACKGROUND:

Field of the Disclosure

The present disclosure relates to a sensing module for material identification, a system, a method, and a computer program.

Description of the Related Art

The “background” description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in the background section, as well as aspects of the description which may not otherwise qualify as prior art at the time of filing, are neither expressly or impliedly admitted as prior art against the present invention.

Due to the global issues in waste management, it is crucially important to improve established methods of separating/sorting waste as well as to develop new methods for this purpose. As a means to reduce the amount of wasted materials, recycling of materials plays an important role. Accordingly, methods of sorting materials for sustainable reuse and recycling are required.

Furthermore, as well as global issues in waste management, methods of validating materials for quality assurance or quality control purposes are also required.

There are already some established methods (and technologies) for separating mixed plastic, including identifying density differences or through selective dissolution of one component of the mixture to sort, for example. However, these methods themselves are not sustainable since they consume lots of resources (such as power, solvents and the like) and they are limited to certain specific materials. Some additional methods of sorting, such as measuring the infra-red spectrum, are time consuming and expensive. Other additional methods, such as magnetic sorting methods, are not specific, such that only certain types of materials (such as magnetic versus non-magnetic materials) can be distinguished from each other.

Furthermore, methods such as optical imaging - identifying shapes, colors, polarization - in both the UV, visible, or infra-red spectrum, are used in quality control and assurance - but these can be confused by similar shaped or similarly colored, especially dark colored, non- reflective, objects. Accordingly, new means and methods are needed that are sustainable and complementary to these methods, with broader versatility in terms of materials ranging from organic to inorganic materials.

It is an aim of the present disclosure to address these issues.

SUMMARY:

Embodiments of the present disclosure are defined by the independent claims. Further aspects of the disclosure are defined by the dependent claims.

In accordance with embodiments of the disclosure, improved identification of materials can be achieved. In particular, embodiments of the disclosure enable rapid identification of materials with higher accuracy of overall sorting process. Moreover, compared to some established methods (and technologies) for separating/sorting materials, embodiments of the present disclosure have a lower power consumption and therefore provide a more efficient way of separating/sorting materials. Furthermore, embodiments of the disclosure improve the accuracy, reduce the cost, increase the purity of sort, and increase the speed and thus the throughput of sorting.

The present disclosure is not particularly limited to these advantageous technical effects. Further technical effects will become apparent to the skilled person when reading the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS:

Figure 1 illustrates an example of an apparatus in accordance with embodiments of the disclosure;

Figure 2 illustrates a sensor module for material identification according to embodiments of the disclosure;

Figure 3 illustrates an example of a trained model in accordance with embodiments of the disclosure;

Figure 4 illustrates an example implementation of a process for material identification in accordance with embodiments of the disclosure;

Figure 5A to 5G illustrate an example material identification in accordance with embodiments of the disclosure;

Figure 6 illustrates an example system in accordance with embodiments of the disclosure;

Figure 7 illustrates a method of material identification in accordance with embodiments of the disclosure. DESCRIPTION OF THE EMBODIMENTS:

The foregoing paragraphs have been provided by way of general introduction, and are not intended to limit the scope of the following claims. The described embodiments, together with further advantages, will be best understood by reference to the following detailed description taken in conjunction with the accompanying drawings (wherein like reference numerals designate identical or corresponding parts throughout the several views).

Referring to Figure 1, an apparatus 1000 according to embodiments of the disclosure is shown. Typically, an apparatus 1000 according to embodiments of the disclosure is a computer device such as a personal computer or a terminal connected to a server. Indeed, in embodiments, the apparatus may also be a server. The apparatus 1000 is controlled using a microprocessor or other processing circuitry 1002. In some examples, the apparatus 1000 may be a portable computing device such as a mobile phone, laptop computer or tablet computing device.

The processing circuitry 1002 may be a microprocessor carrying out computer instructions or may be an Application Specific Integrated Circuit. The computer instructions are stored on storage medium 1004 which may be a magnetically readable medium, optically readable medium or solid state type circuitry. The storage medium 1004 may be integrated into the apparatus 1000 or may be separate to the apparatus 1000 and connected thereto using either a wired or wireless connection. The computer instructions may be embodied as computer software that contains computer readable code which, when loaded onto the processor circuitry 1002, configures the processor circuitry 1002 to perform a method according to embodiments of the disclosure.

Additionally, an optional user input device 1006 is shown connected to the processing circuitry 1002. The user input device 1006 may be a touch screen or may be a mouse or stylist type input device. The user input device 1006 may also be a keyboard or any combination of these devices.

A network connection 1008 may optionally be coupled to the processor circuitry 1002. The network connection 1008 may be a connection to a Local Area Network or a Wide Area Network such as the Internet or a Virtual Private Network or the like. The network connection 1008 may be connected to a server allowing the processor circuitry 1002 to communicate with another apparatus in order to obtain or provide relevant data. The network connection 1002 may be behind a firewall or some other form of network security.

Additionally, shown coupled to the processing circuitry 1002, is a display device 1010. The display device 1010, although shown integrated into the apparatus 1000, may additionally be separate to the apparatus 1000 and may be a monitor or some kind of device allowing the user to visualise the operation of the system. In addition, the display device 1010 may be a printer, projector or some other device allowing relevant information generated by the apparatus 1000 to be viewed by the user or by a third party.

As explained in the Background, due to the global issues in waste management, it is crucially important to improve established methods of separating/sorting waste as well as to develop new methods for this purpose. As a means to reduce the amount of wasted materials, recycling of materials plays an important role. Accordingly, methods of sorting materials for sustainable reuse and recycling are required.

EP22166160.6 (Sustainable Sorting Device and Method Using Self-Powered TENG) describes the use of a self-powered sensing element comprising a triboelectric nanogenerator (TENG) and its use in material identification and/or distinguishing between different materials or different material surfaces.

However, there is a desire to further improve the accuracy and efficiency with which materials can be separated/sorted.

Accordingly, a sensor module for material identification, a system, a method and a computer program are provided in accordance with embodiments of the disclosure.

Figure 2 illustrates a sensor module for material identification in accordance with embodiments of the disclosure.

The sensor module of Figure 2 for material identification, comprising: an array of pixels 2002 and circuitry 2004. The array of pixels 2002 comprises least two triboelectric sensors 2002A, 2002B, wherein each of the at least two triboelectric sensors 2002A, 2002B is made of a different triboelectric material.

Furthermore, the circuitry 2004 configured to: acquire data from the array of pixels; perform pre-processing on data from the array of pixels, the pre-processing including converting the data from the array of pixels to a frequency domain; and perform material identification based on the pre-processed data from the array of pixels, the material identification including the use of a trained model.

The sensor module of the embodiments of the disclosure is able to perform rapid identification of materials with higher accuracy of overall sorting process. Moreover, compared to some established methods (and technologies) for separating/sorting materials, embodiments of the present disclosure have a lower power consumption and therefore provide a more efficient way of separating/sorting materials. Furthermore, the sensor module of embodiments of the disclosure improves the accuracy, reduces the cost, increases the purity of sort, and increases the speed and thus the throughput of sorting.

In particular, the use of the circuitry 2004 to perform pre-processing on the data from the array of pixels in combination with the use of the trained model enables the sensor module 2000 of the present disclosure to achieve these advantageous technical effects.

It will be appreciated that the geometry of the sensor module 2000 and, in particular, the geometry of the pixel array 2002 is not particularly limited in accordance with embodiments of the disclosure. In examples, the array 2002 may comprise a linear array of pixels. However, in examples, the array may comprise a multilinear array of pixels. Indeed, in examples, the geometry of the array may include a non-linear arrangement of the pixels. The specific geometry of the array 2002 which is used may depend upon the situation to which the embodiments of the disclosure are applied. For example, when used in a sliding mode (where the material to be identified is brought into a sliding contact with the sensor module) a linear arrangement of the array 2002 may enable a more reliable contact to be made. This will be described in more detail later.

When a material to be identified is brought into contact with the triboelectric sensors of the sensor module, this generates a unique triboelectric “fingerprint” which can be used for identification of the material.

Furthermore, it will be appreciated that while the sensor module 2000 of Figure 2 of the present disclosure is illustrated with an array 2002 comprising two triboelectric sensors 2002A and 2002B, the present disclosure is not particularly limited in this regard. That is, while at least two triboelectric sensors are provided in accordance with embodiments of the disclosure, the number of triboelectric sensors may include a number greater than two. Indeed, there may be a number of triboelectric sensors much larger than two. The more triboelectric sensors which are included in the array 2002, the higher the resolution of the sensing module will become.

The triboelectric sensors 2002A and 2002B of the array of pixels do not require an external power source. That is, the linear array of pixels of the sensor module are self-powering. Accordingly, the efficiency of the sensor module is improved.

As noted, the sensor module 2000 comprises circuitry 2004. In examples, circuitry 2004 may be a microprocessor carrying out computer instructions or may be an Application Specific Integrated Circuit. The computer instructions may be stored on storage medium which may be a magnetically readable medium, optically readable medium or solid-state type circuitry, for example.

As explained above, the circuitry 2004 is configured to perform pre-processing on the data from the array of pixels. Performing pre-processing on the data from the array of pixels 2002 enables the data from the array of pixels 2002 is optimized for use with the trained model. Indeed, according to embodiments of the disclosure, the processing performed by the circuitry 2004 includes converting the data from the array of pixels to the frequency domain. By converting the data from the array of pixels 2002 to the frequency domain, the accuracy of the material identification performed by the sensor module can be further improved. That is, since the trained model receives data which has been converted to the frequency domain (having been subjected to the pre-processing by the circuitry 2004) noise and jitter can be reduced, thus leading to a more accurate identification of the material.

However, it will be appreciated that the present disclosure is not limited solely to preprocessing including converting the data from the array of pixels to a frequency domain. Many other types of pre-processing may also be performed by circuitry 2004 prior to the use of the trained model for identification of the material.

In examples, the pre-processing performed by the circuitry 2004 may further comprise performing principal component analysis on the data from the array of pixels. Principal component analysis of the data is a statistical method which can be used to reduce the data from the array of pixels to its essential features (so-called principal components).

Advantageously, the use of principal component analysis on the data from the array of pixels can reduce the data from the pixels to a smaller set (the principal components) while maintaining the important patterns (or fingerprints) within the data. By performing the material identification on the principal components in this way, the efficiency of the sensor module can be further improved.

In examples, the pre-processing performed may further include applying a rolling average filter to the data from the array of pixels. Advantageously, applying a rolling average filter to the data from the array of pixels reduces small fluctuations in the signal from the array of pixels, thus producing a more consistent and reliable signal for material identification. This further improves the accuracy and reliability of material identification.

In examples, the pre-processing performed by circuitry 2004 may further includes applying a noise filter to the data from the array of pixels. In examples, the noise filter may include a filter for removing a certain type of noise - such as a Gaussian Noise filter - however the present disclosure is not particularly limited in this regard. Reduction of the noise prior to material identification enables a more accurate and reliable identification of the material to be made.

In examples, the pre-processing may further include applying an offset correction to the data from the array of pixels. In examples, the offset correction can be used in order to correct for variations in the calibration of the sensor module. Therefore, further improvements in the accuracy and reliability of the material identification performed by the sensor module can be achieved.

In examples, the pre-processing performed by the circuitry 2004 of the sensor module 2000 may further include applying a jitter correction to the data from the array of pixels. The way in which the jitter correction is performed by the circuitry 2004 is not particularly limited in accordance with embodiments of the disclosure. Indeed, any type of jitter correction processing can be used in accordance with embodiments of the disclosure as required. However, by removing the jitter in the signal from the pixels of the sensor module 2000, the accuracy and reliability in material identification can be further improved.

In examples, the circuitry 2004 of the sensor module 2000 is further configured to collect voltages on each of the at least two triboelectric sensors with respect to time. In examples, the circuitry 2004 is configured to apply analogue or digital processing to the collected voltages with respect to time. Examples of the analogue or digital processing which can be performed include bandpass filtering, noise rejection, spike rejection or the like. In examples, the circuitry is configured to apply analogue to digital conversion of the processed collected voltages with respect to time. In this way, the data from the triboelectric sensors can be obtained and pre-processed by the circuitry 2004 prior to its use in material identification. In examples, the circuitry 2004 of the sensor module may be communicatively coupled to or with a storage unit (not shown). The storage unit may be either external or internal to the sensor module. In examples, a combination of internal and external storage can be used. The storage unit may be a magnetically readable medium, optically readable medium or solid-state type circuitry. In examples, the storage unit may be any type of non-transitory computer readable storage medium. In examples, the storage unit may contain instructions which are executed by the circuitry 2004 in order to perform one or more of the functions of embodiments of the disclosure. In examples, the storage medium may be used in order to store the trained model which is used to perform the material identification in accordance with embodiments of the disclosure. In examples, the storage medium may be used by the circuitry 2004 to store the processed data. This can include data from the pixel array (before or after pre-processing) and/or a result of material identification once performed by the trained model.

As explained with reference to Figure 2 of the present disclosure, the sensor module 2000 comprises circuitry 2004 which utilizes a trained model for material identification. Further details of the trained model will now be described.

Figure 3 of the present disclosure illustrates an example of a trained model in accordance with embodiments of the disclosure. More specifically, Figure 3 illustrates a process for training a trained model and a process for using the trained model to generate output (a material identification).

In examples, the trained model may include a machine learning model. However, the present disclosure is not particularly limited in this regard and other types of trained model can be used in accordance with embodiments of the disclosure.

In examples, the methods and techniques herein may at least partly be implemented using a supervised machine learning model.

The supervised learning model is a model which is trained using labelled training data (“Training Data” in Figure 3) to learn a function that maps inputs (typically provided as feature vectors) to outputs (i.e. labels). The labelled training data comprises pairs of inputs and corresponding output labels. The output labels are typically provided by an operator to indicate the desired output for each input. The supervised learning model processes the training data to produce an inferred function that can be used to map new (i.e. unseen) inputs to a label. The input data (during training and/or inference) may comprise various types of data, such as numerical values, images, video, text, or audio. In the present disclosure, the training data includes signals generated by material when that material comes into contact with the triboelectric sensors of the sensor module. For example, the training data can include details of unique triboelectric fingerprints of different types of materials. Raw input data may be pre- processed to obtain an appropriate feature vector used as input to the model - for example, features of an image or audio input may be extracted to obtain a corresponding feature vector. It will be appreciated that the type of input data and techniques for pre-processing of the data (if required) may be selected based on the specific task the supervised learning model is used for.

In order to further improve the accuracy and efficiency with which the trained model can be used for material identification, it will be appreciated that the trained model can be trained on a large library or database of different types of materials. This library or database can include multiple entries for different types of materials - at different stages of degradation, for example. Moreover, the library or database can include multiple entries for different combinations of materials. Moreover, the library or database can include multiple entries for different types of materials under different environmental conditions (such as different temperature, humidity and the like).

Once prepared, the labelled training data set is used to train the supervised learning model. During training the model adjusts its internal parameters (e.g. weighting) so as to optimize (e.g. minimize) an error function, aiming to minimize the discrepancy between the model’s predicted outputs and the labels provided as part of the training data (“Adjustment of Model” in Figure 3). In some cases, the error function may include a regularization penalty to reduce overfitting of the model to the training data set.

The supervised learning model may use one or more machine learning algorithms in order to learn a mapping between its inputs and outputs. Example suitable learning algorithms include linear regression, logistic regression, artificial neural networks, decision trees, support vector machines (SVM), random forests, and the K-nearest neighbor algorithm.

In examples, the trained model may be continually updated (re-trained) as new measurements are made by the model and/or as new training data becomes available. Supervised learning can then be performed on this training data to learn a mapping between the input and corresponding outputs.

Once trained, the supervised learning model may be used for inference - i.e. for predicting outputs for previously unseen input data. The supervised learning model may perform classification and/or regression tasks. In a classification task, the supervised learning model predicts discrete class labels for input data, and/or assigns the input data into predetermined categories. In a regression task, the supervised learning model predicts labels that are continuous values.

In the present disclosure, the trained model (once trained) can be used in order to perform material identification (based on the pre-processed data from the array of pixels).

As an example, the trained model (once trained) can be used to predict outputs (“Output” in Figure 3; such as a type of material which has been identified) for a previously unseen input (“Sample Data” in Figure 3; such as pre-processed data from the array of pixels).

In some cases, limited amounts of labelled data may be available for training of the model (e.g. because labelling of the data is expensive or impractical). In such cases, the initially unsupervised learning model may be extended I retrained I additionally-trained with supervision to further use unlabelled data and/or to generate labelled data.

Considering using unlabelled data, the training data may comprise both labelled and unlabelled training data, and semi-supervised learning may be used to learn a mapping between the model’s inputs and outputs. For example, a graph-based method such as Laplacian regularization may be used to extend a SVM algorithm to Laplacian SVM in order to perform semi-supervised learning on the partially labelled training data.

Considering generating labelled data, an active learning model may be used in which the model actively queries an information source (such as a user, or operator) to label data points with the desired outputs. Labels are typically requested for only a subset of the training data set thus reducing the amount of labelling required as compared to fully supervised learning. The model may choose the examples for which labels are requested - for example, the model may request labels for data points that would most change the current model, or that would most reduce the model's generalization error. Semi-supervised learning algorithms may then be used to train the model based on the partially labelled data set.

In this way, a trained model can be trained and used for performing material identification in accordance with embodiments of the disclosure.

As explained with reference to Figure 2 of the present disclosure, the sensor module 2000 can be used for more accurate and reliable identification of material.

A specific example implementation of the sensor module 2000 will now be described. Figure 4 illustrates an example implementation of a process for material identification according to embodiments of the disclosure.

In this specific example implementation, the sensor module comprises a linear array of n pixels (where n is a minimum of 2). Indeed, in this example, a first pixel comprises an electrode 1a of triboelectric material 1a, a second pixel comprises triboelectric material 1b/electrode1b, a third pixel comprises Electrode 2a/triboelectric material 2a and a fourth pixel comprises triboelectric material 2b/electrode2b.

In this specific example implementation, the following hardware initialization (A) may then be performed to prepare the senor module for material identification:

1. Clear status of hardware

2. Set input impedance CH1...CH4.

3. Set trigger settings.

4. Set timebase reference to LEFT.

5. Set timebase position.

6. Set x increment I timebase.

7. Set the analog acquisition sampling rate auto state to OFF.

8. Set analog acquisition sampling rate.

9. Set the record length.

10. Set averaging count.

11. Set encoding / format to ASCII.

12. Set acquisition Mode to SAMPLE by setting averaging to OFF.

13. Set acquisition state to CONTINUOUS run.

However, while certain hardware initialization (A) is performed in this example, it will be appreciated that the present disclosure is not particularly limited in this regard. Indeed, in some examples, a hardware initialization is not required. For example, the hardware may be pre-configured and/or use a number of default or predetermined settings.

Moreover, a number of the steps of the hardware initialization (A) as described above may not necessarily be performed. In examples, only a certain subset of these steps may be performed.

Furthermore, in examples, a hardware initialization may be performed but may be a different type of initialization than hardware initialization (A) described above. For example, the hardware initialization which is performed may vary depending on the specific configuration of the hardware itself. In some examples, the hardware initialization may further include initialization of an analog to digital converter, a multiplexer, a microcontroller or the like. Furthermore, the hardware initialization may include initialization of a triggering method for triggering identification of the material using the sensor module (such as an optical, mechanical, acoustic method of triggering material identification, for example).

Once the sensor module - and associated hardware - has been initialized, the sensor module can then be used for material identification.

In order to perform the material identification, the sensor module (and specifically, the controller of the sensor module) should perform a process of data acquisition in order to acquire data from the pixels of the sensor module.

As noted above, in examples, the sensor module may be configured to perform data acquisition only when a certain trigger event has been detected. The trigger event may be used to detect when a material to be identified is in contact with the sensor module, for example. However, in examples, data acquisition may be continually performed.

In this specific example implementation, the following data acquisition (B) may then be performed:

1. With each sample, repeat measurement e.g. 50 times, also interleave measurements of samples:

2. Collect 4 channels of voltage data from scope.

(Hardware acquisition steps in measurement loop):

3. Set timebase position.

4. Set trigger slope and level.

5. Set acquisition state to stop.

6. Set acquisition mode to SAMPLE.

7. Set acquisition state to SINGLE run.

8. Wait for measurement complete.

9. Read curves of CH 1...CH4.

10. Set acquisition mode to SAMPLE.

11. Set acquisition state to CONTINUOUS run.

12. Append to measurement Raw-datafile, one file per material. Save all acquired curves to hard disk.

However, while certain data acquisition (B) is performed in this example, it will be appreciated that the present disclosure is not particularly limited in this regard. Indeed, in some examples, the data acquisition which is performed may be very different than that described above for (B). As an example, the data acquisition process performed by the sensor module may also include, alternatively or in addition, data acquisition such as bandpass filters, bandwidth optimization, noise suppression or the like.

Moreover, a number of the steps of the data acquisition process (B) as described above may not necessarily be performed. In examples, only a certain subset of these steps may be performed.

Thus, the data acquisition process (B) is an example implementation of the process of acquiring data from the array of pixels of the sensor module as performed by the circuitry 2004 as described with reference to Figure 2 of the present disclosure.

Once the data acquisition process has been performed, data pre-processing may be performed.

Indeed, in this specific example implementation, a data pre-processing may be performed by:

1. With each measurement in a material’s Raw-datafile loaded from hard disk or direct from (E):

2. Apply 50Hz line noise removal to each channel’s data (find frequency, voltage-offset, amplitude & phase for each channel, then subtract from corresponding channel’s data).

3. Remove short-duration voltage spikes, e.g. cosmic rays (polynomial spike filtering, median filtering), or EMI interference (applying a rolling averaging filter).

4. Apply X-offset peak/jitter correction (find location of max. peak, scan for earliest peak within threshold (30% of max peak), and time-shift data to align all peaks).

5. Append to material’s Training-set datafile.

However, while a certain data-processing is performed in this example, it will be appreciated that the present disclosure is not particularly limited in this regard. Indeed, in some examples, the data pre-processing which is performed may be very different than that described above for (C).

As an example, the data pre-processing may further include one or more additional preprocessing methods including jitter correction, offset correct, rolling average filtering or the like.

Moreover, a number of the steps of the data pre-processing (C) as described above may not necessarily be performed. In examples, only a certain subset of these steps may be performed. Data pre-processing such as the data pre-processing (C) is an example implementation of the pre-processing performed by circuitry 2004 as described with reference to Figure 2 of the present disclosure.

In this specific example implementation, once the data pre-processing (C) has been performed, a further pre-processing method of Principal Component Analysis (D) may then be performed:

1. With a collection of training-sets datafiles:

2. If information gain ranking is enabled then find 500 top-ranked features, else find all input dimensions (4K).

3. Calculate 32 Principal Components.

4. Train ANN (2000 iterations, 0.1 sigmoid delta, 0.4 learning rate, 0.05 error limit).

5. Save results to material’s Machine-Learning datafile to hard disk.

In some examples, a different analysis may be performed. Therefore, the present disclosure is not particularly limited to the use of Principal Component Analysis. In examples, principal component analysis then ANN training or a different analysis method may be performed (e.g. multi-cluster analysis, linear discriminant analysis, Non-linear dimensionality reduction techniques, comparison with PCA (supervised) using LIMAP, tSNE, or Isomap- Metric learning, to recognize the similarity or differences between clusters of similar materials (supervised) using Triplet loss, Variational autoencoders (unsupervised).

However, in examples, the step of additional pre-processing in accordance with step (D) is an optional step which need not be performed. Therefore, the present disclosure is not particularly limited in this respect.

Then, a step of validation and material identification (E) may be performed:

1. Load Machine-Learning datafile from hard disk to ANN.

2. Validate / Material ID using ANN: either,

3. With a pre-processed Training-Set ANN validation, or

4. With a direct measurement = Data-acquisition (A&B) Data-pre-processing (C) ANN validation (D).

The step of validation and material identification (E) is an example of performing material identification based on the pre-processed data from the array of pixels, the material identification including the use of a trained model as described with reference to Figure 2 of the present disclosure. <Examples>

A number of specific examples of material identification in accordance with embodiments of the disclosure will now be described with reference to Figures 5A to 5G of the present disclosure.

Turning now to Figures 5A and 5B of the present disclosure, an example material identification in accordance with embodiments of the disclosure is shown. The example material identification shown in Figures 5A and 5B of the present disclosure has been performed in accordance with the example implementation as described with reference to Figure 4 of the present disclosure.

As shown in this example of Figure 5A and 5B, cubes of plastic made of different materials, blends and deterioration can be distinguished.

Specifically, in this specific example the sensor module comprises 4 pixels, where, Pixel 1 = PMMA | Gap | PET, Pixel 2 = PTFE | Gap | PA6, Pixel 3 = LDPE |Gap | PA6, Pixel 4 = PTFE | Gap | LDPE.

Furthermore, in this example, the five different cubes of plastic (each of 5 * 5 x 5 cm size) comprised of polycarbonate (PC) plastic, or PC plastic with high deterioration, or PC with Acrylonitrile Butadiene Styrene (ABS), or ABS, or Poly(methyl methacrylate) - PMMA.

The difference in the principal components (PC#) for the different plastics and materials is shown on a plot of PC1 , PC2 and PC3 in Figure 5A of the present disclosure. Furthermore, the different signals for each of the individual pixels for the different cubes is shown in Figure 5B of the present disclosure. Therefore, each material has its own unique fingerprint.

In accordance with embodiments of the disclosure, a trained model (such as that described with reference to Figure 3 of the present disclosure) can be used in order to perform material identification on the basis on the basis of the different unique fingerprints of the materials.

While this example of Figures 5A and 5B is shown with reference to material identification between five different plastics, it will be appreciated that the present disclosure is not particularly limited in this regard. Indeed, more generally, embodiments of the present disclosure can be applied to the material identification of any type of material, including textiles. Accordingly, the claimed invention is not specifically limited in this regard.

Indeed, Figures 50 and 5D illustrate an example of material identification for a number of different blends of textiles. In this example, ten different textiles (or blends of textiles) are provided. The first is a sample of Polyester 100%. The second is a sample of Cotton 100%. The third is a sample of Polyester 30% and Cotton 70%. The fourth is a sample of Polyester 65% and Cotton 35%. The fifth is a sample of Viscose 100%. The sixth is a sample of Wool 100%. The seventh is a sample of Linen 100%. The eighth is a sample of Viscose 85% and Linen 15%. The ninth is a sample of Viscose 70% and Linen 30%. Finally, the tenth is a sample of Viscose 50% and Linen 50%.

These samples are illustrated in Figure 5C of the present disclosure. In this specific example, a 5 yen coin is shown juxtaposed with each sample for scale. However, it will be appreciated that the dimensions of the samples are not limited to those shown in the figures. More generally, any suitable dimensions of the sample can be used in accordance with embodiments of the disclosure as desired depending on the situation to which the embodiments of the disclosure are applied.

While ten different samples are provided in this example, it will be appreciated that this is to provide an example of the material identification which can be performed by embodiments of the disclosure. The present disclosure is not particularly limited in this regard.

Figure 5D shows how the signal from the sensor module (more specifically, from the pixel array of the sensor module) can be used in order to distinguish between the different types of material shown in Figure 5C (i.e. the ten different samples). These distinctions between the signals from the sensor module can be used to identify the material present. Indeed, even different textiles comprising a same mixture of types of materials but at different ratios can be distinguished. For example, sample 1 (100% polyester) has a very different signal to sample 4 (65% polyester and 35% cotton). This in turn has a very different signal to sample 3 (polyester 30% and cotton 70%).

Moreover, different types of materials also have very different signals. For example, sample 7 (a textile of 100% linen) has a very different signal to sample 6 (a sample of 100% wool).

Therefore, in accordance with embodiments of the disclosure, a trained model (such as that described with reference to Figure 3 of the present disclosure) can be used - once trained - in order to perform material identification on the basis on the basis of the different unique fingerprints of the materials.

Figures 5E and 5F illustrate an example of material identification for a number of black textiles (where each of the samples is formed with a different type of textile (i.e. a different material). This demonstrates that it is not the color of the textiles which is used for identification but rather the unique triboelectric signature of the materials which is measured when those materials come into contact with the pixels of the sensor module.

In this example, five different textiles (or blends of textiles) are provided. The first - sample 1A - is a sample of 100% polyester. The second - sample 2A - is a sample of polyester 65% and cotton 35%. The third - sample 3A - is a sample of 100% cotton. The fourth - sample 4A - is a sample of cotton 40% and linen 60%. Finally, the fifth sample in this example - sample 5A - is a sample of 100% linen.

Again, as mentioned with reference to the example of Figure 5C of the present disclosure, a 5 yen coin is shown juxtaposed with each sample for scale in this specific example.

However, again, it will be appreciated that the dimensions of the samples are not limited to those shown in the figures and the dimensions of the sample can be much larger or much smaller than that shown in this specific example.

These samples were processed in accordance with embodiments of the disclosure to measure the unique signatures of the materials (e.g. using a sensor module 2000 according to the example implementation according to Figure 4 of the present disclosure).

As can be seen in Figure 5F, these five samples occupy a very different region of the parameter space - even though the visual appearance of these samples is very similar. Therefore, the embodiments of the disclosure can be used in order to perform material identification for different samples even when those samples have a very similar visual appearance. Accordingly, embodiments of the disclosure provide material identification which is independent of visual information.

Figure 5G shows clustering of the five samples of Figures 5E an 5F in parameter space.

As can be seen in Figure 5G, the clustering of the five samples in parameter space is improved when the same datasets are processed using a fast Fourier transform (FFT) into the frequency domain. This leads to fewer errors being made by the Artificial Neural Network (ANN), resulting in improved identification of the material.

Indeed, as can be seen in Figure 5G, the clustering of classification (dots), and also the accuracy of ANN prediction (crosses) are improved using same datasets when processed in the frequency-domain (FFT) rather than the time-domain (TD).

Accordingly, embodiments of the disclosure (which perform pre-processing on data from the array of pixels, the pre-processing including converting the data from the array of pixels to a frequency domain) are particularly advantages in improving the accuracy and reliability of material identification. In this way, a sensor module 2000 can be used for material identification.

Hence, more generally, embodiments of the disclosure provide A sensor module for material identification, the sensor module comprising: an array of pixels comprising at least two triboelectric sensors, wherein each of the at least two triboelectric sensors is made of a different triboelectric material; and circuitry configured to: acquire data from the array of pixels; perform pre-processing on data from the array of pixels, the pre-processing including converting the data from the array of pixels to a frequency domain; and perform material identification based on the pre-processed data from the array of pixels, the material identification including the use of a trained model.

A sensor module in accordance with embodiments of the disclosure has been described. Moreover, an example implementation for material identification using the sensor module has also been described.

However, it will be appreciated that the sensor module in accordance with embodiments of the disclosure is a contact sensor. That is, a sample for which a measurement is to be made (for the purpose of material identification) should contact the sensor in order that the sensor module can measure the unique triboelectric fingerprint of the sample.

The way in which the sample is brought into contact with the sensor module for measurement and material identification is not particularly limited in accordance with embodiments of the disclosure. For example, the way in which the sample is brought into contact with the sensor module for measurement may depend on the material which is to be measured and/or on the situation to which the embodiments of the disclosure are applied.

However, in examples, a system for material identification is provided. The system for material identification comprises a sensor modular in accordance with embodiments of the disclosure (such as that described with reference to Figure 2 of the present disclosure) and a material transport device configured to transport a material into contact with the pixel array of the sensor module.

It will be appreciated that while the sensor module is a self-powering sensor module, the system (including the material transport device) may, in some examples, require that an additional power source is provided (e.g. to power the material transport device and/or to perform software/hardware acquisition and analysis). However, this is not necessarily required in all situations. In examples, the material transport device may be a passive transport device (being a transport device which can operate without power). An example of a passive transport device includes a gravity conduit which is used to bring the material into contact with the sensor module. Therefore, even when a material transport device is used in a system with the sensor module to perform the material identification, the system may have very low power consumption.

Figure 6 illustrates a system (6000) for material identification in accordance with embodiments of the disclosure. The example system illustrated in Figure 6 is merely one example of a system which can be used for material identification in accordance with embodiments of the disclosure.

In this example, a sample 6002 to be identified is illustrated. The sample may be any type of material (including a plastic material, a textile material or the like). In this example, the sample is shown as having a small cube shape. However, the shape of the sample is not particularly limited in this regard. More generally, the sample may have any suitable type of shape.

In this example, the sample will be identified using a sensor module 2000 for material identification as described with reference to Figure 2 of the present disclosure.

In order for the sensor module 2000 to make a measurement of the sample (and thus perform identification of the sample) the sample must be brought into contact with the pixel array of the sensor module 2000. Accordingly, the system illustrated in Figure 6 comprises a material transport device configured to transport the material into contact with the pixel array of the sensor module.

The sample to be measured is initially placed on the material transport device (e.g. by a user or the like). Then, the material transport device is configured to transport the sample such that it contacts the sensor.

In this example, the material transport device comprises a plurality of rollers 6000A, 6000B, 6000C and 6000D and a guide path 6004. The rollers 6000A, 6000B, 6000C and 6000D are configured to make rolling contact with the sample 6002 in order to more the sample 6002 along the guide path such that it comes into contact with the sensor module 2000.

While only four rollers are illustrated in this specific example, it will be appreciated that the present disclosure is not particularly limited in this regard. Indeed, a number of additional rollers may be provided along the guide path 6004 in order to assist in movement of the sample along the guide path. Furthermore, while only a single sample is shown in this example, the material transport system may, more generally, be used in order to bring any number of samples into contact with the sensor module 2000. Moreover, while only a single sensor module 2000 is shown in this example, in embodiments a plurality of sensor modules may be provided along the guide path of the material transport device.

It will be appreciated that the material transport device and system illustrated in Figure 6 of the present disclosure provide only one example of a material transport device in accordance with embodiments of the disclosure.

In examples, the material transport device may further comprise a material holder (not shown in Figure 6), the material holder configured to hold a material for contact with the pixel array of the sensor module. The use of a material holder may enable certain types of sample to be more easily contained and brought into contact with the pixel array of the sample module 2000. The type of the material holder which is used is not particularly limited in accordance with embodiments of the disclosure and may vary depending on the situation (including, for example, in dependence on the type of the material the sample to be measured is formed). In some, the material holder may consist of a basket, a tray or the like.

In examples, the material transport device may include one or more other types of device in addition or as an alternative to the material transport device which is illustrated in Figure 6 of the present disclosure. For example, the material transport device may comprise at least one or more of: a conveyor belt, a handheld device, a roller, a set of rollers, a moving plate, a turning disk, a robotic device, a roller conveyor, a vibrating conveyor, a drum conveyor, a screw conveyor, an Olds elevator, a gravity ramp, an electric field, a magnetic field, a blown conduit and/or a gravity conduit.

The type of material transport device which is used may depend - at least in part - on the type of material which is to be measured and/or the situation to the embodiments of the disclosure are applied. For example, when the material to be measured is a textile, a certain type of transport device (such as a roller and/or conveyor) may be particularly advantageous. Alternatively, for addition, the type of material transport device which is used may depend at least on the volume of material which is to be measured. For example, in an industrial setting (such as industrial waste sorting applications) then a large-scale material transport device may be used.

The type of contact which the material transport device causes between the sample to be measured and the pixel array of the sensor module is not particularly limited in accordance with embodiments of the disclosure. That is, provided the sample to be measured (or identified) comes into contact with the pixel array of the sensor module, the measurement can be performed. However, in examples, the type of contact caused between the material and the pixel array of the sensor module may include at least one of: a linear swipe contact, and/or a rotational swipe contact and/or a bouncing swipe contact.

Indeed, advantageously, since the embodiments of the disclosure enable rapid identification of the material to be performed, the type of contact which is made between the material and the sensor module is very small. In other words, the sample does not have to come into prolonged contact with the pixel array of the sensor module. Therefore, the embodiments of the disclosure are easily applicable to an industrial setting and/or large volumes of independent samples (where those samples only come into brief contact with the pixel array).

In the example of Figure 6 of the present disclosure, the material transport system is configured to bring the material into contact with the sensor module while maintaining the position of the sensor module. Maintaining the position of the sensor module may further improve the accuracy and reliability of the measurements which can be performed as a reduction in the requirement to recalibarate the sensor module. Indeed, in examples, the system is configured to maintain the sensor module with at least one of a: a controlled position, a controlled force (contact force), a controlled speed, a controlled acceleration, and/or a controlled position.

Hence, more generally, a method of material identification is provided in accordance with embodiments of the disclosure.

Figure 7 illustrates a method of material identification in accordance with embodiments of the disclosure.

The method starts with step S7000 and proceeds to step S7002.

In step S7002, the method comprises acquiring data from an array of pixels comprising at least two triboelectric sensors, wherein each of the at least two triboelectric sensors is made of a different triboelectric material.

In step S7004, the method comprises performing pre-processing on data from the array of pixels, the pre-processing including converting the data from the array of pixels to a frequency domain.

Then, in step S7006, the method comprises performing material identification based on the pre-processed data from the array of pixels, the material identification including the use of a trained model. The method then proceeds to and ends with step S7008.

It will be appreciated that the present disclosure is not particularly limited to the steps and arrangement of steps illustrated in Figure 7 of the present disclosure. In particular, a number of these steps may be performed in parallel. Furthermore, a number of additional steps may also be performed in addition to the steps illustrated in Figure 7 as has been described hereinbefore.

Thus, in accordance with embodiments of the disclosure, improved identification of materials can be achieved. In particular, embodiments of the disclosure enable rapid identification of materials with higher accuracy of overall sorting process. Moreover, compared to some established methods (and technologies) for separating/sorting materials, embodiments of the present disclosure have a lower power consumption and therefore provide a more efficient way of separating/sorting materials.

Furthermore, it will be appreciated that the methods of the present disclosure may be carried out on conventional hardware (such as that described previously herein) suitably adapted as applicable by software instruction or by the inclusion or substitution of dedicated hardware.

Thus, the required adaptation to existing parts of a conventional equivalent device may be implemented in the form of a computer program product comprising processor implementable instructions stored on a non-transitory machine-readable medium such as a floppy disk, optical disk, hard disk, PROM, RAM, flash memory or any combination of these or other storage media, or realized in hardware as an ASIC (application specific integrated circuit) or an FPGA (field programmable gate array) or other configurable circuit suitable to use in adapting the conventional equivalent device.

Separately, such a computer program may be transmitted via data signals on a network such as an Ethernet, a wireless network, the Internet, or any combination of these or other networks.

In addition, embodiments of the present disclosure can be arranged in accordance with the following numbered clauses:

1. A sensor module (2000) for material identification, the sensor module comprising: an array of pixels (2002) comprising at least two triboelectric sensors (2002A, 2002B), wherein each of the at least two triboelectric sensors is made of a different triboelectric material; and circuitry (2004) configured to: acquire data from the array of pixels; perform pre-processing on data from the array of pixels, the pre-processing including converting the data from the array of pixels to a frequency domain; and perform material identification based on the pre-processed data from the array of pixels, the material identification including the use of a trained model.

2. The sensor module according to clause 1, wherein the array of pixels is configured (i) as a linear array of pixels or (ii) as a plurality of linear arrays of pixels.

3. The sensor module according to clause 1 or clause 2, wherein the pre-processing further includes applying a rolling average filter to the data from the array of pixels.

4. The sensor module according to any preceding clause, wherein the pre-processing further includes applying a noise filter to the data from the array of pixels.

5. The sensor module according to any preceding clause, wherein the pre-processing further includes applying an offset correction to the data from the array of pixels.

6. The sensor module according to any preceding clause, wherein the pre-processing further includes applying a jitter correction to the data from the array of pixels.

7. The sensor module according to any preceding clause, wherein the circuitry is further configured to collect voltages on each of the at least two triboelectric sensors with respect to time.

8. The sensor module according to clause 7, wherein the circuitry is further configured to apply analogue or digital processing to the collected voltages with respect to time.

9. The sensor module according to clause 8, wherein the circuitry is configured to apply analogue to digital conversion of the processed collected voltages with respect to time.

10. The sensor module according to any preceding clause, wherein the pre-processing further includes performing principal component analysis on the data from the array of pixels.

11. The sensor module according to any preceding clause, wherein the trained model comprises at least one of: a machine learning model, a deep learning model, a neural network and/or an Artificial Intelligence model. 12. A system (6000) for material identification, the system comprising: a sensor module (2000) according to any of clauses 1 to 11 ; and a material transport device (6000A... D, 6004) configured to transport a material into contact with the pixel array of the sensor module.

13. The system according to clause 12, wherein the material transport device comprises at least one of: a conveyor belt, a handheld device, a roller, a set of rollers, a moving plate, a turning disk, a robotic device, a roller conveyor, a vibrating conveyor, a drum conveyor, a screw conveyor, an Olds elevator, a gravity ramp, an electric field, a magnetic field, a blown conduit and/or a gravity conduit.

14. The system according to clause 12 or 13, wherein the material transport device is configured to transport the material to contact the pixel array of the sensor module with a swipe contact.

15. The system according to clause 14, wherein the swipe contact is at least one of: a linear swipe contact, and/or a rotational swipe contact, and/or a bouncing swipe contact.

16. The system according to any of clauses 12 to 15, wherein the system is configured to maintain the sensor module with at least one of: a controlled position, a controlled force, a controlled speed, a controlled acceleration and/or a controlled pressure.

17. The system according to any of clauses 12 to 16, wherein the material transport device further comprises a material holder, the material holder configured to hold a material for contact with the pixel array of the sensor module.

18. A method of material identification, the method comprising: acquiring data from an array of pixels comprising at least two triboelectric sensors, wherein each of the at least two triboelectric sensors is made of a different triboelectric material (S7002); performing pre-processing on data from the array of pixels, the pre-processing including converting the data from the array of pixels to a frequency domain (S7004); and performing material identification based on the pre-processed data from the array of pixels, the material identification including the use of a trained model (S7006).

19. A computer program comprising instructions which, when executed by a computer, cause the computer to perform a method of material identification, the method comprising: acquiring data from an array of pixels comprising at least two triboelectric sensors, wherein each of the at least two triboelectric sensors made of a different triboelectric material (S7002); performing pre-processing on data from the array of pixels, the pre-processing including converting the data from the array of pixels to a frequency domain (S7004); and performing material identification based on the pre-processed data from the array of pixels, the material identification including the use of a trained model (S7006).

20. A non-transitory computer readable storage medium comprising the computer program according to clause 19.

It will be appreciated that numerous modifications and variations of the present disclosure are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims, the disclosure may be practiced otherwise than as specifically described herein.

In so far as embodiments of the disclosure have been described as being implemented, at least in part, by software-controlled data processing apparatus, it will be appreciated that a non-transitory machine-readable medium carrying such software, such as an optical disk, a magnetic disk, semiconductor memory or the like, is also considered to represent an embodiment of the present disclosure.

It will be appreciated that the above description for clarity has described embodiments with reference to different functional units, circuitry and/or processors. However, it will be apparent that any suitable distribution of functionality between different functional units, circuitry and/or processors may be used without detracting from the embodiments.

Described embodiments may be implemented in any suitable form including hardware, software, firmware or any combination of these. Described embodiments may optionally be implemented at least partly as computer software running on one or more data processors and/or digital signal processors. The elements and components of any embodiment may be physically, functionally and logically implemented in any suitable way. Indeed the functionality may be implemented in a single unit, in a plurality of units or as part of other functional units. As such, the disclosed embodiments may be implemented in a single unit or may be physically and functionally distributed between different units, circuitry and/or processors.

Although the present disclosure has been described in connection with some embodiments, it is not intended to be limited to the specific form set forth herein. Additionally, although a feature may appear to be described in connection with particular embodiments, one skilled in the art would recognize that various features of the described embodiments may be combined in any manner suitable to implement the technique.

Claims

CLAIMS:

1) A sensor module for material identification, the sensor module comprising: an array of pixels comprising at least two triboelectric sensors, wherein each of the at least two triboelectric sensors is made of a different triboelectric material; and circuitry configured to: acquire data from the array of pixels; perform pre-processing on data from the array of pixels, the pre-processing including converting the data from the array of pixels to a frequency domain; and perform material identification based on the pre-processed data from the array of pixels, the material identification including the use of a trained model.

2) The sensor module according to claim 1, wherein the array of pixels is configured (i) as a linear array of pixels or (ii) as a plurality of linear arrays of pixels.

3) The sensor module according to claim 1, wherein the pre-processing further includes applying a rolling average filter to the data from the array of pixels.

4) The sensor module according to claim 1, wherein the pre-processing further includes applying a noise filter to the data from the array of pixels.

5) The sensor module according to claim 1, wherein the pre-processing further includes applying an offset correction to the data from the array of pixels.

6) The sensor module according to claim 1, wherein the pre-processing further includes applying a jitter correction to the data from the array of pixels.

7) The sensor module according to claim 1 , wherein the circuitry is further configured to collect voltages on each of the at least two triboelectric sensors with respect to time.

8) The sensor module according to claim 7, wherein the circuitry is further configured to apply analogue or digital processing to the collected voltages with respect to time.

9) The sensor module according to claim 8, wherein the circuitry is configured to apply analogue to digital conversion of the processed collected voltages with respect to time.

10) The sensor module according to claim 1, wherein the pre-processing further includes performing principal component analysis on the data from the array of pixels. 11) The sensor module according to claim 1, wherein the trained model comprises at least one of: a machine learning model, a deep learning model, a neural network and/or an Artificial Intelligence model.

12) A system for material identification, the system comprising: a sensor module according to claim 1; and a material transport device configured to transport a material into contact with the pixel array of the sensor module.

13) The system according to claim 12, wherein the material transport device comprises at least one of: a conveyor belt, a handheld device, a roller, a set of rollers, a moving plate, a turning disk, a robotic device, a roller conveyor, a vibrating conveyor, a drum conveyor, a screw conveyor, an Olds elevator, a gravity ramp, an electric field, a magnetic field, a blown conduit and/or a gravity conduit.

14) The system according to claim 12, wherein the material transport device is configured to transport the material to contact the pixel array of the sensor module with a swipe contact.

15) The system according to claim 14, wherein the swipe contact is at least one of: a linear swipe contact, and/or a rotational swipe contact, and/or a bouncing swipe contact.

16) The system according to claim 12, wherein the system is configured to maintain the sensor module with at least one of: a controlled position, a controlled force, a controlled speed, a controlled acceleration and/or a controlled pressure.

17) The system according to claim 12, wherein the material transport device further comprises a material holder, the material holder configured to hold a material for contact with the pixel array of the sensor module.

18) A method of material identification, the method comprising: acquiring data from an array of pixels comprising at least two triboelectric sensors, wherein each of the at least two triboelectric sensors is made of a different triboelectric material; performing pre-processing on data from the array of pixels, the pre-processing including converting the data from the array of pixels to a frequency domain; and performing material identification based on the pre-processed data from the array of pixels, the material identification including the use of a trained model. 19) A computer program comprising instructions which, when executed by a computer, cause the computer to perform a method of material identification, the method comprising: acquiring data from an array of pixels comprising at least two triboelectric sensors, wherein each of the at least two triboelectric sensors made of a different triboelectric material; performing pre-processing on data from the array of pixels, the pre-processing including converting the data from the array of pixels to a frequency domain; and performing material identification based on the pre-processed data from the array of pixels, the material identification including the use of a trained model. 20) A non-transitory computer readable storage medium comprising the computer program according to claim 19.