Classifications

• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
  • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
  • G01J3/28—Investigating the spectrum
  • G01J3/2823—Imaging spectrometer
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
  • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
  • G01J3/28—Investigating the spectrum
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/045—Combinations of networks
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/0002—Inspection of images, e.g. flaw detection
  • G06T7/0004—Industrial image inspection
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
  • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
  • G01J3/28—Investigating the spectrum
  • G01J3/2823—Imaging spectrometer
  • G01J2003/2826—Multispectral imaging, e.g. filter imaging
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
  • G01N21/84—Systems specially adapted for particular applications
  • G01N21/88—Investigating the presence of flaws or contamination
  • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
  • G01N2021/8883—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges involving the calculation of gauges, generating models
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
  • G01N21/84—Systems specially adapted for particular applications
  • G01N21/88—Investigating the presence of flaws or contamination
  • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
  • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N20/00—Machine learning
  • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/048—Activation functions
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10024—Color image
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10032—Satellite or aerial image; Remote sensing
  • G06T2207/10036—Multispectral image; Hyperspectral image
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10048—Infrared image
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20084—Artificial neural networks [ANN]
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30181—Earth observation
  • G06T2207/30188—Vegetation; Agriculture

Definitions

• the present invention relates to methods and devices for characterizing samples from spectral images, in particular acquired by infrared thermography, and using deep neural networks.
• NIR near-infrared
• UV ultraviolet
• NIR near-infrared
• UV ultraviolet
• radiative heat emitted continuously by any body having a temperature above absolute zero (-273, 15 ° C) is used.
• Specific detectors allow this radiation to be captured in certain wavelengths and retranscribed in luminance values related to the surface temperature of the object, creating thermal images.
• thermographic cameras The miniaturization of infrared thermographic cameras, the reduction of their cost of acquisition and the development of computation capabilities of computers have encouraged the use of such cameras as a non-destructive alternative technique in several applications, such as evaluation of damage, fatigue of materials, or thickness estimation of coatings.
• the possibility of penetrating the coating layer without having any influence on the pigments justifies the use of infrared techniques for the inspection of coating thicknesses such as paint.
• This technique has certain limitations, such as high sensitivity to external reflection, emissivity variations, and the use of a heat source as an excitatory source that can not be considered energy-efficient for example because of the use of one or more high-power flashes.
• This inhomogeneity will directly affect the thermal signature of the target coating observed during both the heating and cooling periods.
• the temperature distribution during a Thermographic inspection has been investigated and measures to reduce the effects of nonuniform temperature distribution have been suggested, such as the use of an image reconstruction algorithm based on a Fourier transform to inhibit the effect of non-uniform heating.
• Other methods are used to improve the thermal contrast and overcome these external artifacts, including the use of thermal contrast, absolute thermal contrast, or modified absolute differential contrast.
• a perceptron multilayer neural network was used to detect and characterize defects using pulsed infrared thermography.
• the results show that phase images are less sensitive to noise but an increase in sampling frequency is strongly recommended for this study.
• the illustration of such results can be found for example in the article "Defect detection in pulsed thermography: a comparison of Kohonen and Perceptron neural networks", by Steve Vallerand et al. Proc. SPIE 3700, Thermosense XXI, March 1999.
• CNN convolutional neuron networks
• input data DE are received at sensors S1, S2, S3, each sensor being able to collect the signals representative of a characteristic F (or "feature" in English) of the sample to be analyzed.
• These signals are transmitted to a neuron network M consisting of multiple layers.
• This model M is driven by one or more sets of DApp training data, including images, at which F characteristics have been annotated and which are subject to a learning algorithm AApp allowing learning the recognition of said characteristics.
• An output C is obtained, according to a set of classes C1, C2, C3, data input DE.
• Application No. CN 6022365 discloses a method of detecting defects on the surface of a material, using a radial basis function (RBF) neuron network and infrared thermography images to create a classifier.
• RBF radial basis function
• Application CN 10 5760883 describes, in the field of the monitoring of the operating status of a mining equipment, a method for automatically identifying the key components of a conveyor belt, using infrared thermography and a so-called neural network.
• BP back-propagation
• Application CN 10 2621150 relates to a method for identifying damage on an aircraft liner, using a Large Support Vector Machine (SVM) algorithm based on a matrix of gray-scale cooccurrences. a signal characterizing different types of damage to establish a classifier. Fa detection of damage on the coating of the aircraft and damage themselves can be classified and identified, for subsequent maintenance treatment.
• SVM Large Support Vector Machine
• the invention responds to the need mentioned above by, in one of its aspects, a method of characterizing a sample, using a set of spectral images of the sample to be characterized previously acquired, in particular by infrared thermography. or spectral imaging, and at least one neural network, the method comprising the steps of: generating at least one volume of values D (Nx, Ny, Ne) of a parameter observed from said spectral images, for a plurality of coordinates (x, y) of the pixels N of the images and a plurality of acquisitions Ne,
• each class being representative of at least one characteristic of the sample to be characterized.
• said at least one neural network in particular at least one layer of the network, has been previously driven by means of images other than real spectral images (that is to say, derived from real samples).
• said at least one neural network can be trained in advance using images called “natural images”, in particular natural images of animals, objects, plants, people.
• said at least one neural network can be trained in advance using images called “virtual images”, that is to say artificially created images by humans, such as non-limiting examples:
• said at least one neural network can be driven in a mixed manner, using natural images and virtual images.
• the classification is performed by a classifier independent of the neural network.
• the classifier can be of the wide margin separator (SVM) type.
• the classifier is of the Softamx or RBF type with a Gaussian nucleus.
• the classification can be done at the level of at least one layer of a perceptron.
• the classification is carried out by the neural network used for the extraction of the characteristics, preferably by the final layer of this network.
• the invention allows a rapid implementation of the system, the database and the learning time being reduced.
• the implementation of the method according to the invention can thus be done on a low-cost embedded system, for example a nano-computer, also called nano-PC, or a dedicated card, allowing the operation of resource-intensive applications and / or requiring real-time.
• SVM classifier makes it possible to work with large data, making it possible to process many types of data, significantly improve recognition performance, and significantly reduce computation time. .
• the robustness of the hybrid architecture according to the invention makes it possible to have a post-processing technique of infrared thermographic data that is slightly sensitive to the non-uniformity of the energy deposition generated by the excitation system and to the measurement conditions. for example, different placements of the spectral image acquisition camera in terms of distances or angles of the lens relative to the sample, or lighting conditions for different acquisitions, depending on the time, the temperature, or the season of the year.
• the different acquisitions may correspond to different acquisition times over a predefined acquisition period, particularly in the case of infrared thermography.
• the different acquisitions correspond to acquisitions at different wavelengths, performed at the same time.
• Spectral imaging includes multispectral or hyperspectral imagery.
• Multispectral imaging consists of acquiring a small and limited number of bands discrete, and does not require the use of a spectrometer to analyze the data.
• Hyperspectral imaging allows the acquisition of a large number of narrow spectral bands through the use of a spectral band separation system such as a spectrometer.
• the data volume D advantageously contains P pixels, for each pixel N in the plane x, y correspond to the coordinates (Nx, Ny, Ne), where Ne is the coordinate of the acquisition. This is the same pixel P from one plane to another, recorded at different times or for different wavelengths.
• the spectral evolution of a pixel for example the temporal evolution of temperature, is thus considered as a one-dimensional signal, forming a tuple of values, and used directly for classification.
• the input data is advantageously transmitted to the neural network in the form of images representing curves corresponding to the values D'x, y (Ne) of the input data set as a function of the acquisition Ne.
• This allows the transposition to the one-dimensional signals of deep-learning-type network principles for the study of natural images, including convolution and dimensionality reduction for feature extraction.
• the fact that the neural network is pre-trained on natural images reduces the number of necessary learning data, while the fact that the neural network is pre-trained on "virtual images" makes it possible to precisely refine the image. desired learning.
• the images can be resized according to the standard dimensions imposed by the neural network used.
• the transformation function applied to the values Dx, y (Ne) of the observed parameter can be the identity function, the values remaining unchanged and being used as such by the neural network.
• said at least one transformation function applied to the values of the observed parameter Dx, y (Ne) is a centering, normalization and / or smoothing function.
• the spectral responses Dx, y (Ne) can thus each be centered, for example with respect to an average value calculated on all the images having served for the learning of said network, or with respect to the first image resulting from the first acquisition, and / or normalized with respect to their maximum, or with respect to a reference value, corresponding in particular to a predefined wavelength.
• the smoothing makes it possible to reduce the irregularities and singularities of the responses.
• Using smoothed spectral responses to compute derivatives avoids artefacts or amplification of derivation noise in the resulting signals.
• a calculation function of the first derivative can be applied to the D (Nx, Ny, Ne) values of the observed parameter to obtain the input data (D'x, y (Ne)).
• the parameter observed is the temperature of the sample
• this calculation makes it possible to take into account the rate of cooling of the sample.
• a calculation function of the second derivative can be applied to the values D (Nx, Ny, Ne) of the observed parameter to obtain the input data (D'x, y (Ne)).
• the observed parameter is the temperature of the sample
• this calculation makes it possible to take into account the acceleration of the cooling of the sample.
• the spectral images used are thermal images acquired by infrared thermography, the parameter observed being the temperature of the sample.
• the principle of infrared thermography is based on the measurement of the energy emitted by the surface of a body in a given wavelength interval, corresponding to the temporal acquisition of the thermal radiation in the infrared bands of the electromagnetic spectrum. This energy is transmitted through appropriate optics to a detector.
• radiometric systems allow non-contact measurement of surface temperature fields at rates of up to several hundred Hertz for images whose average size is approximately 80000 pixels.
• the measurement process generally causes only a slight increase in temperature, which does not risk disturbing data acquisition.
• the detectable temperature variations range from a few tenths to a few tens of degrees Celsius / Kelvin.
• Thermal excitation of the specimen can be achieved.
• the numerical analysis of the thermal data is then carried out.
• the surface of the sample to be characterized may be thermally excited prior to the acquisition of the thermal images, for example by means of a surface excitation device by pulsed illumination such as a flash lamp.
• the neural network is driven from virtual images, and preferably via virtual spectral images derived from simulations / computer creation of mathematical models of virtual samples.
• the parameter observed is the temperature of the sample.
• a virtual sample, computer simulated may for example be in the form of a bilayer of materials, namely a substrate covered with a coating.
• a coating may for example be in the form of a bilayer of materials, namely a substrate covered with a coating.
• the skilled person knows how to choose the number of layers of materials (substrate and / or coating) adapted to the intended application.
• Said coating can be numerically discretized into voxels, for each of which a curve LT representing the evolution of the temperature over time t can be defined (for example: selected in a database, etc.) or calculated from parameters provided by those skilled in the art to a mathematical model.
• a voxel includes all of the layer (s) provided for said coating deposited on the substrate.
• This voxel discretization of the coating is particularly advantageous because it is possible to generate as many voxels as desired, which makes it possible to simulate a wide variety of energy distributions in said coating.
• said mathematical model implements inter alia a generalized equation EQM having the form below:
• T (x, y, t) F [QE (x, y, t); PS; PR; ER; t; ...] (EQM) for which the function F depends inter alia on at least:
• QE the amount of energy deposited in a given voxel with plane coordinates (x, y). This value makes it possible to simulate the distribution of the energy deposited in the coating by an idealized thermal excitation means (for example: a flash lamp).
• an idealized thermal excitation means for example: a flash lamp.
• it is a pulse response of the "Dirac" type, which will be called LRID QE .
• LRID QE the use of other idealized impulse responses may be considered.
• PS a set of physical parameters characterizing the substrate (for example: composition of the material, color of the material, appearance of the material (smooth, matte, etc.), thermal diffusivity, thermal effusivity, etc.) obtained for example via databases communicating with said mathematical model and / or calculated within said model.
• This set of PS parameters may include PS parameters characterizing a single layer and / or globally characterizing the set of layers constituting said substrate.
• PR a set of physical parameters characterizing the coating layer (s) deposited on the substrate (for example: composition of the material, color of the material, appearance of the material (smooth, matte, etc.), thermal diffusivity , thermal effusivity, ).
• This set of PR parameters may include PR parameters characterizing a single layer and / or generally characterizing the set of layers of said coating. These parameters are obtained for example via databases communicating with said mathematical model and / or calculated within said model,
• ER a set of parameters characterizing the thickness (s) of the coating.
• This set of parameters ER can include ER parameters characterizing a single layer and / or generally characterizing at least a subset of layers of said coating, each layer having a specific thickness (identical or different from the other layers).
• the parameters provided to the mathematical model can be of any type: constant or variable, respecting various distributions (uniform, Gaussian, ). This makes it possible to simulate for example: a heterogeneity of the coloring pigment, a heterogeneity of the surface of the coating, ....
• the mathematical model output the ISV X virtual spectral images that allow, in certain embodiments of the invention, to drive the neural network.
• the virtual spectral images ISV X result from the convolution of the curve LT (response resulting from the equation EQM of the model) with a curve of the "impulse response" type LRI, such as for example a "gate function P".
• This impulse response corresponds to the action of an excitation means real thermal.
• the use of other realistic impulse responses, replacing the gate function P can be envisaged.
• the environmental thermal noise b (x, y, t) can also be taken into account, once the convolution step has been performed, according to for example the equation: [T (x , y, t) * P] + b (x, y, t)
• the mathematical model envisaged makes it possible to generate a large variety of thermal responses.
• class 1 corresponds to coating thicknesses ranging from 51 pm to 60 pm, and so on
• class 2 corresponds to coating thicknesses ranging from 51 pm to 60 pm, and so on
• the at least one neural network may be a convolutional neural network.
• the neural network may include one or more convolutional layers and / or one or more fully connected layers.
• each convolutional layer produces an activation of an input image
• the first layers extracting basic characteristics, such as the outline
• the upper layers extracting higher level features, such as texture information.
• Said at least one characteristic extracted from the input data may be the thickness or thickness range of the sample or portions of the sample, a representative amount of a property of the sample, by a layer of paint, a thickness of an intermediate layer, for example of the Sol Gel type originating from solution-gelling processes, the level of water stress of a plant, the variation of pigmentation of plants, for example leaves, flowers or fruits of plants.
• the subject of the invention is also a device for characterizing a sample, in particular by infrared thermography or spectral imaging, comprising:
• a data processing module capable of:
• an analysis module comprising at least one neuron network, capable of at least driving said at least one neuron network by using the input data to extract at least one characteristic from the sample to be characterized, and using said characteristic extracted by the neural network to classify the input data according to a plurality of classes, each class being representative of at least one characteristic of the sample to be characterized.
• Said at least one neural network has preferably been previously trained on images other than spectral images, including natural images of animals, objects, plants, people and / or "virtual images".
• the neural network may include one or more convolutional layers and / or one or more fully connected layers.
• the device according to the invention may comprise a classifier independent of the neural network to perform the classification.
• the device may comprise a thermal excitation means of the sample to be characterized, in particular a surface excitation device by illumination, preferably a surface excitation device by pulsed illumination such as a flash lamp.
• the device may further comprise a decision-making module communicating with the analysis module, and a means of action able to act on the sample, said decision-making module being able to slave said action means retroactively. according to the classification results obtained from said analysis module and to trigger an appropriate action towards the sample. This allows for sample control and reliable tracking, for example in non-destructive testing applications.
• the means of action may be a spraying nozzle of a phytosanitary product on crops, capable of spraying a quantity of product suitable for the classification results.
• the characteristics to be extracted can be the leaf mass, the type of vegetation, or even the type of disease of the plant examined.
• the characterization device according to the invention can be easily implemented on an embedded system either at low cost, for example a nano-PC for example of the Raspberry PI type, or specific, for example a TX1 type dedicated card of Nvidia®, according to the intended application. Most of the processing can take place in the intelligent embedded system, for example a nano-PC that can be equipped with a high-definition camera.
• Another subject of the invention is a method for controlling a sample, comprising the step of generating with the device of characterization of a sample as defined above, according to the classification results, information relating to the sample with a view to making a decision to decide on an action to be taken towards the sample to be characterized, and in particular to transmit an instruction of action to a means of action able to implement it.
• the invention further relates, in another of its aspects, to a computer program product for implementing the method for characterizing a sample as defined above, using a set of spectral images of the sample. to characterize previously acquired, in particular by infrared thermography or spectral imaging, and at least one neuron network,
• the computer program product having a medium and recorded on that medium readable instructions by a processor for when executed:
• At least one volume of values (D (Nx, Ny, Ne)) of a parameter observed from said spectral images is generated, for a plurality of coordinates (Nx, Ny) of the pixels of the images and a plurality of acquisitions (Born),
• At least one set of input data (D'x, y (Ne)) is extracted from said data volume (D (Nx, Ny, Ne)), these input data corresponding to the values of the observed parameter for a pixel of the same coordinates (x, y) according to different acquisitions (Ne), to which at least one transformation function has been applied,
• said at least one neural network is driven using the input data to extract at least one characteristic from the sample to be characterized
• said at least one characteristic extracted by the neural network is used to classify the input data according to a plurality of classes, each class being representative of at least one characteristic of the sample to be characterized.
• FIG. 1 previously described, illustrates a classification method using a deep learning algorithm according to the prior art
• FIG. 2 represents an example of data preparation steps in the method according to the invention, with a view to providing them to at least one neuron network
• FIG. 3 illustrates data classification steps by a neural network in the method according to the invention
• FIG. 4 represents an exemplary device for characterizing a sample according to the invention.
• FIG. 11 represents the steps of obtaining virtual spectral images for driving the neural network
• FIG. 2 shows an example of data preparation steps in the method according to the invention, with a view to providing them to at least one neuron network.
• spectral images are acquired by a thermal camera, thus forming so-called thermal images.
• the parameter observed from these images is the temperature of the surface of the sample E.
• a set of thermal images 2 is acquired over a predefined acquisition period T, using for example an infrared thermal camera.
• a data volume D (Nx, Ny, Ne) corresponding to the instantaneous temperature values is generated from the thermal images 2, in a frame where Nx and Ny correspond to the coordinates of the pixels N of the images 2 in the directions (x, y) and Ne corresponds to the acquisition, expressed either in number of the image or in time, or wavelength in the case of multispectral or hyperspectral imaging.
• a set of one-dimensional input data D'x, y (Ne) is extracted from the data volume D (Nx, Ny, Ne), during a step EP.
• these input data D'x, y (Ne) correspond to the instantaneous temperature values, for a pixel of the same coordinates (x, y) according to different recording instants Ne, to which at least one function transformation is applied, detailed in the following.
• IJx images representing curves corresponding to the values D'x, y of the input data set as a function of the recording time Ne are generated for each data set, and are transmitted to a network of neurons R (CNN) convolution in the example.
• the neural network used in the method according to the invention can nevertheless be of any type.
• the method according to the invention further comprises a data pretreatment step PT following the extraction step EP.
• this data preparation step PT consists of applying at least one transformation function to the values Dx, y (Ne), in particular a centering, normalization and / or smoothing function.
• different functions can be applied to the input data set, for example the identity function forming the set J0 corresponding to the original data set, preprocessed or not, and / or a calculation function of the first derivative forming the game J1, and / or a calculation function of the second derivative forming the game J2.
• the sample E to be inspected is a 370x500mm steel metal plate with a layer of paint coating deposited in 4 strips. whose thickness varies from 59 to 95 ⁇ m, as can be seen in Figure 2.
• This sample was placed horizontally against an insulating support to prevent conduction phenomena between the sample and the soil.
• a total surface thermal excitation of the sample can be performed in order to have a high rate of heating of the entire surface.
• the technique of pulsed infrared thermography is used: a thermal wave is sent to the surface of the sample whose excitation profile is as close as possible to a Dirac pulse.
• Halogen lamps can be used, but long-term illumination-induced temperature rise can damage the surface of the sample.
• several flash lamps generating a large amount of energy in a very short time, positioned at different angles, are used.
• the thermal camera records every 5 milliseconds a thermal image, or thermogram, of the front face of the surface of the sample. Following the acquisition of these thermal images, a volume of data is generated as described above. The acquisition period is between ... 0.5 seconds and 2 seconds, being for example equal to 1 seconds.
• each pixel of the data volume recorded by the camera is a one-dimensional signal which is represented by an IJx image used as an input of an RNCNN neural network.
• This RNCNN neuron network has preferably been previously trained on images other than natural spectral images, for example from the "www.image.net" database containing more than 1000 image classes and more than a million images.
• This neural network may belong to an analysis model MP further comprising a perceptron P, comprising for example an input layer, one or more hidden layers and an output layer.
• This analysis model MP is initially able to classify images by the neuron network RN then the perceptron P according to classes K (K1, K2 ).
• the neural network is driven using the input data IJx to extract EF characteristics, corresponding to different thicknesses of paint in the example considered.
• the neural network belongs to an MP analysis model comprising a perceptron P
• the extraction of the characteristics can be carried out by at least one of the layers of the perceptron.
• the extracted characteristics are used to classify the thermal responses according to a plurality of classes C1, C2, each class being representative of a characteristic of the sample to be characterized, here different thicknesses.
• the classification is, in this example and preferably carried out by a classifier independent of the neural network, of the wide margin separator type (SVM).
• SVM wide margin separator type
• each of the paint coating thicknesses visible in FIG. 2, is associated a class C1, C2, C3, C4 of different spatial dimensions. Acquisitions on this coating were made at two different times Tl and T2, creating two volumes of data, the first of dimensions 110x611x400, and the second of dimensions 103x631x400.
• Tl and T2 Two times Tl and T2
• the data volumes do not have the same dimensions, except the temporal dimension since the acquisition period is always the same.
• one-dimensional data sets J0, J1 and J2 have been generated from these volumes.
• Figure 5 shows the classification performance for the J0 dataset from the second volume, including a set of 8000 signals that were randomly selected (2000 for each class). 70% of the data were randomly selected for the training, corresponding to the "training data set", and the remaining 30% for the classification test, corresponding to the "test data set”. It is observed on the diagonal that 97% of class 1, 95% of class 2, 92% of class 3 and 91% of class 4 measurements are well classified and the average accuracy is 93.5%.
• Figure 6 shows the classification performance for dataset Jl. It is seen by looking at the diagonal that 96% of class 1 measurements, 91% of class 2, 91% of class 1 3 and 87% of those in class 4 are well ranked and the average accuracy is 91.25%.
• Figure 7 shows the classification performance for the J2 dataset. It is seen by looking diagonally that 94% of class 1, 86% of class 2, 85% of class 3 and 89% of class 4 are well ranked and the average accuracy is good. is therefore 88.1%.
• the comparison of the classification results on the 3 sets J0, J1 and J2 shows that the best results are those obtained by taking the normalized and smoothed J0 input data.
• the method according to the invention makes it possible to recognize the thermal response of each pixel and to associate it with the right class, in order to reliably find, in the example described, the thickness of each coating strip of the sample.
• the table in FIG. 8 shows the independence of the classification results with respect to the energy deposit, ie the verification that a variation in the homogeneity of the energy deposit on the sample has a very small impact on the classification results of each pixel in the data volume. This allows the realization of experimental measurements in which the parameter of uniformity of the deposit of energy on the surface is little annoying.
• the table in Figure 9 was produced by applying the classifier to 4000 signals different from those used for learning and extracting characteristics by the RNCNN neuron network, but acquired at the same time period T2, for which each class is formed in the same way as when learning, that is to say 1000 signals with 4 flashes, 1000 signals with 3 flashes, 1000 signals with 2 flashes, and 1000 signals with 1 flash.
• 93.075% of the Class 1 measurements, 97.65% of the Class 2 measurements, 97.8% of the Class 3 measurements, and 94.8% of the Class 4 measurements are properly classified.
• the table in Figure 10 shows the independence of the classification results with respect to the measurement conditions.
• the realization of two strictly identical measurements spaced over time on the same sample is indeed almost impossible to obtain.
• the positions of the lamps, the distance from the camera to the sample, the angles of inclination and the rotation of the camera relative to the camera are variable parameters that may affect the classification results.
• the learning data set, acquired at a time period T2 is identical to that used to produce the table of FIG. 9, the classification performance being evaluated by applying the algorithm to 4000 signals.
• different from the signals used for learning, and acquired at a different time period T1, and for which each class is constituted in the same way as during the learning that is to say 1000 signals with 4 flashes, 1000 signals with 3 flashes, 1000 signals with 2 flashes, and 1000 signals with 1 flash.
• FIG. 11 represents the steps of obtaining virtual images intended for driving the neural network.
• the parameter observed is the temperature T of the sample.
• a virtual sample 3 may for example be in the form of a stack of materials, namely a substrate 4 (possibly multilayered) covered with a coating 5 (possibly multilayer) discretized in voxels V (x, y), of position (x, y) in the XY plane of sample 3.
• Each voxel V (x, y) of this example, visualized along the Z axis includes all of the layer (s) F zi provided for said coating deposited in the plane XY.
• the layers F zi here three in number (F zi , F z2 , F z3 ), are represented only at the level of the gray voxel.
• T (x, y, t) F [QE (x, y, t); PS; PR; ER; t; ...] (EQM)
• the mathematical model output the virtual spectral images ISV X which result from the convolution of the curve LT with a curve of the "impulse response" type LRI, such as a "gate function P", and which are intended for the RN neuron network (CNN) training.
• This device 20 is, in the example considered, a non-destructive control drone by spectral imaging.
• the characterization device 20 comprises an acquisition means AQ of a set of spectral images of the sample E, in particular a thermal or spectral camera, scanning all the spectral ranges, for example the infrared, the visible, the near infrared, the middle infrared, the far infrared corresponding to the so-called thermal or thermographic infrared, and any combination of these ranges, especially visible and near infrared.
• the characterization device 20 also comprises a data processing module MAM capable of generating at least one volume of instantaneous data values D (Nx, Ny, Ne), and extracting at least one set of input data D'x , y (Ne) from said data volume D (Nx, Ny, Ne), as previously described.
• the characterization device 20 further comprises an analysis module MP, comprising at least one RNCNN neuron network making it possible to extract characteristics from the input data, the analysis module MP being able to using the extracted characteristics to classify the input data according to a plurality of classes.
• an analysis module MP comprising at least one RNCNN neuron network making it possible to extract characteristics from the input data, the analysis module MP being able to using the extracted characteristics to classify the input data according to a plurality of classes.
• the characterization device 20 may comprise a thermal excitation means EX of the sample E to be characterized.
• the characterization device 20 may furthermore comprise a decision-making module MD communicating with the analysis module MP and an action means ACT able to act on the sample E.
• the decision-making module MP is advantageously suitable. to enslave said ACT action means retroactively according to the classification results obtained and to trigger an appropriate action towards the sample E.
• each spray nozzle corresponding to the ACT action means, can be associated with an embedded hybrid network according to the invention, for example, on a nano-PC using a low cost camera for example in the field. Visible and / or near-infrared that can detect leaf mass, vegetation type, or even the type of disease of the plant being examined, so as to spray the good phytosanitary product with the amount just needed. Multispectral or hyperspectral imaging can then be used to acquire the images.
• the plants can be corn, wheat, or vine. Plant growth can also be monitored by the invention and the possible occurrence of a disease can be predicted.
• the invention is not limited to a convolutional neural network.
• Networks such as “Deep Neural Network” (DNN) or “Deep Belief Network” (DBN) type can be envisaged.
• DNN Deep Neural Network
• DBN Deep Belief Network
• the processing of several groups of data in parallel can be performed by parallel networks.
• the invention can be implemented on any type of hardware, for example a personal computer, a smart phone, a nano-computer, or a dedicated card.
• the invention is not limited to applications for the characterization of infrared thermography coatings. Radiation in the visible, near-infrared, mid-infrared, far-infrared, Terahertz or ultraviolet domains could be used.
• the invention is particularly suitable for non-destructive testing applications, in order to preserve the quality of the samples tested.
• the invention can be used in various applications, for example in low-cost smart on-board sensors, or in "fog computing" decentralized infrastructures, in which the objective is to improve efficiency and reduce the amount of transferred data.
• the invention can be used in many other fields, such as the military, electrical monitoring, geology, or biology or bioinformatics, particularly to follow manufacturing processes and the quality of materials.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Computing Systems (AREA)
• Mathematical Physics (AREA)
• Biophysics (AREA)
• Computational Linguistics (AREA)
• Data Mining & Analysis (AREA)
• Evolutionary Computation (AREA)
• General Health & Medical Sciences (AREA)
• Molecular Biology (AREA)
• Artificial Intelligence (AREA)
• General Engineering & Computer Science (AREA)
• Life Sciences & Earth Sciences (AREA)
• Biomedical Technology (AREA)
• Software Systems (AREA)
• Health & Medical Sciences (AREA)
• Spectroscopy & Molecular Physics (AREA)
• Quality & Reliability (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Radiation Pyrometers (AREA)
• Investigating Or Analyzing Materials Using Thermal Means (AREA)
• Image Analysis (AREA)
• Image Processing (AREA)
• Investigating Or Analysing Materials By Optical Means (AREA)
• Character Discrimination (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=62067607

Family Applications (1)

Country Status (7)

Families Citing this family (16)

Family Cites Families (12)

• 2017
  • 2017-12-01 FR FR1761522A patent/FR3074596B1/fr active Active
• 2018
  • 2018-11-30 CA CA3083079A patent/CA3083079A1/fr active Pending
  • 2018-11-30 EP EP18814839.9A patent/EP3717877A2/fr active Pending
  • 2018-11-30 CN CN201880077916.3A patent/CN111630356B/zh active Active
  • 2018-11-30 JP JP2020547304A patent/JP2021504864A/ja active Pending
  • 2018-11-30 US US16/768,573 patent/US11828652B2/en active Active
  • 2018-11-30 WO PCT/EP2018/083210 patent/WO2019106179A2/fr unknown

Also Published As

Similar Documents

Legal Events