WO2025232193A1

WO2025232193A1 - Methods and systems for evaluating properties of coatings

Info

Publication number: WO2025232193A1
Application number: PCT/CN2024/138520
Authority: WO
Inventors: Lu Shen; Long Wang; Se JIN; Yang Lu
Original assignee: BASF China Co Ltd; BASF Coatings GmbH
Current assignee: BASF China Co Ltd; BASF Coatings GmbH
Priority date: 2024-05-08
Filing date: 2024-12-11
Publication date: 2025-11-13
Anticipated expiration: 2026-11-08
Also published as: CN121359164A

Abstract

The invention relates to the field of coating performance testing, in particular to the field of automated evaluation of coating properties. The disclosure relates to a method, a system and a respective computer element for determining at least one property of a coating being present on a substrate. The disclosure further relates to the use of a property of a coating being present on a substrate and as determined according to the methods, systems and computer elements to adjust a formulation of a coating material related to the coating.

Description

METHODS AND SYSTEMS FOR EVALUATING PROPERTIES OF COATINGS

TECHNICAL FIELD

TECHNICAL BACKGROUND

Coating properties play a crucial role in determining the performance and durability of various industrial and consumer products. Coating properties are determined by subjecting the coated substrate to one or more stress tests, such as adhesion tests, stone chipping tests, corrosion tests or the like. Traditional methods for evaluating coating properties after subjecting the coated substrate to such tests rely on visual inspection of the coated substrates obtained after performing one or more of such tests on the coated substrate by trained inspectors. However, these methods have several limitations, including being time-consuming, subjective, and prone to human error. As a result, there is a need for a more objective and reliable method for evaluating coating properties, which can improve the accuracy and consistency of coating quality control and accelerate the development of new coating materials while reducing the environmental impact associated with the coating material development.

SUMMARY OF THE INVENTION

Disclosed is in an aspect a method for determining at least one property of a coating of a coated substrate, wherein the coated substrate comprises one or more scribe (s) and wherein the coated substrate has been subjected to at least one chemical and/or mechanical stress test, the method comprising:

· providing image data associated with the coated substrate;

· determining region of interest (ROI) data being indicative of the scribe (s) and optionally damaged area (s) of the coating and/or substrate based on the provided image data;

· processing the region of interest data using a data-driven model parameterized and/or trained to generate damage data associated with damaged area (s) present on the coating and/or the substrate and scribe data associated with the scribe (s) in response to being provided by the region of interest data;

· determining property data associated with the at least one property of the coating based on the damage data and the scribe data generated by the data-driven model;

· providing the property data associated with the at least one property of the coating.

In another aspect disclosed is a system for determining at least one property of a coating of a coated substrate, wherein the coated substrate comprises one or more scribe (s) and wherein the coated substrate has been subjected to at least one chemical and/or mechanical stress test, the system comprising:

· an image acquisition unit configured to generate image data associated with the coated substrate;

· a data providing interface configured to provide the generated image data,

· a region of interest data generator configured to determine region of interest (ROI) data being indicative of the scribe (s) and optionally damaged area (s) of the coating and/or substrate based on the provided image data;

· a neural network engine configured to process the region of interest data and including a data-driven model parameterized and/or trained to generate damage data associated with damaged area (s) present on the coating and/or the substrate and scribe data associated with the scribe (s) in response to being provided by the region of interest data;

· a property data determination unit configured to determine property data associated with the at least one property of the coating based on the damage data and scribe data generated by the data-driven model,

· a data providing interface configured to provide the property data associated with the at least one property of the coating.

In yet another aspect disclosed is a computer readable medium having a computer program stored thereon, the computer program including instructions which, when executed by one or more processors, causes the processors to carry out the methods disclosed herein.

In yet another aspect disclosed is a use of property data associated with a coating and being determined by the methods disclosed herein or by the systems disclosed herein for adjusting a formulation of a coating material related to the coating.

Any disclosure, embodiments and examples described herein relate to the methods, the apparatuses, systems, the uses and computer elements lined out above and below. Advantageously, the benefits provided by any of the embodiments and examples equally apply to all other embodiments and examples.

Embodiments

In the following, embodiments of the present disclosure will be outlined by ways of embodiments and/or examples. It is to be understood that the present disclosure is not limited to said embodiments and/or examples.

In the field of coating properties evaluation, visual evaluation has been the traditional method used for decades. However, this approach has several limitations and drawbacks. Visual evaluation is prone to human error and subjective interpretation, leading to inconsistent results and high variability. Moreover, it requires a significant amount of time and resources, and often fails to detect subtle defects or variations in the coating properties.

By determining the scribes and damaged area (s) present on the coating or present on the coating and the substrate using a trained data-driven model based on image data of such coated substrate, the damage of the coating and/or the substrate resulting from the chemical and/or physical stress test the coated substrate was subjected to can be reliably, accurately and efficiently determined. Unlike manual methods, which are be time-consuming and prone to human error, the trained data-driven model can quickly and accurately identify the scribe (s) and damaged area (s) in images of coated substrates. This results in a more reliable and robust assessment of coating properties, which is critical in industries where the quality of coatings can impact the performance and safety of coated products.

By using a modular neural-network architecture comprising a backbone network trained to generate feature maps from input data, a semantic segmentation module trained to determine the damaged area (s) of the coating or of the coating and the substrate from the feature maps and an object detection module trained to detect the scribe (s) from the feature maps, the accuracy and reliability of scribe detection and damaged area detection can be improved without having to employ separate neural networks. This reduces the computational effort and hence the energy consumption required for training the data-driven model and allows to flexibly adapt the neural network architecture to different chemical and/or physical stress test conditions and specifications. This way, a wide range of coating properties can be reliably and accurately determined while reducing the environmental impact associated with the training process.

By using augmentation techniques to increase the amount of training data from a given set of image data associated with coated substrates having undergone one or more chemical and/or physical stress test (s) , the data-driven model can be trained on a larger and more diverse dataset, which can improve its ability to generalize and accurately detect scribes and damaged areas under changing conditions for generating image data and/or for a wide variety of different coatings. This avoids generating multiple coated substrates and performing chemical and/or physical stress tests on such coated substrates to generate sufficient training data. This way, the environmental impact associated with the training of the data-driven model can be reduced by avoiding the consumption of material and energy to prepare the coated substrates and perform the test as well as the generation of waste material (e.g. coated substrates) .

By processing the scribe data prior to determining the property data, a more accurate and reliable determination of the property data can be achieved. Processing of the scribe data allows to remove duplicate detection of scribe (s) as well as inaccurate detection of scribe (s) by the data-driven model. In addition, processing of the scribe data allows to add missing scribe (s) by interpolation. This way, the damaged area (s) can be determined more accurately during determination of the property data, resulting in an improved reliability and accuracy of the determined property data.

Various units, entities, nodes or other computing components may be described as “configured to” perform a task or tasks. Configured to shall recite structure meaning “having circuitry that” performs the task or tasks on operation. The units, circuits, entities, nodes or other computing components can be configured to perform the task even when the unit/circuit/component is not operating. The units, circuits, entities, nodes or other computing components that form the structure corresponding to “configured to” may include hardware circuits and/or memory storing program instructions executable to implement the operation. The units, circuits, entities, nodes or other computing components may be described as performing a task or tasks, for convenience in the description. Such descriptions shall be interpreted as including the phrase “configured to. ”

In general, the methods, apparatuses, systems, computer elements, nodes or other computing components described herein may include memory, software components and hardware components. The memory can include volatile memory such as static or dynamic random-access memory and/or nonvolatile memory such as optical or magnetic disk storage, flash memory, programmable read-only memories, etc. The hardware components may include any combination of combinatoric logic circuitry, clocked storage devices such as flops, registers, latches, etc., finite state machines, memory such as static random-access memory or embedded dynamic random-access memory, custom designed circuitry, programmable logic arrays, etc.

The coated substrate may comprise a coating and a substrate. The coating may be a solid material present on the surface of the substrate. The coating may at least partially cover the surface of the substrate. The coating may consist of a single coating layer. The coating may comprise at least two coating layers, e.g. may be a multilayer coating. The coating layers may be the same or may be different coating layer (s) , e.g. may be coating layers produced using different coating materials. The coating may be produced by applying at least one coating material to the surface of the substrate and drying and/or curing the applied coating material (s) . The applied coating material (s) may be cured separately. At least a part of the applied coating material (s) may be cured jointly. This way, energy may be saved, reducing the environmental impact of the production of the coated substrate. The coating material may be a pigmented coating material including at least one color and/or effect pigment. The coating material may be a transparent coating material not including any color and/or effect pigments. The pigmented coating material may result in opaque coating layers or in semitransparent coating layers, depending on the amount of pigments present within the coating material and the film thickness of the coating layer. The coating material may be applied onto the surface using spray coating, dip coating, roll coating or bar coating.

The substrate may refer to the material underlying the coating, e.g. the material being in direct contact with the coating. The substrate may be a discrete component, part or article. The substrate may have a 2D shape or a 3D shape. The substrate may have defined dimensions. The substrate may be a metallic substrate, a plastic substrate or a substrate containing plastic and metallic parts. The substrate may comprise a pretreated surface onto which the coating is produced. This way, a standardized underground for coatings produced from different coating materials may be provided, allowing to compare different coatings with respect to their property/ies. The pretreated surface may include a single layer coating or a multilayer coating. The pretreated surface may include a conversion coating layer, an electrocoating layer and optionally a primer layer. The pretreated surface may include a primer layer.

The property associated with the coating may signify or indicate the resistance or stability of the coating to external environmental influences. External environmental influences may include mechanical forces and/or environmental conditions such as sunlight and/or humidity and/or corrosive substances. The property may include adhesion, such as wet adhesion, steam jet adhesion and/or dry adhesion, corrosion resistance and/or chemical resistance. The property may be determined by subjecting the coating to a chemical and/or physical stress test. Subjecting the coating to a chemical and/or physical stress test may include intentionally damaging the coating and/or the substrate prior to performing such stress test. The coating and/or the substrate may be damaged by introducing scribe (s) or scratch (es) into the coating or through the coating into the substrate. The scribe (s) may be introduced using one or more blades. The scribe may be a single scribe. The scribes may form a pattern, such as a regular or irregular pattern. The stress test may be performed according to a defined technical specification. The technical specifications may be generated and provided by standard setting organizations and/or by consumers of coating material (s) used to produce the coating, such as end-product producers.

The region of interest (also denoted as ROI hereinafter) may indicate a defined part of the image data that includes data being indicative of the property of the coating. The ROI may be identifiable by the presence of one or more scribe (s) and/or boundaries present around the scribe (s) . The ROI may be associated with region of interest data (also denotes as ROI data hereinafter) . The region of interest data may be a subset of the image data encoding such data being indicative of the property of the coating. The ROI data may comprise key features, structures, or attributes associated with the property of the coating. The ROI data may serve as a focal area of the image data for generating damage data and scribe data using the data-driven model.

The data-driven model may include at least one neural network architecture with at least one input layer, one or more hidden layers and at least one output layer. The data-driven model may be based on sequential and/or parallel neural network (s) . One or more neural network (s) may be connected to be performed sequentially and/or parallel. At least one layer of the data-driven model may include an input layer or channel, one or more hidden layers or channels and an output layer or channel. The data-driven model may include parameters, such as kernels, weights, biases, functional relationships, operators, constraints or the like, which are trained based on training data set (s) . The data-driven model trained to generate the damage data and the scribe data may receive region of interest data at the input layer and may generate the damage data and the scribe data in response to receiving such region of interest data. The data driven-model may be trained to generate the damage data and the scribe data from the ROI data. The data-driven model may be parameterized according to a training data set including labelled ROI data. The label (s) may relate to damaged area (s) of the coating and/or the substrate and scribe (s) present within the coating or throughout the coating and in the substrate. The data-driven model may be connected to one or more algorithms or routines. The one or more algorithms and/or routines may allow to determine the property data based on the damage data and the scribe data generated by the data-driven model.

A formulation of a coating material related to the coating may refer to at least one formulation of the coating material used to prepare the coating. The formulation may be associated with formulation data. The formulation data may include a coating material identifier associated with the coating material, input material data associated with input materials and amounts of such input materials required to produce the respective coating material. The formulation data may be used to prepare the coating material the formulation data is associated with.

In an embodiment the coating comprises at least one coating layer. The coating may be a single layer coating consisting of exactly one coating layer. The coating may be a multilayer coating comprising at least two coating layers. At least a part of the coating layers may be different from each other, e.g. may be prepared using different coating materials. The coating layer may be prepared by applying a coating material onto the substrate or onto a coating layer already been present on the substrate.

In an embodiment the one or more scribe (s) are oriented in a defined pattern on the coated substrate. The defined pattern may be a random pattern or a regular pattern. The defined pattern may include one scribe or a plurality of scribes. The defined pattern may be a cross. The defined pattern may be a grid. The grid may contain a at least two horizontal and at least two vertical scribes forming the grid. The horizontal and vertical scribes may intersect at defined angles, such as angles of 90 degree.

In an embodiment the chemical and/or mechanical stress test includes an adhesion test, a salt spray test or a combination thereof. The adhesion test may include a crosscut adhesion test, a steam jet adhesion test or a wet adhesion test. The adhesion test may allow to determine the adhesion of the coating to the substrate and/or the adhesion of coating layer (s) present within the coating to underlying and/or overlying coating layers and/or the substrate. The salt spray test may allow to determine the corrosion resistance of the coating and/or the substrate.

In an embodiment the one or more scribe (s) are obtained by introducing the scribe (s) at one or more defined location (s) on the coated substrate into the coating and/or by introducing the scribe (s) at one or more defined location (s) on the coated substrate through the coating to the underlying substrate. The scribe (s) may be introduced using a cutting edge or a cross-cutting tool. The cross cutting tool may include a plurality of cutting edges spaced apart by a defined distance. This allows to introduce scribe (s) having a defined distance from each other.

In an embodiment providing the image data includes acquiring image data of the coated substrate and providing the acquired image data. The image data of the coated substrate may be acquired using a camera configured to acquire image data of the coated substrate. The camera may be an industrial camera, such as an infrared camera, a color camera, a 3D camera, or any other camera configured to generate image data of the coated substrate. The image data may be acquired such that it contains data being indicative of the scribe (s) and optionally damaged area (s) of the coating and/or substrate based on the provided image data. The image data may be acquired under defined lightning conditions. This may allow to obtain a higher contrast between the scribe (s) , the damaged area (s) and the undamaged area (s) present on the coating, hence improving the accuracy and reliability of the determination of the damage data and the scribe data.

In an embodiment the image data includes data related to the one or more scribe (s) present in the coating and/or the substrate and optionally data related to damaged area (s) of the coating and/or the substrate. The image data may further include data related to non-damaged area (s) of the coating and/or the substrate. The image data associated with the coating may be indicative of damaged area (s) present on the coating and/or the underlying substrate and one or more scribe (s) present within the coating and/or the substrate.

In an embodiment determining the region of interest (ROI) data includes determining at least one region of interest (ROI) in the provided image data and extracting the determined at least one region of interest (ROI) as region of interest data from the provided image data. The ROI data may be a subset of the image data. In case multiple physical and/or chemical stress tests have been performed on the coated substrate, two or more ROIs may be determined and region of interest data may be generated per determined ROI. The region of interest data may indicate at least one region of interest (ROI) in the image data associated with the coated substrate. The ROI may indicate a particular portion of the image data that includes data to be processed by the data-driven model to generate the damage data and the scribe data. The ROI may correspond to a square region of 2.5 cm × 2.5 cm of the coated substrate. The ROI may include one or more of the scribe (s) . The ROI may further include damaged area (s) and/or undamaged area (s) . The ROI may include a grid formed by the scribes. The ROI may have other sizes and/or shapes. The ROI may be identified by its position, size or form. The ROI data may be generated in response to receiving a user input indicating a region of interest (ROI) in the image data. The region of interest data may be generated using a data-driven model trained to generate ROI data in response to receiving image data. Use of the ROI data allows to focus on a particular portion of the image data when generating the damage data and the scribe data, allowing to improve the efficiency of generating such data and improving the accuracy and reliability of the generated damage data and scribe data.

In an embodiment the data-driven model includes

· a feature extractor trained to generate feature maps from the regions of interest data and to provide the extracted feature maps to a semantic segmentation module and an object recognition module,

· the semantic segmentation module trained to generate damage data encoding the spatial layout of damaged area (s) present in the region of interest based on the received feature maps,

· the object detection module trained to generate scribe data encoding the spatial layout of the scribe (s) present in the region of interest based on the received feature maps.

The feature extractor may include a neural network trained to generate feature maps from the provided region of interest data. The feature maps may include 2D or 3D matrices containing features extracted from the ROI data. The extracted features may include edges, corners, colors and/or textures. Each element in the feature map may correspond to a function of a local neighborhood (patch) in the ROI data. This way, the feature maps map the local features of the ROI data. The feature maps may include feature maps having a pyramidal hierarchy with respect to their resolution. The feature extractor may be convolutional neural network (CNN) . The feature extractor may be based on a Feature Pyramid Network (FPN) defined over a ResNet architecture. The use of a combination of the ResNet and the FPN allows to supplement features extracted from higher layers of the ResNet with features from lower layers of the ResNet to obtain feature maps having a high resolution as well as strong semantics. This way, small objects present in the ROI data may be detected with higher accuracy since such objects may only be present in some of the feature maps generated by the ResNet.

The semantic segmentation module may include a neural network trained to generate damage data encoding the spatial layout of damaged area (s) present in the region of interest based on the feature maps generated by the feature extractor. The semantic segmentation module may be trained to merge the features included in the feature maps received from the feature extractor into a single output (e.g. the damage data) . The neural network may be trained to increase the spatial resolution of the feature maps generated by the feature extractor, to pool the feature maps having increased spatial resolution and to transform the pooled feature maps into the damage data.

The object detection module may include a neural network trained to generate scribe data encoding the spatial layout of the scribe (s) present in the region of interest based on the feature maps generated by the feature extracting module. The scribe data may include position coordinates of points lying on the scribe (s) and/or coordinates of the vertexes or intersections of scribe (s) , for example vertexes or intersections of a grid formed by the scribes and/or scribe area (s) formed by the scribe (s) . The neura l network may be trained to generate bounding box (es) around the scribe (s) based on the feature maps provided by the feature extractor and to generate scribe data including the position data.

The object detection module may include a region proposal network (RPN) , a region of interest (ROI) Align layer, a bounding box head and a classification head. A head may refer to final layers of the object recognition module which take the features extracted by previous layers and produce an output, such as coordinates and/or a category. The RPN may produce region proposals or candidate bounding boxes that may include objects, such as scribe (s) , present in the ROI data encoded by the feature maps. It may operate on the feature maps and may provide prospective regions of interest. The region proposals may include an objectness score. The objectness score may indicate the resemblance of the region proposal to a real object (e.g. scribe (s) , damaged area (s) , undamaged area (s) ) encoded by the ROI data. A high objectness score implies a likely presence of an object of interest within the proposed region, whereas a low score suggests that the region is probably background or doesn't contain any relevant object. The ROI Align layer may align the features present within a proposal with the spatial grid of the output feature map. The output generated by the ROI Align layer may be provided to different heads. One head may include a bounding-box head configured to generate bounding boxes using the output of ROI Align layer around the scribe (s) . The bounding boxes may encode the position data of the scribe (s) . The classification layer may be configured to generate classifications, such as scribe (s) and no scribe (s) using the output of the ROI Align layer.

The neural network (s) included in the feature extractor, semantic segmentation module and object recognition module may be connected to each other to allow transfer of data, such as feature maps from the neural network (s) of the feature extractor to one or more other neural network (s) , such as neural network (s) of the semantic segmentation module and the object recognition module.

The feature maps generated by the feature extractor may be provided to the semantic segmentation module and the object detection module simultaneously. This may allow to determine the damage data and the scribe data simultaneously, hence reducing the overall time required to determine the property data. This way, property data for a large amount of coated substrates may be determined, hence increasing the efficiency of the method. This may reduce consumption of coating material (s) used to produce coated object (s) since deviations of coating property/ies from target property/ies can be determined faster, hence allowing faster adaption of the coating material formulation and avoiding the generation of waste coating material comprising property/ies not matching given target property/ies. This way, the environmental impact of coating material development can be reduced.

In an embodiment the damage data includes a binary matrix indicating the presence of the damaged area (s) and the absence of the damaged area (s) . The binary matrix may include only values of 1 and 0. The value of 1 may indicate the presence of a damaged area while the value of 0 may indicate the absence of a damaged area, e.g. may indicate an undamaged area. The binary matrix may correspond to an array of values of 1 or 0. Each value of 0 and 1 may indicate whether the associated pixel belongs to a damaged area or an undamaged area. The binary matrix may be used to generate image data indicating the damaged area (s) and the non-damaged areas. The non-damaged area (s) may be masked out since these areas are associated with pixel values of 0.

In an embodiment the scribe data includes position data encoding coordinates of one or more points located on the scribe (s) 108, coordinates of the vertexes or intersections of the grid formed by the scribes 108 and/or scribe area (s) formed by the scribe (s) .

In an embodiment determining the property data includes

· generating merged data by combining the damage data with the scribe data,

· determining the amount of damaged area and the amount of non-damaged area based on the merged data and

· determining property data associated with the at least one property based on the determined amount of damaged area and the determined amount of non-damaged area or based on the determined amount of damaged area and predefined evaluation data.

The merged data may include or encode the spatial layout of the damaged area (s) present on the coating or present on the coating and the substrate and the spatial layout of the scribe (s) present within the coating or present within the coating and the substrate. The merged data may further include the region of interest data. The merged data may correspond to a matrix containing the features of the binary matrix included in the damage data and position data included in the scribe data. Generating merged data may include merging the binary matrix included in the damage data with the position data included in the scribe data.

The amount of damaged area (s) may be determined by dividing the number of pixels encoding damaged area (s) by the total number of pixels. The total number of pixels may correspond to the total number of pixels of the merged data, for instance the total number of pixels present within an image generated from the merged data.

Determining the property data may include providing predefined evaluation data. The predefined evaluation data include the amount of damaged areas and associated coating property classifications. The property data may be determined by mapping or matching the determined amount of damaged area with amounts of damaged areas included in the predefined evaluation data. Matching or mapping the determined amount of damaged area with the amount of damaged areas included in the predefined evaluation data may include determining the physical and/or chemical stress test the merged data is associated with and selecting an amount of damaged areas associated with such physical and/or chemical stress test from the predefined evaluation data for mapping or matching.

The property data may be determined by comparing the amount of damaged area to the amount of non-damaged area. The property data may hence indicate a fraction of the damaged area with respect to the total area of the ROI (e.g. damaged and non-damage area) indicated by the ROI data.

In an embodiment the data-driven model is trained to generate the damage data and the scribe data using at least one training data set including region of interest data, labelled region of interest data and/or labelled randomized region of interest data. The data-driven model may be pre-trained on a training data set including images of a generic domain, e.g. images not being related to coated substrates, and may be fine-tuned based on a training data set including images of a specific domain, e.g. images of scratched coated substrates having been subjected to chemical and/or physical stress test (s) . The images of the specific domain may include the ROI data, the labelled region of interest data and/or the labelled randomized region of interest data.

The labelled region of interest data may include labelled scribe (s) and labelled damaged area (s) . The labelling may be performed manually. Labelling the scribe (s) and the damaged area (s) may include drawing bounding boxes around the scribe (s) and the damaged area (s) and labelling the bounding box (es) with a respective label, such as scribe and damaged area. The labelled training data may further include acquisition data including lightning data and/or camera data. Lighting data may include lightning conditions used during image acquisition. Camera data may include camera settings used during acquisition, such as camera type, exposure time or the like. Use of the acquisition data during the fine-tuning of the data-driven model may result in a more reliable and accurate determination of damage data and scribe data based on input image data acquired under different lightning conditions and/or camera settings.

The training data set may be extended using image augmentation techniques to generate randomized training data from the provided image data or the ROI data. Image augmentation may allow to synthetically increase the size of the training data without having to generate numerous scratched coated substrates and subject such scratched coated substrates to chemical and/or physical stress tests. This avoids generation of waste and consumption of energy and material required for preparation of the scratched coated substrates and the testing. This way, the environmental impact associated with the training of the data-driven model may be reduced while improving the accuracy of the determined damage data and scribe data for varying coating colors, damage patterns, scribe patterns, damaged area patterns and image acquisition conditions (e.g. lightning conditions, resolution, exposure time of the camera etc. ) .

The labelled randomized region of interest data may be generated from the image data or the region of interest data by performing at least one transformation operation on the image data or region of interest data and labelling the scribe (s) and damaged area (s) in the randomized data. The at least one transformation operation may include horizontal and/or vertical flipping, random cropping, random scaling, random contrast setting and/or random intensity setting.

In an embodiment, the method further comprises a step of generating target scribe data based on the scribe data generated by the data-driven model, wherein the property data is determined based on the damage data generated by the data-driven model and the generated target scribe data. Generating the target scribe data may allow to condense the scribe data and to focus on the central region of the scribe (s) indicated by the scribe data. This way, a region of interest may be extracted from the scribe data. Generating target scribe data may allow to enhance or improve the scribe data generated by the data driven model. Enhancing or improving the scribe data may include removing duplicate scribe (s) and/or wrongly detected scribe (s) and/or interpolating undetected scribe (s) (e.g. missing scribe (s) ) . Improving or enhancing the scribe data may allow to improve the accuracy and reliability of the property data determined based on such enhanced scribe data.

Generating the target scribe data may include

· classifying the scribe (s) indicated by the scribe data into horizontal and vertical groups,

· filtering the scribe (s) indicated by the scribe data and/or determining missing scribes (s) by calculating line distance (s) between the horizontal and/or vertical groups

· optionally interpolating determined missing scribe (s) based on the calculated line distance (s) .

In an embodiment providing the generated property data may include providing the generated property data to a data storage. The generate property data may be provided to the data storage along with further data associated with the property data, such as a coating identifier, the generated damage data, the generated scribe data, the merged data, the target scribe data, the ROI data and/or the image data. Providing the generated property data may include providing the generated property data for display. The property data may be provided for display along with the further data.

In an embodiment of the system, the image acquisition means includes a sample container configured to hold the coated substrate (s) , a camera and a lightning means including one or more light sources. The camera may include an infrared camera, a color camera, and/or a 3D camera. The lightning means may be configured to illuminate the coated substrate (s) present within the sample container. The camera may be configured to acquire or generate image data of the coated substrate.

In an embodiment of the system, the system may further include a database configured to store the determined property data. The database may further be configured to store additional data associated with the property data, such as a coating identifier, the generated damage data, the generated scribe data, the merged data, the target scribe data, the ROI data and/or the image data.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following, the present disclosure is further described with reference to the enclosed figures. The same reference numbers in the drawings and this disclosure are intended to refer to the same or like elements, components, and/or parts. The description of drawings is provided for illustrative purposes and shall not be considered limiting. The embodiments and examples are illustrative embodiments and examples to further line out the concepts lined out herein. The figures include schematic illustrations and shall not be considered limiting. Other embodiments and examples that fall under the concepts lined out herein are possible and may not be explicitly described herein.

FIG. 1 illustrates a schematic method for preparing a coated substrate and subjecting the coated substrate to chemical and/or physical stress tests.

FIG. 2A illustrates an example of a system for determining at least one property of a coating of a coated substrate.

FIG. 2B illustrates another example of a system for determining at least one property of a coating of a coated substrate.

FIG. 3 illustrates an example implementation of a neural network engine for generating damage data associated with damaged area (s) present on the coating and/or the substrate and scribe data associated with scribe (s) present on the coating and/or the substrate.

FIG. 4 illustrates an example of output data generated by modules of the neural network engine illustrated in FIG. 3.

FIG. 5A illustrates an example architecture of a feature extractor and a semantic segmentation module included in the neural network engine illustrated in FIG. 3 and FIG. 4.

FIG. 5B illustrates an example architecture of the feature extractor illustrated in FIG. 5A.

FIG. 6 illustrates an example architecture of the object recognition module illustrated in FIG. 3 and FIG. 4.

FIG. 7 illustrates an example of a processing unit included in the system illustrated in FIG. 2A and FIG. 2B.

FIG. 8 illustrates a flow chart of an example of a method for determining at least one property of a coating of a coated substrate.

FIG. 9A illustrates an embodiment of the method illustrated in FIG. 8.

FIG. 9B illustrates another embodiment of the method illustrated in FIG. 8.

FIG. 10 illustrates a sequence diagram of an example method for determining at least one property of a coating of a coated substrate.

FIG. 11 illustrates a method for training the neural network engine illustrated in FIG. 2A to FIG. 6.

DETAILED DESCRIPTION

FIG. 1 illustrates a schematic method for preparing a coated substrate and subjecting the coated substrate to one or more chemical and/or physical stress test (s) . The coated substrate may include a substrate and a coating. The coating may cover at least a part of the surface of the substrate.

A substrate 102 may be provided. The substrate 102 may have a defined geometric form and/or defined dimensions. The substrate 102 may correspond to a panel having defined geometric dimensions. The substrate 102 may be selected from uncoated metal substrates or metal substrates being coated with a cured electrocoat layer and/or a cured filler layer; (ii) plastic substrates optionally being coated with a cured primer layer; and (iii) substrates comprising metallic and plastic parts and optionally being coated with a cured electrocoat layer and/or a cured filler layer. Suitable metal substrates include steel, iron, aluminum, copper, zinc and magnesium substrates as well as substrates made of alloys of steel, iron, aluminum, copper, zinc and magnesium.

Metal substrates being coated with a cured electrocoat may be produced by electrophoretic application of an electrocoat material to a substrate and subsequent curing of the applied electrocoat material. The electrocoat material may be a cathodic or anodic electrocoat material, preferably a cathodic electrocoat material. Electrocoat materials are aqueous coating materials comprising anionic or cationic polymers as binders. These polymers contain functional groups which are potentially anionic, i.e. can be co nverted to anionic groups, for example carboxylic acid groups, or functional groups which are potentially cationic, i.e. can be converted to cationic groups, for example amino groups. The conversion to charged groups is generally achieved by the use of appropriate neutralizing agents (organic amines (anionic) , organic carboxylic acids such as formic acid (cationic) . The electrocoat materials generally comprise typical anticorrosion pigments. The cathodic electrocoat materials preferred in the context of the invention comprise preferably cationic polymers as binders, especially hydroxy-functional polyether amines, which preferably have aromatic structural units. These polymers are especially used in combination with blocked polyisocyanates known per se. The application of the electrocoating material proceeds by electrophoresis. For this purpose, the metallic workpiece to be coated is first dipped into a dip bath containing the coating material, and an electrical DC field is applied between the metallic workpiece and a counterelectrode. The workpiece thus functions as an electrode; the nonvolatile constituents of the electrocoat material migrate, because of the described charge of the polymers used as binders, through the electrical field to the substrate and are deposited on the substrate, forming an electrocoat film. For example, in the case of a cathodic electrocoat, the substrate is thus connected as the cathode, and the hydroxide ions which form there through water electrolysis neutralize the cationic binder, such that it is deposited on the substrate and forms an electrocoat layer. After the electrolytic application of the electrocoat material, the coated substrate is removed from the bath, optionally rinsed off with, for example, water-based rinse solutions, then optionally flashed off and/or intermediately dried, and finally cured.

Metal substrates being coated with a cured filler layer may be produced by applying a filler coating composition to the substrate, optionally flashing off and/or intermediately drying said applied composition and finally curing said composition. Suitable filler coating compositions are known in the state of the art. The filler coating composition may be applied onto a cured electrocoating layer prepared as previously described.

Preferred plastic substrates are basically substrates comprising or consisting of (i) polar plastics, such as polycarbonate, polyamide, polystyrene, styrene copolymers, polyesters, polyphenylene oxides and blends of these plastics, (ii) synthetic resins such as polyurethane RIM, SMC, BMC and (iii) polyolefin substrates of the polyethylene and polypropylene type with a high rubber content, such as PP-EPDM, and surface-activated polyolefin substrates. The plastics may furthermore be fiber-reinforced, in particular using carbon fibers and/or metal fibers.

Coated and uncoated metal substrates may be pretreated prior to preparing the coating on the substrate. Pretreatment may include cleaning and/or application of conversion coatings. Cleaning may be effected mechanically, for example by means of wiping, grinding and/or polishing, and/or chemically by means of etching methods, such as surface etching in acid or alkali baths using, for example, hydrochloric acid or sulfuric acid, or by cleaning with organic solvents or aqueous detergents. Application of conversion coatings may include phosphation and/or chromation.

A coated substrate 106 may be prepared by applying one or more coating material (s) to the provided substrate 102 and drying and/or curing the applied coating material (s) . The cured coating material (s) may form the coating 104. The coating 104 may comprise one or more coating layer (s) . At least a part of the applied coating material (s) may be cured jointly. The coating material (s) may be applied to the substrate by dipping, bar coating, spraying, rolling, compressed air spraying (pneumatic application) , or electrostatic spray application (ESTA) . The coating material (s) may include pigmented coating material (s) comprising at least one color and/or effect pigment and/or transparent clearcoat coating material (s) . The pigmented coating material (s) may include at least one polymer and optionally at least one crosslinking agent. The pigmented coating material (s) may further include at least one solvent, such as water and/or organic solvent (s) . The transparent coating material (s) may include at least one polymer and at least one crosslinking agent. The transparent coating material (s) may further include at least one solvent, such as organic solvent (s) .

One or more scribe (s) 108 may be introduced into the prepared coated substrate 106. The scribe (s) may be introduced into the coating 104 of the coated substrate 106 or through the prepared coating 104 into the substrate 102 to obtain a scratched coated substrate 110. Scribe (s) 108 may be introduced using a sharp tool or a laser and serve to create intentional defects in the coating or in the coating and the substrate. The scribe (s) 108 may be of various shapes and sizes, and can be arranged in a regular pattern or randomly. The purpose of the scribe (s) 108 is to simulate potential damage that the coating or the coating and the substrate might experience in real-world conditions, and to provide a way to assess the coating's or the coating's and the substrate's resistance to such damage. After introducing the scribe (s) 108, residual coating and optionally substrate may be removed, for example via a brush.

The scratched coated substrate 110 may be subjected to one or more mechanical and/or chemical stress test (s) . Mechanical stress tests may include adhesion tests. Adhesion tests may include cross-cut adhesion tests, such as cross-cut adhesion tests according to DIN EN ISO 2409 (2020) . Cross-cut adhesion tests may include attaching a tape to the scribed area and pulling of the tape. Adhesion tests may include steam jet adhesion tests, such as steam jet adhesion tests according to DIN EN ISO 16925 (2022) . Steam jet adhesion tests may include treating the edges of the scribes with a stem jet. Chemical stress tests may include exposure to various corrosive or reactive substances, such as corrosion tests. The corrosion tests may be performed according to DIN EN ISO 9227 (2023) , DIN EN ISO 11997-1 (2018) and/or PV 1210 (2016) . The mechanical and/or chemical stress test (s) may result in one or more damaged area (s) 112 of the coating or the coating and the substrate. The damaged area (s) 112 may include area (s) where the coating or the coating and the substrate has at least been partially damaged. The damage may include at least partial delamination of the coating from the substrate, at least partial removal of the coating from the substrate, at least partial corrosion of the substrate or a combination thereof. The damage may relate to the property/ies of the coating. The damage may indicate the property/ies of the coating.

Image data of the scratched coated substrate 110 having been subjected to one or more chemical and/or physical test (s) may be acquired and/or determined, for example as described in the context of FIG. 2A and FIG. 2B. Based on such image data, property data associated with the property/ies of the coating may be determined, for example as described in the context of FIG. 2A to FIG. 9.

The property data may be compared to predefined property data. Predefined property data may include target property data and associated thresholds. If the determined property data does not fulfil predefined property data (e.g. is above or below a threshold associated with the target property data and rated as “fail” ) , adjustment of the formulation of coating material (s) used to prepare the coating may be initiated. After preparing an adjusted coating material based on the adjusted formulation, a new coated substrate may be prepared using the adjusted coating material (s) and the coated substrate may be subjected to one or more chemical and/or physical tests as previously described. This way, coating material (s) can be adjusted with respect to their property/ies in an efficient and reliable way, hence reducing the amount of waste generated during development of coating materials and improving the overall environmental impact associated with the coating material production.

FIG. 2A and FIG. 2B illustrate examples of systems for determining at least one property of a coating of a coated substrate. The coated substrate may include a substrate and the coating. The coating may cover at least a part of the surface of the substrate. The at least one property may include adhesion of the coating to the underlying substrate, impact resistance of the coating and/or corrosion resistance of the coating. The coated substrate may be prepared by as described in the context of FIG. 1.

The systems may include a data acquisition unit 202. The data acquisition unit 202 may be configured to provide image data of the coated substrate after such coated substrate has been subjected to one or more chemical and/or physical stress test (s) , for example as illustrated in FIG. 1. The data acquisition unit 202 may include a sample container 204 configured to hold the coated substrates. The sample container 204 may be a closed box. The closed box may avoid negative influences of environmental lightning conditions present around the data acquisition unit 202 on the image data acquired by data acquisition unit 202. The sample container 204 may have an opening to allow acquisition of image data using image data acquisition means 208. The sample container 204 may comprise at least one transparent wall to allow acquisition of image data using image data acquisition means 208. The data acquisition unit 202 may further include a lightning means 206. Lightning means 206 may include one or more light sources transmitting light beams assisting the image data acquisition means 208 during generating of the image data of the coated substrates. Data acquisition unit 202 may further include image data acquisition means 208 configured to acquire image data of the coated substrates located within the sample container 204. The image data acquisition means 208 may be an industrial camera, such as an infrared camera, a color camera, a 3D camera, or any other camera configured to generate image data of the coated substrates. The image data acquisition means 208 may be configured to capture different reflectivity characteristics of damaged area (s) and undamaged area (s) of the coating or the coating and the substrate. The combination of sample container 204 and image data acquisition means 208, in particular the combination of a closed box as sample container 204 and an infrared camera as image data acquisition means 208, may allow to capture the different reflectivity characteristics of damaged and undamaged areas. This way, the damage and undamaged areas as well as the scribes may be more reliably determined by the data-driven model, resulting in an improved accuracy and reliability of the generated property data associated with the property of the coating.

The systems may further include processing means 210. Processing means 210 may be configured to receive image data associated with the coated substrate. Image data associated with the coated substrate may be indicative of damaged area (s) present on the coating or present on the coating and the underlying substrate and one or more scribe (s) present within the coating or present within the coating and the substrate. Processing means 210 may be configured to process the received image data and to generate property data associated with the at least one property of the coating. Processing means 210 may include ROI data generator 212 configured to generate region of interest data based on received image data, for example as described in the context of FIG. 3 and FIG. 8. The region of interest data may indicate at least one region of interest (ROI) present within the image data associated with the coated substrate. With reference to FIG. 4, the ROI may correspond to a square region of 2.5 cm × 2.5 cm. The ROI may include one or more scribe (s) 108. The ROI may include a grid formed by the scribes 108. The ROI may have other sizes and/or shapes. In case multiple physical and/or chemical stress tests are performed on the coated substrate 106, two or more ROIs may be selected and region of interest data may be generated per selected ROI. In another example (not shown) , the ROI data generator 212 may be included in client device 222. Processing means 210 may further include a neural network engine 214. The neural network engine 214 may include a data-driven model. The data-driven model may be a trained data-driven model. Training may be performed as described in the context of FIG. 11. The neural network engine 214 may include one or more module (s) . Each module may include a trained neural network. The modules may include a feature extractor 302 (see FIG. 3 to FIG. 5A) . Feature extractor 302 may be configured generate feature maps based on received region of interest data. Feature maps may include 2D or 3D array (s) that represents the presence of certain features or patterns in the region of interest data at a particular scale and location. Each pixel or voxel in the feature map may correspond to a specific location in the region of interest data and may contain a value that represents the strength or confidence of the presence of a particular feature at that location. The modules may further include semantic segmentation module 304 (see FIG. 3 to FIG. 5A) . Semantic segmentation module 304 may be trained to generate damage data associated with damaged area (s) present on the coating or present on the coating and the substrate after subjecting the coated substrate containing the scribe (s) to one or more chemical and/or physical stress test (s) . The modules may further include object recognition module 306 (see FIG. 3 to FIG. 6) . Object recognition module 306 may be trained to generate scribe data associated with the scribe (s) present within the coating or present within the coating and the substrate.

Processing means 210 may further include a post processor 216 configured to process the scribe data generated by neural network engine 214, for example as described in the context of FIG. 8. Processing means 210 may further include merging unit 218 configured to merge the damage data and the scribe data generated by neural network engine 214 and/or configured to merge the damage data generated by neural network engine 214 and the target scribe data generated by post processor 216, for example as described in the context of FIG. 9.

In one embodiment and with reference to FIG. 2A, the system may further include a client device 222. Client device 222 may be a stationary device and/or a mobile device including an input unit and/or a display unit, such as, keyboard, keypad, touch screen, and the like. The client device 222 may be coupled via communication interfaces to data acquisition unit 202 and processing means 210. The communication interface may refer to a software and/or hardware interface for establishing communication such as transfer or exchange of signals or data with data acquisition unit 202 and/or processing means 210. Software interfaces may be e.g. function calls, APIs. Communication interfaces may comprise transceivers and/or receivers. The communication may either be wired, or it may be wireless. Communication interface may be based on or it supports one or more communication protocols. The communication protocol may a wireless protocol, for example: short distance communication protocol such as or WiFi, or long distance communication protocol such as cellular or mobile network, for example, second-generation cellular network ( "2G" ) , 3G, 4G, Long-Term Evolution ( "LTE" ) , or 5G. Alternatively, or in addition, the communication interface may even be based on a proprietary short distance or long distance protocol. The communication interface may support any one or more standards and/or proprietary protocols. Client device 222 may be in a client-server relationship with processing means 210 in which client device 222 acts as client and processing means 210 acts as server.

The client device 222 may be configured to receive image data from data acquisition unit 202 and to provide the received image data to processing means 210. The client device 222 may further be configured to control data acquisition unit 202, such as lightning means 206 and/or image data acquisition means 208 included in data acquisition unit 202. The client device 222 may further be configured to receive property data determined by processing means 210. The client device 222 may be configured to display received image data and/or property data within a graphical user interface. The user interface may be displayed within a display of client device 222 or within a display connected to client device 222. In another embodiment (not shown) , client device 222 may be configured to generate region of interest data, for example as described in the context of FIG. 3.

Client device 222 may be connected to a database, such as database 224. Client device 222 may be configured to provide data received from data acquisition unit 202 and/or processing means 210 to database 224 for storage. Data received from data acquisition unit 202 and/or processing means 210 may include image data, data associated with the lightning means 206 and/or image data acquisition means 208, such as lightning settings, camera settings, damage data, scribe data, target scribe data, property data or a combination thereof. Client device 222 may be configured to gather data stored in database 224, for example upon receiving a respective user input. The user input may include identifier (s) , such as identifier (s) associated with the coating and/or the coated substrate, based on which client device 222 may be configured to gather data matching the identifier (s) from database 224.

In another embodiment and with reference to FIG. 2B, data acquisition unit 202 may be directly connected to processing means 210. Likewise, database 224 may be directly connected to processing means 210. Database 224 may be connected to a client device (not shown) to allow access to data stored in such database.

FIG. 3 illustrates an example implementation of a neural network engine for generating damage data associated with damaged area (s) present on the coating or present on the coating and the substrate and scribe data associated with scribe (s) present within the coating or present through the coating into the substrate. The neural network engine may correspond to neural network engine 214 described in the context of FIG. 2A and FIG. 2B. The neural network engine may be contained within processing means 210 described in the context of FIG. 2A and FIG. 2B. The neural network engine may be configured to generate damage data and scribe data from region of interest data, for example as described in the context of FIG. 5A, FIG. 6 and FIG. 8. The region of interest data may be generated by ROI data generator 212, for example as described in the context of FIG. 8. The neural network engine may be connected to merging unit 218. The neural network engine may further be connected to post processor 216.

The neural network engine may include a data-driven model. The data-driven model may be a trained data-driven model. A trained data-driven may refer to a model structure together with parameters for the model structure that have been trained or tuned. FIG. 11. The data-driven model may be trained according to the method described in the context of FIG. 11.

FIG. 3 illustrates example couplings, signals and data that are passed among various modules of processing means 210, such as feature extractor 302, semantic segmentation module 304, object recognition module 306, post processor 216, merging unit 218 and property determinator 220. FIG. 3 illustrates example couplings, signals and data that are passed among various modules of the neural network engine 214, such as the feature extractor 302, semantic segmentation module 304, object recognition module 306 and optionally human-in-the-loop module 308. The neural network engine 214 may learn to generalize and scale its various modules, such as feature extractor 302, semantic segmentation module 304 and object recognition module 306, from a particular domain to another domain by having at least a part of the modules pre-trained on a general domain. The pre-trained models may be pre-trained, e.g. parameterized, using training data set containing unlabeled data of the general domain. The general domain may include fundamental linguistic, visual, world, and commonsense knowledge, which are domain-agnostic. These pre-trained modules may be fine-tuned or semantically conditioned to domain specific knowledge using domain specific training data, for example as described in the context of FIG. 11.

The modules, such as feature extractor 302, object recognition semantic segmentation module 304 and object recognition module 306 may form the data-driven model. The feature extractor 302, semantic segmentation module 304 and object recognition module 306 may each include at least one neural network. Such neural networks may be connected to each other to allow transfer of data, such as feature maps, from one neural network to one or more other neural networks, such as semantic segmentation module 304 and object recognition module 306. Each neural network may be trained individually. In addition or alternatively, one or more neural networks within the model structure may be trained jointly, for example as described in the context of FIG. 11.

In the neural network, nodes may be connected to one another via one or more edges. The neural network engine 214 may include an input layer, an output layer, and one or more intermediate layers. The input layer may be present within a different module than the output layer. For example, the input layer may be present within feature extractor 302 while the output layer may be present within semantic segmentation module 304 and object recognition module 306. Individual nodes may process their respective inputs according to a predefined function, and provide an output to a subsequent layer, or, in some cases, a previous layer. The inputs to a given node may be multiplied by a corresponding weight value for an edge between the input and the node. In addition, nodes may have individual bias values that are also used to produce outputs. Various training procedures may be applied to learn the edge weights and/or bias values, such as the training procedure described in the context of FIG. 11.

The neural network may have different layers that perform different specific functions. For example, one or more layers of nodes can collectively perform a specific operation, such as pooling, encoding, or convolution operations. A layer may refer to a group of nodes that share inputs and outputs, e.g., to or from external sources or other layers in the neural network or other modules of neural network engine 214. An operation may refer to a function that can be performed by one or more layers of nodes. The different layers may form a neural network architecture. The neural network architecture may refer to an overall architecture of a layered model, including the number of layers, the connectivity of the layers, and the type of operations performed by individual layers. The architecture may include one or more neural networks connected to each other. Instantiating the neural network architecture with weights may result in a neural network model, e.g. the model may be regarded as a network architecture with its weights or parameters. Parameters may refer to learnable values such as edge weights and bias values that can be learned by training a machine learning model, such as a neural network.

Feature extractor 302 may be coupled for communication and interaction with ROI data generator 212 and semantic segmentation module 304. Feature extractor 302 may further be coupled to human-in-the-loop module 308 to send requests for training data and/or to receive training data from human-in-the-loop module 308. Feature extractor 302 may receive region of interest data from ROI data generator 212. With reference to FIG. 4, ROI data generator 212 may be configured to generate region of interest data from the image data received from data acquisition unit 202. Region of interest data as generated by ROI data generator 212 may indicate a region of interest in the image data associated with the coating. The ROI may indicate a particular portion of the image data that includes data to be processed by the neural network engine 214 to generate the damage data and the scribe data. The ROI may be identified by its position, size or form. The ROI data may be a subset of the image data provided by data acquisition unit 202. The region of interest may be associated with one or more scribe (s) and optionally one or more damaged area (s) present on the coating or present on the coating and the substrate. The region of interest data may be generated by determining at least one region of interest (ROI) in the provided image data and extracting the determined at least one region of interest (ROI) as region of interest data from the provided image data. The region of interest data may be generated in response to receiving a user input indicating a region of interest (ROI) in the image data. The region of interest data may be generated using a data-driven model trained to generate region of interest data in response to receiving image data. The data-driven model may be a convolution neural network, for example as described in A. Krizhevsky, I. Sutskever, and G. Hinton, ImageNet classification with deep convolutional neural networks, NIPS, 2012. Use of ROI data allows to focus on a particular portion of the image data when generating the damage data and the scribe data, allowing to improve the efficiency of generating such data and improvi ng the accuracy and reliability of the generated damage data and scribe data.

Feature extractor 302 may be configured to generate feature maps from the region of interest data received from ROI data generator 212. Feature maps (also denoted as activation maps in this disclosure) may include 2D or 3D matrices containing features extracted from the ROI data. The extracted features may include edges, corners, colors and/or textures. Each element in the feature map may correspond to a function of a local neighborhood (patch) in the ROI data, hence the feature maps map the local features of the ROI data. Feature extractor 302 may include a neural network trained to generate feature maps from the provided region of interest data. The feature maps generated by feature extractor 302 may include feature maps having a pyramidal hierarchy with respect to their resolution. FIG. 5A illustrates an example architecture and example operations performed by feature extractor 302 on the ROI data. Feature extractor 302 may be configured to provide the generated feature maps to semantic segmentation module 304 and object recognition module 306. The generated feature maps may be provided to semantic segmentation module 304 and object recognition module 306 simultaneously. This allows to generate damage data and scribe data simultaneously, hence reducing the overall time required to determine the property data. This way, property data for a large amount of coated substrates may be determined, hence increasing the efficiency of the method. This may reduce consumption of coating material (s) used to produce coated object (s) since deviations of coating property/ies from target property/ies can be determined faster, hence allowing faster adaption of the coating material formulation and avoiding the generation of waste coating material comprising property/ies not matching given target property/ies. This way, the environmental impact of coating material development can be reduced.

Returning to FIG. 3, semantic segmentation module 304 may be coupled for communication and interaction with feature extractor 302 and merging unit 218. Semantic segmentation module 304 may further be coupled to human-in-the-loop module 308 to send requests for training data and/or to receive training data from human-in-the-loop module 308. Semantic segmentation module 304 may be configured to generate damage data associated with damaged area (s) on the coating or on the coating and the substrate. Semantic segmentation module 304 may be trained to generate the damage data based on feature maps received from feature extractor 302. The damage data may encode the spatial layout of the damaged area (s) present in the region of interest data. Semantic segmentation module 304 may be configured to classify all damaged area (s) as a single instance of the object “damaged area” . Semantic segmentation module 304 may include a neural network trained to generate damage data encoding the spatial layout of damaged area (s) present in the region of interest data based on the feature maps received from feature extractor 302. Semantic segmentation module 304 may include a neural network trained to increase the spatial resolution of the feature maps generated by feature extractor 302, to pool the feature maps having increased spatial resolution and to transform the pooled feature maps into the damage data. The damage data may include a binary matrix (e.g. a matrix only containing 0s and 1 s) indicating the presence of features (e.g. damaged area (s) ) and the absence of such features. The binary matrix may include values of 1 (indicating the presence of damaged area (s) ) and 0 (indicating the absence of damaged area (s) ) . The binary matrix may correspond to an array of values of 1 or 0. Each value of 0 and 1 may indicate whether the associated pixel belongs to a damaged area or an undamaged area.

With reference to FIG. 4, the binary matrix may be converted into image data. The image data may correspond to a mask. The mask may mask non-damaged area (s) such that damaged area (s) 402 are remaining. The generated mask may correspond to a binary image where the pixels denoting damaged area (s) are marked. The mask may indicate which elements or area (s) of the coating or the coating and the substrate are damaged and undamaged. This way, the damaged area (s) can be highlighted or isolated from undamaged area (s) . The mask may be generated by applying a masking function to the feature maps received from feature extractor 302. The masking function may indicate which values in the matrices (e.g. feature maps) received from feature extractor 302 should be used (e.g. may indicate damaged area (s) ) and which values in the matrices received from feature extractor 302 should not be used (e.g. may indicated undamaged area (s) ) for generating the damage data. The masking function may be learned by the neural network of semantic segmentation module 304 during the training. FIG. 5A illustrates an example architecture and example operations performed by semantic segmentation module 304.

Returning to FIG. 3, object recognition module 306 may be coupled for communication and interaction with feature extractor 302 and merging unit 218. Object recognition module 306 may be coupled for communication and interaction with feature extractor 302 and post processor 216. Object recognition module 306 may further be coupled to human-in-the-loop module 308 to send requests for training data and/or to receive training data from human-in-the-loop module 308. Object recognition module 306 may be configured to generate scribe data associated with the scribe (s) present within the coating or present through the coating into the substrate. In contrast to semantic segmentation module 304, object recognition module 306 may be configured to classify the scribe (s) as single instances of the detected object “scribe” , hence allowing a finer segmentation of the feature maps provided by feature extractor 302 than the semantic segmentation module 304. Object recognition module 306 may be configured to generate scribe data based on feature maps received from feature extractor 302. Object recognition module 306 may include a neural network trained to generate scribe data encoding the spatial layout of scribe (s) present in the region of interest based on the feature maps received from feature extractor 302.

The scribe data may encode position data of the scribe (s) 108. The position data may include coordinates of one or more points located on the scribe (s) 108, coordinates of the vertexes or intersections of the grid formed by the scribes 108 and/or scribe area (s) formed by the scribe (s) . The position data may be converted into image data. The neural network may be trained to generate bounding box (es) around the scribe (s) based on the feature maps provided by the feature extractor 302 and to generate scribe data including the position data. With reference to FIG. 4, the image data resulting from conversion of the position data may indicate the scribe (s) 108 present within the region of interest data. In the image illustrated in FIG. 4, the scribe data is overlaid over the ROI data.

Returning to FIG. 3, the human-in-the-loop module 308 may be coupled to provide teaching actions and/or instances to one or more module (s) of neural network engine 214. Human-in-the-loop module 308 may allow to implement a holistic human-in-the-loop machine learning paradigm, including both machine teaching and active learning. The machine teaching AI paradigm may combine the power of an intelligent human in the loop, as the teacher, with an AI system that learns to improve over time through efficiently interacting with its teacher. The teacher in the loop has some basic understanding of the capabilities and prior learnings of the model that the teacher is interacting with and is meant to provide some min imal supervision over the work of the neural network engine 214, as well as provide feedback on errors and mistakes. Through this efficient human-in-the-loop paradigm, the neural network engine 214 may be fine-tuned on specific domains with minimal training instances and in a short period of time.

The human-in-the-loop module 308 may be coupled to receive labelled training data (e.g. training data annotated by humans) from any number of humans. Any one of these humans may be considered teachers as it has been described above. The human-in-the-loop module 308 may use various statistical algorithms for vetting and training data set (s) as they get collected, for further ensuring the quality and accuracy. The human-in-the-loop module 308 may allow supervised learning, semi-supervised learning, or unsupervised learning for building, training and re-training the data-driven model (s) included in neural network engine 214 based on the type of data available and the particular machine learning technology used for implementation. The human-in-the-loop module 308 may include various other components such as a deployment module, an evaluation module, a generalization module, a collection module and an instantiation module to implement the process described below for continually improving the operation and accuracy of the neural network engine 214. The human-in-the-loop module 308 is particularly advantageous because it provides the ability to take human feedback into account to improve the operation of the neural network engine 214. In some implementations, the human-in-the-loop module 308 may be used to control behavior by taking feedback into account and have guarantees for generating (or not generating) a particular output given a particular input.

Human-in-the-loop module 308 may train neural network (s) of one or more module (s) of the neural network engine 214 and may deploy that neural network (s) for evaluation. Training may be done, for example, as described in the context of FIG. 11. For example, the neural network (s) may be deployed as part of the feature extractor 302, the semantic segmentation module 304 and/or the object recognition module 306. As the neural network (s) operate (s) in a specified setting, the human-in-the-loop module 308 may evaluate the accuracy of the neural network (s) and may collect failure cases. Based on the evaluation and collected failure cases, the human-in-the-loop module 308 may determine teaching set (s) . These teaching sets may include labelled data for training actions. Using the teaching set (s) , the human-in-the-loop module 308 may instantiate specific teaching actions and/or instances and may provide them to neural network (s) of one or more module (s) of neural network engine 214 for training.

Post processor 216 may be coupled for communication and interaction with object recognition module 306 and merging unit 218. Post processor 216 may be configured to generate target scribe data from the scribe data received from object recognition module 306. Post processor 216 may be configured to generate the target scribe data by classifying the scribe (s) indicated by the scribe data into horizontal and vertical groups, filtering the scribe (s) indicated by the scribe data and/or determining missing scribes (s) by calculating line distance (s) between the horizontal and/or vertical groups and optionally interpolating determined missing scribe (s) based on the calculated line distance (s) . Generating target scribe data may improve the accuracy of the generated property data since the amount of damaged area (s) present on the coating and/or the substrate may be determined more accurately based on the more accurate scribe data generated by post processor 216.

Merging unit 218 may be coupled for communication and interaction with semantic segmentation module 304. Merging unit 218 may further be coupled for communication and interaction with object recognition module 306 or post processor 216. Merging unit 218 may be configured to generate merged data by merging the damage data received from semantic segmentation module 304 and the scribe data received from object recognition module 306. Merging unit 218 may be configured to generate merged data by merging the damage data received from semantic segmentation module 304 and the target scribe data received from post processor 216. Merging the received data may include merging the data received from semantic segmentation module 304 and object recognition module 306. Merging the received data may include merging the binary mask received from semantic segmentation module 304 and the position data received from post processor 216. Merging unit 218 may be configured to merge the received data by generating scribe (s) based on the position data received from semantic segmentation module 304 or post processor 216 and merging the generated scribe (s) with the received damage data. The merged data may include or encode the spatial layout of the damaged area (s) present on the coating or present on the coating and the substrate and the spatial layout of the scribe (s) 108 present within the coating or present through the coating into the substrate. The merged data may further include the region of interest data. The merged data may correspond to a matrix containing the features of the binary matrix included in the damage data and the position data included in the scribe data. Image data may be generated from such matrix. With reference to FIG. 4, the merged data generated by merging unit 218 may encode an image of determined damaged area (s) 402 and determined scribe (s) 108. The image may further indicate undamaged area (s) of the coating. As illustrated in FIG. 4, the scribe (s) 108 may form a grid and the damaged area (s) 402 (white areas in FIG. 4) and undamaged areas 404 (black areas in FIG. 4) may be present as cells within the grid.

Property determinator 220 may be coupled for communication and interaction with merging unit 218. Property determinator 220 may be configured to generate the property data from the merged data received by merging unit 218. The merged data may include a feature map or matrix indicating the spatial layout of the damaged and undamaged area (s) 402, 404 as well as the spatial layout of the scribe (s) 108. Property determinator 220 may be configured to generate property data by determining the amount of damaged area and the non-damaged area based on the merged data and determining the property data based on the determined amount of damaged area and the determined amount of non-damaged area or based on the determined amount of damaged area and predefined evaluation data.

The predefined evaluation data may be stored in a database, such as database 310 connected to property determinator 220. The predefined evaluation data may include the amount of damaged areas and associated coating property classifications. The predefined evaluation data may be associated with a given chemical and/or physical stress test. The property data may be determined by mapping or matching the determined amount of damaged area with amounts of damaged areas included in the predefined evaluation data. Matching or mapping the determined amount of damaged area with the amount of damaged areas included in the predefined evaluation data may include determining the physical and/or chemical stress test the merged data is associated with and selecting an amount of damaged areas associated with such physical and/or chemical stress test from the predefined evaluation data for mapping or matching. The amount of damaged areas may be selected based on data associated with the physical and/or chemical stress test, such as identifier (s) associated with such respective test (s) . The identifier (s) may be associated with the image data and/or the ROI data determined by ROI data generator 212. With reference to FIG. 2A, property determinator 220 may further be coupled for communication and interaction with client device 222. Property determinator 220 may be configured to provide the generated property data to client device 222. With reference to FIG. 2B, property determinator 220 may further be coupled for communication and interaction with database 224. Property determinator 220 may be configured to provide the generated property data to database 224.

By determining the scribes and damaged area (s) present on the coating or present on the coating and the substrate using a trained data-driven model based on image data of such coated substrate, the damage of the coating and/or the substrate resulting from the chemical and/or physical stress test the coated substrate was subjected to can be reliably, accurately and efficiently determined. Unlike manual methods, which are time-consuming and prone to human error, the trained data-driven model can quickly and accurately identify the scribe (s) and damaged area (s) in image data of coated substrates. This results in a more reliable and robust assessment of coating properties, which is critical in industries where the quality of coatings can impact the performance and safety of coated products.

By using a modular neural-network architecture comprising a backbone network trained to generate feature maps from input data, a semantic segmentation module trained to determine the damaged area (s) of the coating or of the coating and the substrate from the feature maps and an object detection module trained to detect the scribe (s) from the feature maps, the accuracy and reliability of scribe detection and damaged area detection can be improved without having to employ different neural networks. This reduces the computational effort and hence the energy consumption required for training the data-driven model and allows to flexibly adapt the neural network architecture to different chemical and/or physical stress test conditions and specifications. This way, a wide range of coating properties can be reliably and accurately determined while reducing the environmental impact associated with the training process.

By processing the scribe data prior to determining the property data, a more accurate and reliable determination of the property data can be achieved. Processing of the scribe data allows to remove duplicate detection of scribe (s) by the data-driven model as well as wrong detection of scribe (s) . In addition, processing of the scribe data allows to add missing scribe (s) by interpolation. This way, the damaged area (s) can be determined more accurately during determination of the property data, resulting in an improved reliability and accuracy of the determined property data.

FIG. 5A illustrates an example architecture of the feature extractor 302 and the semantic segmentation module 304 included in the neural network engine 214 illustrated in FIG. 3 and FIG. 4. The feature extractor 302 may be trained to generate feature maps which may be used as input for the semantic segmentation module 304 and object recognition module 306.

With reference to FIG. 5B, feature extractor 302 may be based on a Feature Pyramid Network (FPN) 506 defined over a ResNet 504 architecture. The ResNet 504 may consist of five convolutional layers (conv1, conv2, . . ., conv5) , an average pooling layer, a fully connected (FC) layer and softmax (pooling layer, FC layer and softmax not shown in FIG. 5B) . At least a part of the convolutional layers (such as conv2 to conv5) may include or use residual block (s) (not shown in FIG. 5B) . The number of residual block (s) in such convolutional layers may be different for at least a part of the convolutional layers. Each residual block may include two or three convolutional layers, and an identity shortcut connection that bypasses these layers. The output of the convolutional layers in the residual block is added to the input provided to such residual block. This operation is called a shortcut connection or skip connection. The allows to create a direct path from the input to the output, which helps to propagate the gradient back through the layers during training and allows ResNet 504 to learn the residual mapping, which is the difference between the input and the output. ResNet 504 may be selected from ResNet18, ResNet-34, ResNet-50, ResNet-101, and ResNet-152, where these architectures differ in the residual block structure and the number of residual blocks. ResNet 504 may be trained to extract features of the ROI data 502 at decreasing levels of resolution from conv2 to conv5. A stride of 2 may be used in the first 3x3 convolutions of the first residual blocks for conv3, conv4, and conv5 to downsample feature maps to get larger receptive field. The feature maps generated by the convolutional layers con2 to conv5 (e.g. C2 to C5) may be provided to FPN 506 build on top of ResNet 504.

FPN 506 uses a top-down architecture with lateral connections to build an in-network feature pyramid. This allows to combine low-resolution, semantically strong features with high-resolution, semantically weak features through a top-down pathway and lateral connections (see FIG. 5A) . A 1x1 convolution may be applied to each feature map C2-C5 generated by ResNet 504 to unify the number of channels of those feature maps to 256. The unified feature maps may be upsampled by 2 (e.g. the spatial resolution of such feature maps may be increased) and element-wise added to the unified feature maps at the next levels, i.e., feature maps C4-C2. The process propagates coarser but semantically stronger features to the more finer feature maps therefore each level of the pyramid will consist of more complex, richer features. Another 3x3 convolution may be augmented to each of the resulting feature maps from C5 to C2 to provide the feature maps P5 to P2. Feature maps P2 to P5 may be provided to semantic segmentation module 304 and object recognition module 306.

The use of a combination of ResNet 504 and FPN 506 allows to supplement features extracted from higher layers of the ResNet 504 with features from lower layers of the ResNet 504 to obtain feature maps with high resolution and strong semantics. This way, small objects present in the ROI data 502 may be detected with higher accuracy since such objects may only be present in some feature maps (e.g. feature maps generated by lower layers) generated by the ResNet 504. This way, the feature maps P2 to P5 generated by the FPN 506 allow detection of damaged area (s) and scribe (s) having different sizes in the ROI data 502, allowing to improve the accuracy and reliability of the property data generated by the processing means 210.

Referring back to FIG. 5A, the feature maps P2 to P5 generated by FPN 506 may be provided to semantic segmentation module 304. Semantic segmentation module 304 may be configured to merge the features included in the feature maps P2 to P5 into a single output (e.g. damage data) . Starting from the deepest FPN output P5 (e.g. at 1/32 scale) , three upsampling stages may be performed to yield a feature map at 1/4 scale. Each upsampling stage may consist of 3x3 convolution, group norm, ReLU, and 2x bilinear upsampling. This strategy may be repeated for FPN outputs P4 to P2 at scales 1/16, 1/8, and 1/4 with progressively fewer upsampling stages. This way, a set of feature maps at the same 1/4 scale may be obtained. These feature maps may be summed element-wise. A final 1x1 convolution, 4x bilinear upsampling, and softmax may be used to generate the per-pixel class labels for damaged and undamaged area (s) at the original ROI data 502 resolution. The per-pixel class labels for damaged and undamaged area (s) at the original ROI data 502 resolution may correspond to the damage data. The damage data generated by semantic segmentation module 304 may be provided to merging unit 218.

In addition, feature maps P2 to P5 generated by FPN 506 may be provided to object recognition module 306. Feature map P2 to P5 generated by FPN 506 may be provided simultaneously to semantic segmentation module 304 and object recognition module 306. This may allow to determine the damage data and the scribe data simultaneously, hence reducing the overall time required to determine the property data. This way, property data for a larger amount of coated substrates may be determined, hence increasing the efficiency of the method. This may reduce consumption of coating material (s) used to produce coated object (s) since deviations of coating property/ies from target property/ies can be determined faster, hence allowing faster adaption of the coating material formulation and avoiding the generation of waste coating material comprising property/ies not matching given target property/ies.

With reference to FIG. 6, object recognition module 306 may be configured to generate scribe data associated with scribe (s) present within the coating or present through the coating into the substrate using the feature maps, such as feature maps P2 to P5, generated by feature extractor 302, such as generated by FPN 506 of feature extractor 302, as input. Object recognition module 306 may include a region proposal network (RPN) 604, a region of interest (ROI) Align layer 606, a bounding box head (bb head) and a classification head. A head may refer to final layers of the object recognition module 306 which take the features extracted by previous layers and produce an output, such as coordinates and a category. The RPN 604 may produce region proposals or candidate bounding boxes that may include objects, such as scribe (s) , present in the ROI data 502 encoded by the feature maps 602 (e.g. feature maps P2 to P5 generated by FPN 506) . It may operate on the feature maps 602 and may provide prospective regions of interest. The region proposals may include an objectness score. The objectness score may indicate the resemblance of the region proposal to a real object (e.g. scribe (s) , damaged area (s) , undamaged area (s) ) encoded by the ROI data. A high objectness score implies a likely presence of an object of interest within the proposed region, whereas a low score suggests that the region is probably background or doesn't contain any relevant object.

To generate region proposals, a small network may be slided over the feature maps 602 generated by the feature extractor 302. This small network may take as input an n × n spatial window of the input feature maps. Each sliding window may be mapped to a lower-dimensional feature. This feature may be fed into two sibling fully-connected layers, a box-regression layer and a box-classification layer. This architecture may be implemented with an n x n convolutional layer, such as a 3 x 3 convolutional layer, followed by two sibling 1 × 1 convolutional layers (for regression and box-classification, respectively) .

At each sliding-window location, multiple region proposals may be predicted. Denoting the number of maximum possible proposals for each location as k, the regression layer has 4k outputs encoding the coordinates of k boxes, and the bounding-box layer outputs 2k scores that estimate probability of object or not object for each proposal. The k proposals may be parameterized relative to k reference boxes (denoted as anchors hereinafter) . An anchor may be centered at the sliding window in question, and may be associated with a scale and aspect ratio. 3 scales and 3 aspect ratios may be used, resulting in k = 9 anchors at each sliding position. For each anchor, the RPN 604 may generate a binary class label (object or not object) and bounding box regression parameters. The class label may be used to decide whether that proposed region should be considered for further processing, and the bounding box regression parameters may be used to adjust the coordinates of the anchor to better fit the object. The anchors with high objectness scores may be selected and their coordinates may be adjusted using the bounding box regression parameters to generate the final set of region proposals. Non-Maximum Suppression may be used to reduce the number of proposals by discarding the region proposals which have a large overlap with higher scoring proposals.

The proposals may be provided along with the feature maps 602 to ROI Align 606. The ROI Align layer 606 may align the features present within a proposal with the spatial grid of the output feature map. The ROI Align layer may split each proposal into a specified number of spatial RoI bins or grids of equal size. These grids may be used to extract features related to the ROI from the feature maps 602. The RoI bins may be overlaid with the received feature maps. Bilinear interpolation may be used to compute the exact values of the input features at four regularly sampled locations in each RoI bin from the nearby grid points on the feature map, and the result may be aggregated, for example using max or average functions.

The bounding-box head may be trained to generate bounding boxes using the output of ROI Align layer around the scribe (s) . The classification layer may be trained to generate classifications, such as scribe and no scribe using the output of the ROI Align layer, and scribe line is further classified as positive diagonal lines (with a slope that goes from the bottom left to the top right) and negative diagonal lines (with a slope that goes from the top left to the bottom right) . The bounding-box head provides scribe data including position data. The position data may include a series of position coordinates of vertexes of scribe (s) . Based on such coordinates intersections of a grid formed by the scribes 108, and/or scribe area (s) formed by the scribe (s) 108 can be determined.

The object recognition module 306 may be implemented using the framework described in Kaiming He et. al, Mask-CNN, arXiv: 1703.06870v3, 24 January 2018 or using the framework described in Shaoqing Ren et. al, Faster R-CNN, arXiv: 1506.01497, 6 January 2016.

The scribe data generated by object recognition module 306 may be provided to post processor 216 or merging unit 218 for further processing.

FIG. 7 illustrates an example of a processing unit included in the system illustrated in FIG. 2A and FIG. 2B. The processing means 210 illustrated in FIG. 7 may be used to implement the methods shown in FIG. 8 to FIG. 11. The processing means 210 may be used to generate property data based on received image data associated with the coated substrate. Processing means 210 may implement neural network engine 214, merging unit 218 and property determinator 220 as illustrated in FIG. 2A to FIG. 6. Processing means 210 may implement neural network engine 214, post processor 216, merging unit 218 and property determinator 220 as illustrated in FIG. 2A to FIG. 6. Processing means 210 may represent a physical and tangible processing mechanism.

Processing means 210 may include one or more hardware processor (s) 702. The hardware processor (s) 702 may include, without limitation, one or more Central Processing Units (CPUs) , and/or one or more Graphics Processing Units (GPUs) , and/or one or more Application Specific Integrated Circuits (ASICs) , etc. More generally, any hardware processor can correspond to a general-purpose processing unit or an application-specific processor unit.

Processing means 210 may also include computer readable storage media 704, corresponding to one or more computer-readable media hardware units. The computer readable storage media 704 may retain any kind of information 706, such as machine-readable instructions, settings, data, etc. Without limitation, for instance, the computer readable storage media 704 may include one or more solid-state devices, one or more magnetic hard disks, one or more optical disks, magnetic tape, and so on. Any instance of the computer readable storage media 704 may use any technology for storing and retrieving information. Further, any instance of the computer readable storage media 704 may represent a fixed or removable component of the processing means 210. Further, any instance of the computer readable storage media 704 may provide volatile or non-volatile retention of information.

The processing means 210 may utilize any instance of the computer readable storage media 704 in different ways. For example, any instance of the computer readable storage media 704 may represent a hardware memory unit (such as Random Access Memory (RAM) ) for storing transient information during execution of a program by the computing device, and/or a hardware storage unit (such as a hard disk) for retaining/archiving information on a more permanent basis. In the latter case, the processing means 210 may also include one or more drive mechanism (s) 708 (such as a hard drive mechanism) for storing and retrieving information from an instance of the computer readable storage media 704.

Processing means 210 may perform the methods described in the context of FIG. 8 to FIG. 11 when the hardware processor (s) 702 carry out computer-readable instructions stored in any instance of the computer readable storage media 704.

Alternatively, or in addition, processing means 210 may rely on one or more other hardware logic components 710 to perform operations using a task-specific collection of logic gates. For instance, the hardware logic component (s) 710 may include a fixed configuration of hardware logic gates, e.g., that are created and set at the time of manufacture, and thereafter unalterable. Alternatively, or in addition, the other hardware logic components 710 may include a collection of programmable hardware logic gates that can be set to perform different application-specific tasks. The latter category of devices includes, but is not limited to Programmable Array Logic Devices (PALs) , Generic Array Logic Devices (GALs) , Complex Programmable Logic Devices (CPLDs) , Field-Programmable Gate Arrays (FPGAs) , etc.

FIG. 7 generally indicates that hardware logic circuitry 712 includes any combination of the hardware processor (s) 702, the computer readable storage media 704, and/or the other hardware logic components 710. That is, processing means 210 may employ any combination of the hardware processor (s) 702 that execute machine-readable instructions provided in the computer readable storage media 704, and/or one or more other hardware logic components 710 that perform operations using a fixed and/or programmable collection of hardware logic gates. More generally stated, the hardware logic circuitry 712 may correspond to one or more hardware logic components of any type (s) that perform operations based on logic stored in and/or otherwise embodied in the hardware logic component (s) .

Processing means 210 may also include an input/output interface 714 for receiving various inputs (via input device (s) 716) , and for providing various outputs (via output device (s) 724) . Input devices may include a keyboard device, a mouse input device, a touchscreen input device, a digitizing pad, one or more static image cameras, one or more video cameras, one or more depth camera systems, one or more microphones, a voice recognition mechanism, any movement detection mechanisms (e.g., accelerometers, gyroscopes, etc. ) . Output device (s) may include a display device 718 and an associated graphical user interface presentation (GUI) 720. The display device 718 may correspond to a liquid crystal display device, a light-emitting diode display (LED) device, a cathode ray tube device, a projection mechanism, etc. Other output devices may include a printer, one or more speakers, a haptic output mechanism, an archival mechanism (for storing output information) , and so on. Processing means 210 may also include one or more network interface (s) 726 for exchanging data with other devices via one or more communication conduit (s) 728. One or more communication bus (es) 730 communicatively may couple the above-described components together.

The communication conduit (s) 728 may be implemented in any manner, e.g., by a local area computer network, a wide area computer network (e.g., the Internet) , point-to-point connections, etc., or any combination thereof. The communication conduit (s) 728 may include any combination of hardwired links, wireless links, routers, gateway functionality, name servers, etc., governed by any protocol or combination of protocols.

FIG. 7 shows processing means 210 as being composed of a discrete collection of separate units. In some cases, the collection of units may correspond to discrete hardware units provided in a computing device chassis having any form factor. FIG. 7 shows illustrative form factors in its bottom portion. In other cases, ROI data generator 212 may include a hardware logic component that integrates the functions of two or more of the units or modules shown in FIG. 3 to FIG. 6. For instance, processing means 210 may include a system on a chip (SoC or SOC) , corresponding to an integrated circuit that combines the functions of two or more of the units or modules shown in FIG. 3 to FIG. 6.

FIG. 8 illustrates a flow chart of an example of a method for determining at least one property of a coating of a coated substrate. The coated substrate may include a substrate and the coating. The coating may cover at least a part of the surface of the substrate. The at least one property may include adhesion property/ies, impact resistance property/ies and/or corrosion resistance property/ies. The method may be implemented by the systems described in the context of FIG. 2A and FIG. 2B. The method may be implemented by neural network engine 214 including feature extractor 302, semantic segmentation module 304 and object recognition module 306 as described in the context of FIG. 3 to FIG. 6.

The coating may comprise one or more coating layers, for example as described in the context of FIG. 1. The coated substrate may be prepared as described in the context of FIG. 1. The at least one property may include adhesion of the coating to the underlying substrate, impact resistance of the coating and/or corrosion resistance of the coating. The coated substrate may comprise one or more scribe (s) within the coating and/or through the coating into the substrate. The scribe (s) may be oriented in a defined pattern on the coating. Hence, the scribe (s) may form a defined pattern, such as a grid (see for example FIG. 1) . The coated substrate comprising the one or more scribe (s) may be prepared as described in the context of FIG. 1. The coated substrate comprising the one or more scribe (s) may have been subjected to one or more physical and/or chemical stress tests, for example as described in the context of FIG. 1.

With reference to FIG. 2A, FIG. 2B and FIG. 10, image data associated with the coated substrate may be provided (see block 802) . The image data may include data related to the one or more scribe (s) and optionally data related to damaged area (s) of the coating and/or the substrate. The image data may indicate the one or more scribe (s) , undamaged area (s) of the coating and/or the substrate and optionally damaged area (s) of the coating or coating and the substrate. Providing the image data may include capturing an image of at least a part of the coated substrate comprising the scribe (s) with an image data acquisition means 208 and providing the generated image data. The image data acquisition means 208 may include a camera as described in the context of FIG. 2A and FIG. 2B. The image data acquisition means 208 may be located within a data acquisition unit 202, for example as illustrated in FIG. 2A and FIG. 2B. The image of the coated substrate may be captured by placing the coated substrate into the sample container 204 and capturing an image using image data acquisition means 208. A lightning means 206 may be used during image acquisition by image data acquisition means 208. The lightning means 206 may include one or more light (s) for illuminating the coated substrate located within the sample container 204.

With continued reference to FIG. 2A, FIG. 2B and FIG. 10, region of interest (ROI) data being indicative of the scribe (s) and optionally damaged area (s) of the coating or the coating and the substrate may be determined based on the provided image data (see block 804) . The ROI data may further be indicative of undamaged area (s) of the coating and the substrate. The ROI data may be generated by determining at least one region of interest (ROI) in the provided image data and extracting the determined at least one region of interest (ROI) as region of interest data from the provided image data. With reference to F IG. 3, one or more ROI (s) may be determined in response to receiving a user input indicating at least one region of interest (ROI) in the image data. The ROI (s) may be determined using a data-driven model trained to determine ROI (s) and to generate region of interest data in response to receiving image data. The ROI (s) may relate to a defined region covering at least a part of the scribe (s) within the image of the coated substrate. For instance, the ROI (s) may each relate to a square region of 2.5 cm × 2.5 cm covering at least a part of the scribe (s) . The square region may contain 8 x 8 scribe lines. These scribe lines may form a 7 x 7 grid.

Each ROI may indicate a particular chemical and/or physical test performed on the scratched coated substrate. This way, several property/ies of the coating can be determined using a single coated substrate, hence reducing the amount of waste required to determine the coating property/ies and improving the overall environmental impact associated with the method.

With continued reference to FIG. 2A, FIG. 2B and FIG. 10, damage data associated with damaged area (s) present on the coating or present on the coating and the substrate and scribe data associated with the scribe (s) may be generated by inputting the ROI data into a data-driven model, such as neural network engine 214 (see block 806) . With reference to FIG. 3, the neural network engine 214 may include feature extractor 302, semantic segmentation module 304 and object recognition module 306. Feature extractor 302 may be trained to generate feature maps from the received ROI data and may provide the extracted feature maps to the semantic segmentation module 304 and the object recognition module 306. Semantic segmentation module 304 may be trained to generate damage data encoding the spatial layout of damaged area (s) present in the region of interest based on the received feature maps and may provide the damage data to merging unit 218. The region of interest may be indicated or associated with the region of interest data. The region of interest may be defined by the region of interest data. Object recognition module 306 may generate scribe data encoding the spatial layout of the scribe (s) present in the ROI based on the received feature maps and may provide the scribe data to merging unit 218 or post processor 216. With further reference to FIG. 7, the data-driven model may be instantiated and executed on hardware processor (s) 702 of processing means 210.

The data driven-model may be trained to generate the damage data and the scribe data from the ROI data. The data-driven model may be parameterized according to a training data set including labelled image data, for example as described in the context of FI G. 11. The label (s) may relate to damaged area (s) of the coating or the coating and the substrate and scribe (s) present within the coating or present through the coating and in the substrate. The trained data-driven model may generate damage data indicating damaged area (s) of the coating or the coating and the substrate and scribe data indicating the scribe (s) , for example as described in the context of FIG. 3 to FIG. 6. The damage data may correspond to a binary mask indicating the damaged and undamaged area (s) as described in the context of FIG. 3. The scribe data may correspond to a binary mask indicating the pixels present within bounding box (es) indicating the scribe (s) and pixels outside of the bounding box (es) indicating the scribe (s) . The bi nary mask may indicate 8 x 8 scribe lines forming a 7 x 7 grid. The trained data-driven model may hence allow to transform region of interest data defining an ROI in the image data of the coated substrate into damage data and scribe data which can be used to determine the property/ies associated with the coating.

A compiler executed by a CPU present within processing means 210 may determine a task list for the neural network engine 214. The neural network engine 214 to be executed may include one or more neural network (s) . With reference to FIG. 3 to FIG. 6, the neural network (s) may include network layers or sub-layers that are instantiated or implemented as a series of tasks executed by the processing means 210. The neural network (s) may be instantiated by the processing means 210. To do so the neural network (s) may be converted into a task list to become executable by the processing means 210. The neural network (s) may be converted by the CPU to the task list. The task list includes a linear link-list defining a sequence of tasks including tasks per feed forward layer and/or normalization layer. Each task may be associated with a task descriptor that defines a configuration of the neural network engine 214 to execute the task. Each task may correspond with a single network layer of the neural network (s) , a portion of a network layer of the neural network (s) , or multiple network layers of the neural network (s) . Based on the task list generated by the CPU and provided to the neural network engine 214, the neural network engine 214 may instantiate the neural network (s) by executing the tasks of the task list under the control of a neural task manager.

The neural task manager may receive a task list from a compiler executed by the CPU, store tasks in its task queues, choose a task to perform, and send instructions to other components of the neural processor circuit for performing the chosen task. The neural task manager may include one or more task queues. Each task queue may be coupled to the CPU and the task arbiter. Each task queue may receive from the CPU a reference to a task list of tasks that when executed by the neural processor circuit instantiates the neural network (s) . The reference stored in each task queue may include a set of pointers and counters pointing to the task list of the task descriptors in the system memory. Each task queue may be further associated with a priority parameter that defines the relative priority of the task queues. The task descriptor of a task may specify a configuration of the neural processor circuit for executing the task.

It may be determined whether to process the scribe data (see decision block 808) . Processing the scribe data may allow to enhance or improve the scribe data generated by the data driven model, such as generated by the neural network of object recognition module 306. Enhancing or improving the scribe data may include removing duplicate scribe (s) and/or wrongly detected scribe (s) and/or interpolating undetected scribe (s) (e.g. missing scribe (s) ) . Improving or enhancing the scribe data may allow to improve the accuracy and reliability of the property data determined based on such enhanced scribe data.

If the scribe data is to be processed target scribe data may be determined based on the scribe data determined by the data-driven model (see block 812) . The target scribe data may be generated by

· filtering the scribe (s) indicated by the scribe data and/or determining missing scribes (s) by calculating line distance (s) between the horizontal and/or vertical groups,

In one example, the target scribe data may indicate 6 x 6 scribe lines forming a 5 x 5 grid. Generating the target scribe data may hence allow to condense the scribe data and to focus on the central ROI region of the scribe (s) indicated by the scribe data. This way, the property data may be determined with a higher accuracy and reliability, hence improving coating formulation adjustment based on the determined property/ies and reducing generation of waste coating material resulting in coatings having property/ies not fulfilling required specifications. This allows to reduce the environmental impact of the coating material production while at the same time reliably ensuring that coatings produced from such coating materials fulfil required specifications.

The target scribe data resulting from processing the scribe data may be used to determine the property data.

Property data associated with the least one property of the coating may be determined based on the damage data and scribe data generated by the data-driven model or based on the damage data generated by the data-driven model and the generated target scribe data (see bock 814) . With reference to FIG. 9A and FIG. 10, generating property data may include

· generating merged data by merging the damage data and the scribe data or the target scribe data (see block 902 of FIG. 9A) ,

· determining the amount of damaged area and the amount of non-damaged area based on the merged data (see block 904 of FIG. 9A) and

· determining property data associated with the at least one property based on the determined amount of damaged area and the determined amount of non-damaged area or based on the determined amount of damaged area and predefined evaluation data (see block 908 in FIG. 9A) .

The merged data may include or encode the spatial layout of the damaged area (s) present on the coating or the coating and the substrate and the spatial layout of the scribe (s) 108 present within the coating or present throughout the coating into the substrate. The merged data may further include the region of interest data. If the scribe (s) form a grid, the merged data may encode two types of cells created by merging the damage data with the scribe data or the target scribe data. One type may refer to damaged cells and the other may refer to undamaged cells. Damaged cells may denote cells in which the coating and/or the substrate has been damaged (e.g. the cell can be regarded as damaged area (s) ) . The merged data may correspond to a matrix containing the features of the binary matrices included in the damage data and the scribe data or the target scribe data.

Generating merged data may include merging the binary matrix included in the damage data generated by semantic segmentation module 304 and the position data included in the scribe data generated by object recognition module 306. The position data may be used to represent lines within the binary matrix. For instance, the binary matrix may be converted into image data and lines representing scribe (s) may be depicted in the image based on the position data included in the scribe data. The binary matrix may be converted into image data by interpreting the values in the matrix as pixel values, e.g. by assigning color values to values, such as 0s and 1 s, included in the binary matrix. For instance, a value of 1 in the binary matrix may be assigned to a white color value while a value of 0 in the binary matrix may be assigned to a black color value or vice versa. This may allow to provide an image of the features extracted by neural network engine 214 from the ROI data and allows a user to determine the accuracy and reliability of the extracted features by comparing the generated image with an image of the coated substrate used as input to extract such features.

The amount of damaged area (s) may be determined by dividing the number of pixels encoding damaged area (s) by the total number of pixels. The total number of pixels may correspond to the total number of pixels present within the image data, such as image data generated from the merged data. The amount of damaged area (s) may be determined using a defined region of the merged data. For instance, the defined region may be square region in the center of the merged data. The square region may cover a predefined number of cells, such as 10 cells × 10 cells, 7 cells × 7 cells or 5 cells x 5 cells. If the scribes form a grid comprising several cells, the area of a single cell may be determined from the dimensions of the grid. The amount of damaged area may be determined from the number of damaged cell (s) present within the grid.

The property data may be determined by mapping or matching the determined amount of damaged area with amounts of damaged areas included in the predefined evaluation data. The predefined evaluation data may include a correlation between the amounts of damaged areas and the degree of damage according to a given specification. This way, the percentage of damaged pixels may be correlated with a degree of damage according to a given specification. The correlation may be obtained from labelled data indicating the degree of damage and the percentage of damaged pixels. Matching or mapping the determined amount of damaged area with the amount of damaged areas included in the predefined evaluation data may include determining the physical and/or chemical stress test the merged data is associated with and selecting an amount of damaged areas associated with such physical and/or chemical stress test from the predefined evaluation data for mapping or matching. The amount of damaged areas may be selected based on data associated with the physical and/or chemical stress test, such as identifier (s) associated with such respective test (s) . The identifier (s) may be associated with the provided image data and/or the determined ROI data. The predefined evaluation data may be provided (see block 906 in FIG. 9A) . Providing the predefined evaluation data may include providing a data storage storing such predefined evaluation data. The predefined evaluation data may be generated based on evaluation criteria associated with the chemical and/or physical stress tests. For example, the predefined evaluation data may be generated from evaluation criteria contained within specifications, such as technical standards. The technical standards may be defined and provided by standard setting organizations, such as the American Society for Testing and Materials (also known as ASTM International) , the German Institute for Standardization (Deutsches Institut für Normung) , the Japanese Industrial Standards Committee (JISC) and/or the Standardization Administration of China (SAC) . The technical standards may be defined and provided by coating material consumers, such as end product producer (s) producing coated substrates. The predefined evaluation data may include the amount of damaged areas and associated coating property classifications. The predefined evaluation data may be associated with a given chemical and/or physical stress test.

The property data may be determined by comparing the amount of damaged area to the amount of non-damaged area. The property data may hence indicate a fraction of the damaged area with respect to the total area (e.g. damaged and non-damage area) .

If the scribe data is not to be processed property data associated with the least one property of the coating based on the damage data and scribe data generated by the data-driven model may be determined as described in the context of block 814 (see block 810) .

Returning to FIG. 8, the determined property data may be provided (see block 816) . Providing the determined property data may include providing the determined property data for display. For example and with reference to FIG. 2A, the determined property data may be provided to client device 222 for display. Client device 222 may display the determined property data within a graphical user interface. The determined property data may be displayed along with further data, such as an identifier associated with the coating, image data generated from ROI data, damage data, scribe data, target scribe data, merged data and/or predefined evaluation data used to generate the property data. This may allow a user to judge the accuracy and reliability of the determined property data.

Providing the determined property data may include providing the determined property data to a data storage. For example and with reference to FIG. 2B, the determined property data may be provided to a database configured to store such property data. The property data may be provided to such database along with further data, such as an identifier associated with the coating, ROI data, damage data, scribe data, target scribe data, merged data and/or predefined evaluation data used to generate the property data.

With further reference to FIG. 9B, the determined property data may be compared to target coating property data and adjustment of the coating material (s) related to the coating may be triggered if the determined property data does not fulfil the target coating property data.

Target property data may be provided (see block 910 of FIG. 9B) . Target property data may be provided from a database storing such target property data. The target property data may include target property/ies. The target property/ies may define maximum allowable property/ies. The target property/ies may correspond to property/ies of coatings defined by technical specifications, such as technical specifications of consumers of the coating materials, such as end product producers consuming such coating materials to produce coated end products.

The determined property data may be compared to the provided target property data (see block 912 of FIG. 9B) . The determined property data may be compared per property related or associated with the target property data with respective target property data. Respective target property data may be determined using coating identifier (s) and/or coating identifier (s) . Such identifier (s) may be used to gather respective target property data from the database storing such target property data.

It may be determined whether the property data is acceptable, e.g. whether the determined property data matches the target property data (see block 914 of FIG. 9B) .

If the determined property data matches the target property data, the method may end. The determined property data may match the target property data if the determine property data is below the maximum allowable property/ies defined by the target property data.

If the determined property data does not match the target property data, adjustment of formulation (s) of coating material (s) related to the coating may be triggered (see block 916 of FIG. 9B) . The coating material (s) related to the coating may include coating material (s) used to prepare the coating. The formulation (s) may be associated with formulation data including input material data associated with input material (s) and associated amounts of such input material (s) . The formulation data may be used to prepare the coating material. Adjustment of the formulation may include modifying the formulation data. Modifying the formulation data may include modifying input material (s) and/or associated amount (s) . Modifying input material (s) may include removing and/or adding input material (s) . The adjusted formulation may be used to prepare an adjusted coating material. The adjusted coating material may be used to prepare a coating on the substrate, for example as described in the context of FIG. 1. The coated substrate may be scratched and subjected to one or more chemical and/or physical stress tests as described in the context of FIG. 1. The property data may be determined for such coated substrate. The determined property data may be compared to property data associated with the original coating material formulation. This way, an influence of input material (s) and/or amounts of such input material (s) on one or more property/ies of the coating may be determined.

By using a modular neural-network architecture comprising a backbone network trained to generate feature maps from input data, a semantic segmentation module trained to determine the damaged area (s) of the coating or of the coating and the substrate from the feature maps and an object detection module trained to detect the scribe (s) from the feature maps, the accuracy and reliability of scribe detection and damaged area detection can be improved without having to employ different neural networks. This reduces the computational effort and hence the energy consumption required for training the data-driven model and allows to flexibly adapt the neural network architecture to different chemical and/or phys ical stress test conditions and specifications. This way, a wide range of coating properties can be reliably and accurately determined while reducing the environmental impact associated with the training process.

FIG. 11 illustrates a method for training the neural network engine illustrated in FIG. 2A to FIG. 6. The neural network engine may be a neural network engine 214 including several modules or units, such as feature extractor 302, semantic segmentation module 304 and object recognition module 306 as described in the context of FIG. 3.

The neural network engine 214 may be pre-trained on a training data set including generic images not being related to coated substrates (e.g. including images of a generic domain) . The MS COCO (Microsoft Common Objects in Context) data may be used for pre-training the neural network engine 214. The MS COCO dataset is a large-scale object detection, segmentation, key-point detection, and captioning dataset. The dataset consists of 328K images. It contains images split into training, validation and test sets. The dataset has annotations for object detection, captioning, keypoints detection, stuff image segmentation, panoptic and dense pose.

The pre-trained neural network engine 214 may be fine-tuned based on a training data set including images of the specific domain, e.g. images of scratched coated substrates having been subjected to chemical and/or physical stress test (s) . The training data set may be generated by providing image data associated with such scratched coated substrates. The scratched coated substrates may be prepared as described in the context of FIG. 1. The image data of such scratched coated substrates may be acquired or generated using data acquisition unit 202 described in the context of FIG. 2A and FIG. 2B.

Image data associated with the coating present on the substrate may be provided (see block 1102) . The image data may be provided as described in the context of FIG. 8. Region of interest data may be determined from the provided image data (see block 1104) . The ROI data may be determined as described in the context of FIG. 3 and FIG. 4. The ROI data may indicate the scratched coating area having been subjected to chemical and/or physical stress test (s) . Multiple ROI data sets may be determined from image data associated with a single scratched coated substrate. Each of such ROI data sets may be associated with a defined chemical or physical stress test. This may allow to generate ROI data for several chemical and/or physical stress tests having been performed on a single scratched coated substrate.

Labelled training data may be generated based on the determined ROI data (see block 1106) . Labelled training data may be generated by labelling the ROI data. Labelling the ROI data may include labelling the scribe (s) 108 present in the ROI data and labelling damaged area (s) 402 present in the ROI data. The labelling may be performed manually. Labelling the scribe (s) 108 and the damaged area (s) may include drawing bounding boxes around the scribe (s) 108 and the damaged area (s) and labelling the bounding box (es) with a respective label, such as scribe and damaged area. The labelled training data may further include acquisition data including lightning data and/or camera data. Lighting data may include lightning conditions used during image acquisition. Camera data may include camera settings used during acquisition, such as camera type, exposure time or the like. Use of the acquisition data during fine-tuning of the neural network engine 214 may result in a more reliable and accurate determination of damage data and scribe data based on input image data acquired under different lightning conditions and/or camera settings.

The training data set may be extended using image augmentation techniques to generate random training data from the provided image data or the ROI data (see block 1108) . Image augmentation may allow to synthetically increase the size of the training data without having to generate numerous scratched coated substrates and subject such substrates to chemical and/or physical tests. This avoids generation of waste and consumption of energy and material required for preparation of the scratched coated substrates and the testing. This way, the environmental impact associated with the training of the neural network engine 214 may be reduced while improving the accuracy of the determined damage data and scribe data for varying coating colors, damage patterns, scribe patterns, damaged area patterns and image acquisiti on conditions (e.g. lightning conditions, resolution, exposure time of the image data acquisition means 208 etc. ) .

Random training data may be generated from the provided image data and/or the ROI data by performing at least one transformation operation on such data. The transformation operation (s) may include horizontal and/or vertical flipping, random cropping, random scaling, random contrast setting and/or random intensity setting. Random cropping and/or random scaling may allow enrich the training data with images associated with different resolutions and/or sharpness. Random contrast setting and random intensity setting may allow to enrich the training data with images associated with different lightning conditions and/or exposure conditions.

The random training data may be used to generate labelled random training data (see block 1110) . Generating labelled random training data may be performed as previously described. The labelled random training data may further include acquisition data associated with the image data the transformation operation was performed on.

The data-driven model may be provided (see block 1112) . The data driven model may be a data-driven model pre-trained on a training data set from a generic domain as previously described. The data-driven model may be implemented on or part of neural network engine 214 described in the context of FIG. 3 to FIG. 6. The data-driven model may include one or more neural networks, such as described in the context of FIG. 3 to FIG. 6.

The generated labelled training data and/or the labelled random training data as well as the region of interest data may be provided to the data-driven model for training (see block 1114) . This step can be considered as fine-tuning the provided pre-trained data-driven model. For training, the scribe (s) and damaged area (s) indicated by the ROI data may be known and may be correlated with the damage data and scribe data produced by the data-driven model or the merged data produced by merging unit 218. By training, the damage data and scribe data associated with the ROI data may be determined by correlating generated damage data and scribe data with the labels. For example, labels may provide known scribe (s) and damaged area (s) defining the damage data and scribe data determined by the data-driven model. Correlating damage data and scribe data with label (s) may include the data-driven model assessing differences between the generated damage data and scribe data and the label (s) . The data-driven model may thus perform error function analysis on the differences between the damage data and scribe data for given label (s) and refine the data generation process until the data generation process accurately determines the label (s) . Thus, the data-driven model may be trained by the labelled training data and/or the labelled random training data to accurately determine the damage data and the scribe data.

The neural networks, such as the neural networks included in the data-driven model may be trained jointly on the data. The losses from different neural network (s) and/or heads present within such network (s) may have different scales and normalization policies. Re-weighting between the losses may be used to avoid degradation of the final performance for generating the damage data and scribe data. By tuning the re-weighting, it is possible to train the data-driven model such that its performance is comparable to separate task-specific models while reducing the computing time and hence also the energy and resources required for training. This may allow to reduce the environmental impact of the training process.

The trained data-driven model may be provided (see block 1116) . The trained data-driven model may be used within the method described in the context of FIG. 8 to FIG. 9B.

The present disclosure has been described in conjunction with preferred embodiments and examples as well. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed invention, from the studies of the drawings, this dis-closure and the claims.

Any steps presented herein can be performed in any order. The methods disclosed herein are not limited to a specific order of these steps. It is also not required that the different steps are per-formed at a certain place or in a certain computing node of a distributed system, i.e. each of the steps may be performed at different computing nodes using different equipment/data processing.

As used herein “determining” also includes “initiating or causing to determine” , “generating” also includes “initiating and/or causing to generate” and “providing” also includes “initiating or causing to determine, generate, select, send and/or receive” . “Initiating or causing to perform an action” includes any processing signal that triggers a computing node or device to perform the respective action.

In the claims as well as in the description the word “comprising” or “including” or similar wording does not exclude other elements or steps and shall not be construed limiting to the elements or steps lined out. The indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation or further elements may be included.

Providing in the scope of this disclosure may include any interface configured to provide data. This may include an application programming interface, a human-machine interface such as a display and/or a software module interface. Providing may include communication of data or submission of data to the interface, in particular display to a user or use of the data by the receiving entity.

Any disclosure and embodiments described herein relate to methods, systems, apparatuses, devices, chemicals, materials, services, uses, computer program elements lined out above and vice versa. Advantageously, the benefits provided by any of the embodiments and examples equally apply to all other embodiments and examples and vice versa.

All terms and definitions used herein are understood broadly and have their general meaning.

Claims

A method for determining at least one property of a coating being present on a substrate of a coated substrate, wherein the coating comprises one or more scribe (s) and wherein the coated substrate has been subjected to at least one chemical and/or mechanical stress test, the method comprising:

- providing image data associated with the coated substrate;

- determining region of interest (ROI) data being indicative of the scribe (s) and optionally damage area (s) of the coating and/or substrate based on the provided image data;

- processing the region of interest data using a data-driven model parameterized and/or trained to generate damage data associated with damaged area (s) present on the coating and/or the substrate and scribe data associated with the scribe (s) in response to being provided by the region of interest data;

- determining property data associated with the at least one property of the coating based on the damage data and the scribe data generated by the data-driven model,

- providing the property data associated with the at least one property of the coating.
The method of claim 1, wherein the one or more scribe (s) are obtained by introducing the scribe (s) at one or more defined location (s) on the coated substrate into the coating and/or by introducing the scribe (s) at one or more defined location (s) on the coated substrate through the coating to the underlying substrate.
The method of claim 1 or 2, wherein determining the region of interest (ROI) data includes determining at least one region of interest (ROI) in the provided image data and extracting the determined at least one region of interest (ROI) as region of interest data from the provided image data.
The method of any one of claims 1 to 3, wherein the data-driven model includes

- a feature extractor trained to extract feature maps from the regions of interest data and to provide the extracted feature maps to a semantic segmentation module and an object recognition module,

- the semantic segmentation module trained to generate damage data encoding the spatial layout of damaged area (s) present in the region of interest based on the received feature maps,

- the object detection module trained to generate scribe data encoding the spatial layout of the scribe (s) present in the region of interest based on the received feature maps.
The method of claim 4, wherein the feature extractor includes a neural network trained to generate feature maps from the provided region of interest data.
The method of claim 5, wherein the feature maps include feature maps having a pyramidal hierarchy with respect to their resolution.
The method of any one of claims 4 to 6, wherein the semantic segmentation module includes a neural network trained to generate damage data encoding the spatial layout of damage area (s) present in the region of interest based on the feature maps generated by the feature extractor.
The method of claim 7, wherein the neural network is trained to increase the spatial resolution of the feature maps generated by the feature extractor, to pool the feature maps having increased spatial resolution and to transform the pooled feature maps into the damage data.
The method of any one of claims 4 to 8, wherein the object detection module includes a neural network trained to generate scribe data encoding the spatial layout of scribe (s) present in the region of interest based on the feature maps generated by the feature extractor.
The method of claim 9, wherein the neural network is trained to generate bounding box (es) around the scribe (s) based on the feature maps provided by the feature extractor and to generate scribe data including position coordinates of points lying on the scribe (s) and/or coordinates of the vertexes or intersections of scribe (s) .
The method of any one of claims 1 to 10, wherein determining the property data includes

- generating merged data by combining the damage data with the scribe data,

- determining the amount of damaged area and the amount of non-damaged area based on the merged data and

- determining property data associated with the at least one property based on the determined amount of damaged area and the determined amount of non-damaged area or based on the determined amount of damaged area and predefined evaluation data.
The method of any one of claims 1 to 11, wherein the data-driven model is trained to generate the damage data and the scribe data using at least one training data set including region of interest data, labelled region of interest data and/or labelled randomized region of interest data.
The method of any one of claims 1 to 12, further comprising a step of generating target scribe data based on the scribe data generated by the data-driven model, wherein the property data is determined based on the damage data generated by the data-driven model and the generated target scribe data.
A system for determining at least one property of a coating of a coated substrate, wherein the coating comprises one or more scribe (s) and wherein the coated substrate has been subjected to at least one chemical and/or mechanical stress test, the system comprising:

- an image acquisition unit configured to generate image data associated with the coated substrate;

- a data providing interface configured to provide the generated image data,

- a region of interest data generator configured to determine region of interest (ROI) data being indicative of the scribe (s) and optionally damage area (s) of the coating and/or substrate based on the provided image data;

- a neural network engine configured to process the region of interest data and including a data-driven model parameterized and/or trained to damage data associated with damaged area (s) present on the coating and/or the substrate and scribe data associated with the scribe (s) in response to being provided by the region of interest data;

- a property data determination unit configured to determine property data associated with the at least one property of the coating based on the damage data and scribe data generated by the data-driven model,

- a data providing interface configured to provide the property data associated with the at least one property of the coating.
Use of property data associated with a coating and being determined by the method of any one of claims 1 to 12 or by the system of claim 14 for adjusting a formulation of a coating material related to the coating.