GB2609969A

GB2609969A - System and method for inspecting an object

Info

Publication number: GB2609969A
Application number: GB2111915.1A
Authority: GB
Inventors: Ryan Louis
Original assignee: Ash Tech Ltd
Current assignee: Ash Tech Ltd
Priority date: 2021-08-19
Filing date: 2021-08-19
Publication date: 2023-02-22
Also published as: GB202111915D0; WO2023021209A1

Abstract

A system 100 for inspecting an object 102 includes a light source 104 for generating an input light pattern, an imaging device 106 for capturing an image of the object that includes an output light pattern, and a processing module 108 for analysing one or more output light patterns in one or more images of the object to identify a defect in the object. The object may be a glass vial or syringe which acts to refract light, while the defect could be a particle or bubble in the object. A machine learning model may be used to identify the defect.

Description

TITLE

SYSTEM AND METHOD FOR INSPECTING AN OBJECT

FIELD

The present disclosure relates to a system and method for inspecting an object using a light pattern, and more particularly to identifying a defect such as an immobile glass particle in glass vials and syringes using a light pattern.

BACKGROUND

In pharmaceutical industry glass vials and syringes are used to store and administer medicines, drugs, vaccines and the like to patients. Glass vials and syringes may break due to the presence of surface flaws and application of stress during tube forming, tube-to-container converting or pharmaceutical filling operations creating immobile glass particles and/or chips.

The immobile glass particles are small subvisible particles to large visible chips. The detection of immobile glass particles in liquid filled syringes or vials is a difficult problem to solve due to false negatives caused by dirt, air bubbles and fibers on or in the glass.

EP3312592A1 discloses a device that detects air bubbles and glass particles in a container filled with liquid by irradiating the container with light. However, the device employs at least two light sources which differ from one another in their spectral light color and/or their polarization. The use of multiple light sources that differ from one another in their optical properties is quite inefficient, cumbersome, expensive, and leads to generation of a large number of false negatives in detection of defects.

In view of the above, there is a need for a system that is compact, can be set up easily to detect defects such as immobile glass particles in liquid filled syringes or vials, and should provide a significant reduction in false negatives in detection of immobile glass particles.

SUMMARY

The present invention relates to a system and method for inspecting an object, as set out in the appended claims.

In one aspect, there is provided a system for inspecting an object. The system includes a light source for generating an input light pattern, the object for receiving the input light pattern, an imaging device for capturing an image of the object that includes an output light pattern, and a processing module for analysing one or more output light patterns in one or more images of the object to identify a defect in the object.

In an embodiment of the present invention, the light source is disposed at a left side of the object for backlighting the object with the input light pattern, and the imaging device is disposed at a right side of the object, such that the light source and the imaging device face each other, with the object in between.

In an embodiment of the present invention, when the object is one of: a glass vial and a syringe filled with a fluid, the object acts as a cylindrical convex lens that refract the input light pattern to generate the output light pattern.

In an embodiment of the present invention, the input light pattern includes a plurality of colours disposed in parallel with respect to each other, and each colour separated from corresponding adjacent colour by a black line.

In an embodiment of the present invention, the output light pattern is a clear inverted input light pattern with respective black lines, when the imaging device captures the image at a first focal point that is outside the object and is at a first distance from the imaging device.

In an embodiment of the present invention, the output light pattern is a blurred inverted input light pattern including a smooth pattern among adjacent colours without black lines, when the imaging device captures the image at a second focal point that is inside the object and is at a second distance from the imaging device.

In an embodiment of the present invention, the first and second distances are determined based on refractive indices of the object, and the fluid inside the object.

In an embodiment of the present invention, the processing module is configured to identify an immobile glass particle in the object as the defect, upon detecting a randomly dispersed irregularly shaped light in the smooth pattern, and wherein the randomly dispersed irregularly shaped light has a position which remains constant and a colour which changes across the one or more images obtained by rotating the object.

In an embodiment of the present invention, the processing module is configured to identify an air bubble in the object as the defect, upon detecting a miniature light pattern similar to the input light pattern in the output light pattern, and wherein the air bubble acts as a concave lens that receives the input light pattern and generates the miniature input light pattern at a left side of the air bubble.

In an embodiment of the present invention, the processing module includes an Artificial Intelligence (Al) based processing system that employs a machine learning model for identifying the defect in the object, wherein the machine learning model includes at least one of: a classification model, a temporal classification model, an object detection model, a temporal object detection model, and an anomaly detection model.

In an embodiment of the present invention, the light source projects the input light pattern on the object, and the imaging device captures one or more images of the projected light pattern on the object.

In an embodiment of the present invention, the system further includes a robotic arm attached to the object for rotating the object and enabling the imaging device to capture the one or more images of the object in one or more different orientations.

In an embodiment of the present invention, the object is configured to perform at least one of: reflect and refract the input light pattern.

In an embodiment of the present invention, the object includes one of: a glass syringe, a glass vial, a contact lens, a plastic tube, and a metal surface.

In an embodiment of the present invention, the defect includes at least one of: an immobile glass particle in the object, an air bubble, a dust particle, a fiber particle, a glass abrasion, and a deformation defect in the object.

In another aspect, there is provided a method for inspecting an object. The method includes generating an input light pattern by a light source, capturing an image of the object by an imaging device that includes an output light pattern, and analysing one or more output light patterns in one or more images of the object to identify a defect in the object.

In an embodiment of the present invention, the method further includes generating the output light pattern as a clear inverted input light pattern with respective black lines, upon capturing the image at a first focal point that is outside the object and is at a first distance from the imaging device.

In an embodiment of the present invention, the method further includes generating the output light pattern as a blurred inverted input light pattern including a smooth pattern among adjacent colours without black lines, upon capturing the image at a second focal point that is inside the object and is at a second distance from the imaging device.

In an embodiment of the present invention, the method further includes identifying an immobile glass particle in the object as the defect, upon detecting a randomly dispersed irregularly shaped light in the smooth pattern, and wherein the randomly dispersed irregularly shaped light has a position which remains constant and a colour which changes across the one or more images obtained by rotating the object.

In an embodiment of the present invention, the method further includes identifying an air bubble in the object as the defect, upon detecting a miniature light pattern similar to the input light pattern in the smooth pattern, and wherein the air bubble acts as a concave lens that receives the input light pattern and generates the miniature input light pattern at a left side of the air bubble.

In a third aspect, there is provided a non-transitory computer readable medium having stored thereon computer-executable instructions which, when executed by a processor, cause the processor to generate an input light pattern by a light source, capture an image of an object by an imaging device that includes an input light pattern, and analyse one or more output light patterns in one or more images of the object to identify a defect in the object.

There is also provided a computer program comprising program instructions for causing a computer program to carry out the above method which may be embodied on a record medium, carrier signal or read-only memory.

BRIEF DESCRIPTION OF DRAWINGS

The novel features of the present invention are set forth in the appended claims hereto. The subject matter itself, however, as well as a preferred mode of use, further objectives, and advantages thereof, will best be understood by reference to the following detailed description of illustrative embodiments when read in conjunction with the accompanying drawings and wherein: FIG.1 illustrates a system for inspecting an object, in accordance with an embodiment of the present invention; FIG.2 illustrates an input light pattern used for inspecting the object, and images including first and second output light patterns, in accordance with an embodiment of the present invention; FIG.3 illustrates various type of disruptions in the output light pattern caused by various defects in the object, in accordance with an embodiment of the present invention; FIGs.4A and 43 illustrate an image of light pattern taken by the imaging device at a focal point inside the object when the air bubble is present in the object, in accordance with an embodiment of the present invention; FIG. 5A illustrate a processing module for analysing the images captured by the imaging device to identify a defect in the object, in accordance with an embodiment of the present invention; FIG. 5B illustrates a histogram of the cross-validation tests performed for the processing module, in accordance with an embodiment of the present invention; FIG.5C illustrates first and second example model outputs generated based on glass syringe detection and non-glass syringe detection by the processing module, respectively, in accordance with an embodiment of the present invention; FIG. 6 illustrates another type of input light pattern that may be used for inspecting an object, in accordance with an embodiment of the present invention; FIG.7 is a flowchart illustrating a method for inspecting an object, in accordance with an embodiment of the present invention; and FIG.8 illustrates validation test results of inspection of various number of glass syringes/vials, in accordance with an embodiment of the present invention.

DETAILED DESCRIPTION OF INVENTION

FIG.1 illustrates a system 100 for inspecting an object 102, in accordance with an embodiment of the present invention. FIG.2 illustrates an input light pattern 202 used for inspecting the object 102, and images including first and second output light patterns 204 and 206.

The object 102 is typically any object that is capable of reflecting and/or refracting the input light pattern 202. Examples of the object 102 include but are not limited to: a glass vial filled with fluid, a glass syringe filled with fluid, a contact lens, a plastic tube, a metal surface. The inspecting the object 102 may include identifying a defect such as an immobile glass particle, an air bubble, a dust particle, a fiber particle, a glass abrasion, and a deformation defect in the object 102. For example, when the object 102 is a metal surface, the defect may include imperfections on the metal surface that reflect light. When the object 102 is a glass vial/syringe filled with fluid, the defect may include an immobile glass particle, an air bubbles and dirt in the fluid and also deformation (non-smooth) defect in the glass.

The input light pattern 202 includes a plurality of colours disposed in parallel with respect to each other, and each colour separated from corresponding adjacent colour by a black line. However, it would be apparent to one of ordinary skill in the art, that the input light pattern 202 is not limited to the one shown herein, it may include other custom light patterns based on the type of object and defect to be identified. In an example, when the object 102 includes a glass vial/syringe filled with fluid, the light source 104 generates a custom input light pattern based on optical properties of the glass and fluid to emphasize difference between glass particles, dirt, fibers and air bubbles.

The system 100 includes a light source 104 for generating the input light pattern 202, the object 102 for receiving the input light pattern 202, an imaging device 106 for capturing images of the object 102 that includes the first and second output light patterns 204 and 206, and a processing module 108 for analysing the output light patterns 204 and 206 to identify a defect in the object 106. The system 100 further includes a robotic arm 110 attached to the object 102 for rotating the object 102 and enabling the imaging device 106 to capture the one or more images of the object 102 in one or more different orientations, i.e. from all required angles. The imaging device 106 includes, for example, a charge coupled (CCD) camera.

As shown herein, the light source 104 is disposed at a left side of the object 102 for backlighting the object 102 with the input light pattern 202, and the imaging device 106 is disposed at a right side of the object 102, such that the light source 104 and the imaging device 106 face each other, with the object 102 is in between. However, it would be apparent to one of ordinary skill in the art, that the light source 104 and the imaging device 106 may be positioned in such a manner that the light source 104 projects the input light pattern 202 on the object 102, and the imaging device 106 captures one or more images of the projected light pattern to enable identification of defects by identifying disruptions in the projected light pattern.

B

FIG.2 illustrates the object 102 as one of: a glass vial and a syringe filled with a fluid and images including first and second output light patterns 204 and 206. The object 102 refracts the output light patterns 204 and 206 when there is no defect in the object 102.

In the plan view, the object 102 acts as a cylindrical convex lens that refract the input light pattern 202 and forms the output light patterns 204 and 206 at a right side thereof. The images including output light patterns 204 and 206 are captured by the imaging device 106 at two different focal points.

The imaging device 106 generates an image including the first output light pattern 204 when it focusses on an outside wall of the object at a distance D' from the imaging device 106. Thus, the imaging device 106 captures an image of the object 102 at a first focal point that is outside a cylindrical wall of the object 102. The output light pattern 204 is a clear inverted image of the input light pattern 202 with respective black lines.

The imaging device 106 generates an image including the second output light pattern 206 when it focusses on an inside wall of the object 102 at a distance 'F' from the imaging device 106. Thus, the imaging device 106 captures an image of the object 102 at a second focal point that is inside of a cylindrical wall of the object 102. The output light pattern 206 is an inverted blurred image of the input light pattern 202. The output light pattern 206 includes a smooth pattern among adjacent colours, in which the transitions between colours are relatively smooth without black lines. Thus, the glass syringes/vials (i.e. object 102) with no defects are imaged as a smooth surface with transitioning light colours. If there is any glass particle, dirt, fiber, or air bubble inside or at a surface of the glass vial/syringe (i.e. object 102), then the smooth pattern may be disrupted.

Further, it is to be noted that when the object 102 includes a glass syringe/glass vial filled with fluid, the first and second distances D' and 'F' are determined based on the refractive index of the glass of the glass syringe/vial, and the fluid contained inside the glass syringe/vial.

FIG.3 illustrates various type of disruptions in the output light pattern 300 caused by various defects in the object 102, i.e. glass syringes/vials filled with fluids. The output light pattern 300 is obtained by the imaging device 106 by focusing on an internal wall of the object 102.

When the object 102, i.e. the glass syringe/vial includes a defect such as an immobile glass particle 302, then the output light pattern 300 includes a randomly dispersed irregular shaped light at corresponding position of immobile glass particle 302. The position of the randomly dispersed irregularly shaped light in the output light pattern 300 remains constant but its colour changes across the images obtained by rotating the object 102. The glass particle 302 is irregular in shape and refract the light from the input light pattern in random ways. It is highlighted as it disperses the light randomly and its colour changes as it rotate. The unique light pattern response/signature for immobile glass particles enables the identification and differentiation with other contamination/particles.

When the object 102, i.e. the glass syringe/vial includes a defect such as dirt/fiber 308, then there is not much change in the output light pattern 300. The dirt and fibers undergo minimal changes as they are imaged while being rotated. The unique light pattern response/signature for dirt and fibers enables the identification and differentiation with other contamination/particles.

When the object 102, i.e. the glass syringe/vial includes a defect such as an air bubble 304, then the output light pattern 300 includes a miniature light pattern similar to the input light pattern at corresponding position of the air bubble in the object 102. Further, referring to FIGs.4A and 4B, the air bubble 304 near the edge of the syringe/vial 102 acts as a concave lens that generates a miniature light pattern 402 similar to the input light pattern 202 at a left side of the air bubble 304. The miniature light pattern 402 due to the air bubble 304 is in sharper focus than the light pattern 400 where there is no air bubble 304 and as the air bubble 304 acts as a concave lens, the miniature light pattern 402 is not inverted. The black lines in the miniature light pattern 402 are clearly visible in the air bubble 304 and as the miniature light pattern 402 is not inverted, the revealed bubble 304 is highlighted and easily identifiable. The air bubbles have a distinct pattern acting as optical lenses inverting the structured light pattern.

Referring FIGs.1 and 5A, the processing module 108 is an Artificial Intelligence (Al) based module that employs a machine learning model for processing the information captured by the imaging device 106 for identifying the defect in the object 102. The machine learning model includes at least one of: a classification model, a temporal classification model 502, an object detection model 504, a temporal object detection model, and an anomaly detection model 506. The Al based module 108 employs a single or ensemble of computer vision/machine learning/deep learning algorithms to accurately detect glass particles and exclude air bubbles, dirt and fibers. The ensemble modeling is a process where multiple diverse machine learning models may be combined to predict an outcome differentiating between air bubbles, dirt, fibers and immobile glass particles on internal surfaces of glass syringes/vials.

In the processing module 108, the output of the anomaly detection module 506, the object detection module 504, and the temporal classification module 502 is provided to a Support Vector Machine (SVM) classifier 508. The SVM classifier 508 is trained on the SVM Classifier Training data, wherein cross validation is used to get an average accuracy of the final ensemble model. A histogram (as shown in FIG.5B) of the cross-validation tests is shown opposite giving an average accuracy of 96%. The SVM may be finally trained on a complete SVM Classifier Training dataset prior to being used for testing on the validation dataset.

FIG.5C illustrates first and second example model outputs 510 and 512 generated based on glass syringe detection and non-glass syringe detection by the processing module 108 respectively. Both the first and second example model outputs 510 and 512 illustrates model probability for anomaly detection model, object detection model, and temporal object detection model of the processing module 108.

FIG 6 illustrates another type of input light pattern 600 that may be used for inspecting an object such as the object 102.

FIG.7 is a flowchart illustrating a method 700 for inspecting an object, in accordance with an embodiment of the present invention. At step 702, the method includes generating an input light pattern by a light source. At step 704, the method includes capturing an image of the object by an imaging device that includes an output light pattern. At step 706, the method includes analysing one or more output light patterns in one or more images of the object to identify the defect in the object.

Example

FIG.8 illustrates validation test results 800 of inspection of various number of glass syringes/vials by implementing the method and system of the present invention. It may be noted that the overall defect detection rate is found to be 96.5% (55 out of 57 syringes) with false positive rate as 0% (33 out of 33 syringes).

Although the present invention has been described with reference to specific embodiments, this description is not meant to be construed in a limiting sense. Various modifications of the disclosed embodiments, as well as alternate embodiments of the subject matter, will become apparent to persons skilled in the art upon reference to the description of the subject matter. It is therefore contemplated that such modifications can be made without departing from the spirit or scope of the present invention as defined.

The embodiments in the invention described with reference to the drawings comprise a computer apparatus and/or processes performed in a computer apparatus. However, the invention also extends to computer programs, particularly computer programs stored on or in a carrier adapted to bring the invention into practice. The program may be in the form of source code, object code, or a code intermediate source and object code, such as in partially compiled form or in any other form suitable for use in the implementation of the method according to the invention. The carrier may comprise a storage medium such as ROM, e.g. a memory stick or hard disk. The carrier may be an electrical or optical signal which may be transmitted via an electrical or an optical cable or by radio or other means.

In the specification the terms "comprise, comprises, comprised and comprising" or any variation thereof and the terms include, includes, included and including" or any variation thereof are considered to be totally interchangeable and they should all be afforded the widest possible interpretation and vice versa.

The invention is not limited to the embodiments hereinbefore described but may be varied in both construction and detail.

Claims

Claims 1. A system for inspecting an object, comprising: a light source for generating an input light pattern; the object for receiving the input light pattern; an imaging device for capturing an image of the object that includes an output light pattern; and a processing module for analysing one or more output light patterns in one or more images of the object to identify a defect in the object.
2. The system as claimed in any preceding claim, wherein the light source is disposed at a left side of the object for backlighting the object with the input light pattern, and the imaging device is disposed at a right side of the object, such that the light source and the imaging device face each other, with the object in between.
3. The system as claimed in any preceding claim, wherein when the object is one of: a glass vial and a syringe filled with a fluid, the object acts as a cylindrical convex lens that refract the input light pattern to generate the output light pattern.
4. The system as claimed in any preceding claim, wherein the input light pattern includes a plurality of colours disposed in parallel with respect to each other, and each colour separated from corresponding adjacent colour by a black line.
5. The system as claimed in claim 4, wherein the output light pattern is a clear inverted input light pattern with respective black lines, when the imaging device captures the image at a first focal point that is outside the object and is at a first distance from the imaging device.
6. The system as claimed in claim 4, wherein the output light pattern is a blurred inverted input light pattern including a smooth pattern among adjacent colors without black lines, when the imaging device captures the image at a second focal point that is inside the object and is at a second distance from the imaging device.
7. The system as claimed in claims 5 and 6, wherein the first and second distances are determined based on refractive indices of the object, and the fluid inside the object.
8. The system as claimed in claim 6, wherein the processing module is configured to identify an immobile glass particle in the object as the defect, upon detecting a randomly dispersed irregularly shaped light in the smooth pattern, and wherein the randomly dispersed irregularly shaped light has a position which remains constant and a colour which changes across the one or more images obtained by rotating the object.
9. The system as claimed in claim 6, wherein the processing module is configured to identify an air bubble in the object as the defect, upon detecting a miniature light pattern similar to the input light pattern in the smooth pattern, and wherein the air bubble acts as a concave lens that receives the input light pattern, and generates the miniature light pattern at a left side of the air bubble.
10. The system as claimed in claim 6, wherein the processing module is configured to identify a dirt particle and a fiber particle in the object as the defect, upon detecting a minimal change in the smooth pattern.
11. The system as claimed in any preceding claim, wherein the processing module includes an Artificial Intelligence (Al) based processing system that employs a machine learning model for identifying the defect in the object, wherein the machine learning model includes at least one of: a classification model, a temporal classification model, an object detection model, a temporal object detection model, and an anomaly detection model.
12. The system as claimed as claimed in any preceding claim, wherein the light source projects the input light pattern on the object, and the imaging device captures one or more images of the projected light pattern on the object.
13. The system as claimed in any preceding claim further comprising a robotic arm attached to the object for rotating the object and enabling the imaging device 40 to capture the one or more images of the object in one or more different orientations.
14. The system as claimed in any preceding claim, wherein the defect includes at least one of: an immobile glass particle in the object, an air bubble, a dust particle, a fiber particle, a glass abrasion, and a deformation defect in the object. 10
15. A method for inspecting an object, comprising: generating an input light pattern by a light source; capturing an image of the object by an imaging device that includes an output light pattern; and analysing one or more output light patterns in one or more images of the object to identify a defect in the object.
16. The method as claimed in claim 14, wherein the light source is disposed at a left side of the object for backlighting the object with the input light pattern, and the imaging device is disposed at a right side of the object, such that the light source and the imaging device face each other, with the object in between, and the input light pattern includes a plurality of colours disposed in parallel with respect to each other, and each colour separated from corresponding adjacent colour by a black line.
17. The method as claimed in claim 15 further comprising generating the output light pattern as a clear inverted input light pattern with respective black lines, upon capturing the image at a first focal point that is outside the object and is at a first distance from the imaging device.
18. The method as claimed in claim 15 further comprising generating the output light pattern as a blurred inverted input light pattern including a smooth pattern among adjacent colours without black lines, upon capturing the image at a second focal point that is inside the object and is at a second distance from the imaging device.
19. The method as claimed in claim 17 further comprising identifying an immobile glass particle in the object as the defect, upon detecting a randomly dispersed irregularly shaped light in the smooth pattern, and wherein the randomly dispersed irregularly shaped light has a position which remains constant and a colour which changes across the one or more images obtained by rotating the object.
20. The method as claimed in claim 17 further comprising identifying an air bubble in the object as the defect, upon detecting a miniature light pattern similar to the input light pattern in the smooth pattern, and wherein the air bubble acts as a concave lens that receives the input light pattern, and generates the miniature light pattern at a left side of the air bubble.
21. A non-transitory computer readable medium having stored thereon computer-executable instructions which, when executed by a processor, cause the processor to: generate an input light pattern by a light source; capture an image of an object by an imaging device that includes an output light pattern; and analyse one or more output light patterns in one or more images of the object to identify a defect in the object.