WO2020225377A1

WO2020225377A1 - Improved digital microscope

Info

Publication number: WO2020225377A1
Application number: PCT/EP2020/062746
Authority: WO
Inventors: Louis Ryan
Original assignee: Ash Technologies Ltd.,
Priority date: 2019-05-07
Filing date: 2020-05-07
Publication date: 2020-11-12
Also published as: GB201906371D0

Abstract

A digital imaging system comprising a camera, a lens and a screen to view an object. The system has a module which compares a sample image with an object image which positioned in the field of view of the digital imaging system. The sample image may be created by imaging a reference sample part using the digital image system and recording parameters of the sample image such as interest points and/or image reference lines and/or shapes and/or text. The module may then compare the sample image with the object image by aligning the sample image with the object image and determining differences between the sample image and the object image.

Description

Title

Improved Digital Microscope

Field

The disclosure relates to a digital microscope imaging system and method.

Background

Digital microscope imaging systems are well known art in the art. Digital microscopes can be used in the measurement, verification or validation in a quality assurance process across many industries.

An integrated full HD (1080p) digital microscope and measurement system is known in the art. Such a microscope can provide a custom platform that incorporates a highly configurable and upgradeable set of inspection and measurement applications such as focus stacking, 2D measurement, side by side and overlay image comparison.

In a typical example an image processing and control engine can deliver full HD live video image quality at 60 frames per second enabling it to be utilised as a solution in a broad range of quality control, testing, rework, assembly, inspection and documentation tasks. A Graphical User Interface can be a mouse controlled Interface delivering intuitive and efficient operation and control of a full application suite. Camera control functions (auto and manual focus, exposure, gain, brightness, white balance) can be provided.

Current digital microscopes suffer from a number of problems including: Lens distortion errors; calibration errors; or camera zoom and focus errors.

In addition, the measurement and comparison of different samples in industrial applications such as component inspection requires the samples to be correctly aligned to allow accurate and repeatable measurement. The known solution to this problem is to either manually align samples or use a jig to correctly position the sample. The former relies on the judgement of the user and is therefore subject to human error, the latter requires a new jig to be created for each different type of component to be inspected and is therefore expensive and time consuming. In addition, some image alignment is known, for example US 2019/0122913 A1 describes a method of image alignment in which areas of a test image and sample image are selected to assist with alignment.

Summary

In accordance with a first aspect of the invention there is provided, as set out in the appended claims, a digital microscope imaging system comprising: a camera; a lens and a screen to view an object, wherein the system comprises a module which compares a sample image with an object image which has been positioned in the field of view of the digital imaging system to determine differences between the sample image and the object image wherein, the sample image is created by imaging a reference sample part using the digital image system and recording parameters of the sample image and wherein, the module compares the sample image with the object image by presenting the sample image and the object image on the viewing screen alternately at a predetermined frequency.

In one embodiment, the recorded parameters comprise interest points and/or image reference lines and/or shapes and/or text.

In one embodiment, the module compares the sample image with the object image by aligning the sample image with the object image and determining differences between the sample image and the object image.

In one embodiment, the module compares the sample image with the object image by overlaying said images on the screen to provide a visual depiction of the differences. In one embodiment, the frequency is one image per second.

In one embodiment, the frequency is approximately two images per second, that is both the sample image and the object image are presented during a period of one second.

In one embodiment, the sample image and the object image are presented alternately during the entire time period.

In one embodiment the sample image and the object image are presented for substantially equal amounts of time through the period.

In one embodiment, the sample image is presented for longer than the object image.

In one embodiment, the object image is presented for longer than the sample image.

In one embodiment, the relative length of times for which the sample image and the object image are presented is controllable.

In one embodiment, differences between the sample image and the object image are displayed as flashing elements on the screen.

In one embodiment, the module aligns the images by:

detecting keypoints and descriptors between the sample image and the object image, matching the descriptors;

find the homography matrix;

translating and/or rotating the sample image to the object image’s position or translating and/or rotating the object image to the sample image’s position.

In one embodiment, the module aligns the images by template matching. In one embodiment, the module aligns the images by region based matching.

In one embodiment, the module aligns the images by deep learning matching.

In one embodiment, the module aligns the images by fast fourier transform (FFT) matching.

In one embodiment the object image is processed using dithering.

In one embodiment, the object image is processed using alpha blending.

There is also provided a computer program comprising program instructions for causing a computer program to carry out the above method which may be embodied on a record medium, carrier signal or read-only memory.

Brief Description of the Drawings

The invention will be more clearly understood from the following description of an embodiment thereof, given by way of example only, with reference to the accompanying drawings, in which:-

Figure 1 illustrates an example architecture of the imaging system and method according to one embodiment of the invention;

Figure 2 illustrates an example of using the present invention to compare a sample (master) image with an object (inspect) ; and

Figure 3 image illustrates an example method of aligning objects in accordance with the present invention.

Detailed Description of the Drawings

Figure 1 illustrates an example architecture of the imaging system according to one embodiment of the invention, indicated generally by the reference numeral 10. A monitor 1 1 displays an image to a user based on a captured image from a camera 12 and lens 13 combination. The image can be an object or unit under test 14 that the system can be configured to inspect. A processor 15 and storage module 16 to process and store data is provided. The imaging system can be hardware embodied as an integrated digital microscope such as a digital microscope sold under the trade name Omni’ by the applicants of the present invention. The processor 15 comprises hardware software and firmware which facilitates the operation of the digital microscope and a module which compares a sample image with an object image which have been positioned in the field of view of the digital imaging system. The sample image is created by imaging a reference sample part using the digital microscope and recording parameters of the sample image.

Figure 2 is an example of a method of using a digital microscope to inspect objects in accordance with the present invention when used to compare a known exemplar part with copy parts in order to ensure that the quality of the copy parts is sufficiently high; this requires a comparison between each of the copies and the exemplar. The process 20 creates a sample image from the exemplar part and the module in the processor records one or more parameter 24 which define(s) characteristics of the exemplar in the sample image. In this embodiment, the parameter is a set of reference lines and/or reference points which are added to the sample image of the exemplar part. Once this data is recorded, a copy object is viewed/imaged 26 and the images of the sample and object are aligned 28 using the reference lines and without the need for a jig. The sample image and the object image are compared 30 and any differences between them are determined.

Figure 3 image illustrates an example method of aligning objects in accordance with the present invention when using the following example of a method of using the system in accordance with the present invention.

In this example, the system is used for comparing multiple parts against an image of a known good part. The output result which illustrates the differences between the parts is that any differences between the two parts will appear to‘flash’ as the images are alternated on the monitor 11.

1. The‘good part’ is placed under the microscope and set the focal length by zooming to full magnification. Now adjust the height of the until the image is in focus.

2. The zoom and Illumination settings are adjusted to the required levels. 3. The user interface is used to open the measurement toolbar by clicking on the measurement icon. Reference lines, shapes and text are drawn on the image to create a sample image so that the next parts to be tested can be aligned for comparison. 4. The sample image with the reference lines, shapes and text is saved as a measurement file.

5. A sample part is placed under the microscope to create an object image and is aligned it with the measurement reference lines of the sample image. In this example of the present invention, alignment is achieved images by detecting key points and descriptors between the sample image and the object image, matching the descriptors finding a homography matrix, translating and/or rotating the sample image to the object image’s position or translating and/or rotating the object image to the sample image’s position. In other embodiments of the present invention the module aligns the images by template matching or region- based matching or by deep learning matching

6. A comparator icon on the user interface is selected to access the sample image file.

7. Images of the object image and the saved sample image are presented on the screen alternately, both being shown once per second . Any differences between the images will be perceived as‘flashing’ changes on what is viewed on the screen.

8. The comparator icon is selected to exit and the sample part is replaced with the next sample and repeat the process from step 5.

Figure 3 shows two sets of images. 44 a, b and c show the sample or“master” image and 46 d, e and f show object or“inspect” images. 44a is the sample image as taken and 46 d is the object image as taken. 44b is the sample image upon which reference points 36, 38, 40 and 42 have been selected. The equivalent reference points on the object image 46 e 36, 38, 40 and 42 are shown. Images 44c and 46f show lines drawn between corresponding reference points 36, 38, 40 and 42 on the images and figure 48g shows the images 44 and 46 overlayed.

The present invention removes the need for a Jig to align objects for comparison with a“good” sample or exemplar. This means that the time and cost of creating a new jig for each different type of component to be inspected is avoided. Another example of the method is described below.

1. Take image or load a previously saved image of sample / master (sample image)

2. Place object (using alpha blending / dithering) to be inspected and take image (object image)

3. Detect Keypoints and Descriptors between image-1 and image-2

4. Match the descriptors

5. Find the Flomography Matrix

6. Translate and rotate image-1 to image-2’s position Other Image Alignment methods include: Template matching; Region Based Matching; Deep learning matching and FFT Matching In examples of the present invention, Alpha Blending requires a display image with 50% pixel value from master image and 50% from live image. In examples of the present invention, dithering requires displaying the image with alternate pixel value from master image from live image.

The embodiments in the invention described with reference to the drawings comprise a computer apparatus and/or processes performed in a computer apparatus. However, the invention also extends to computer programs, particularly computer programs stored on or in a carrier adapted to bring the invention into practice. The program may be in the form of source code, object code, or a code intermediate source and object code, such as in partially compiled form or in any other form suitable for use in the implementation of the method according to the invention. The carrier may comprise a storage medium such as ROM, e.g. CD ROM, or magnetic recording medium, e.g. a memory stick or hard disk. The carrier may be an electrical or optical signal which may be transmitted via an electrical or an optical cable or by radio or other means.

It will be appreciated that in the context of the present invention that the term ‘graticule’ can be interpreted broadly and to mean ‘reticle’ as the terms are technically equivalent.

It will be further appreciated that the invention can be applied to testing objects in a range of industries, for example Medical Devices, Electronic Component testing and objects that require testing of Precision Engineering objects.

In the specification the terms "comprise, comprises, comprised and comprising" or any variation thereof and the terms include, includes, included and including" or any variation thereof are considered to be totally interchangeable and they should all be afforded the widest possible interpretation and vice versa.

The invention is not limited to the embodiments hereinbefore described but may be varied in both construction and detail.

Claims

1. A digital microscope imaging system comprising:

a camera;

a lens; and

a viewing screen for viewing digital images created by the camera and lens,

characterised in that: the system comprises a module which compares a sample image with an object image which has been positioned in the field of view of the digital imaging system to determine differences between the sample image and the object image wherein, the sample image is created by imaging a reference sample part using the digital image system and recording parameters of the sample image and wherein, the module compares the sample image with the object image by presenting the sample image and the object image on the viewing screen alternately at a predetermined frequency.

2. A system as claimed in claim 1 wherein, the recorded parameters comprise interest points and/or image reference lines and/or shapes and/or text.

3. A system as claimed in claim 1 or claim 2 wherein, the module compares the sample image with the object image by aligning the sample image with the object image and determining differences between the sample image and the object image.

4. A system as claimed in any preceding claim wherein, the module compares the sample image with the object image by overlaying said images on the viewing screen to provide a visual depiction of the differences.

5. A system as claimed in any preceding claim wherein, the predetermined frequency is one image per second.

6. A system as claimed in claims 1 to 4 wherein, the frequency is approximately two images per second, that is both the sample image and the object image are presented during a period of one second.

7. A system as claimed in claims 1 to 6 wherein, the sample image and the object image are presented alternately continuously during a time period.

8. A system as claimed in any preceding claim wherein, the sample image and the object image are presented for substantially equal amounts of time through the time period.

9. A system as claimed in claims 1 to 7 wherein, the sample image is presented for longer than the object image.

10. A system as claimed in claims 1 to 7 wherein, the object image is presented for longer than the sample image.

11. A system as claimed in any preceding claim wherein, the relative length of times for which the sample image and the object image are presented is controllable.

12. A system as claimed in any preceding claim wherein, differences between the sample image and the object image are displayed as flashing elements on the screen.

13. A system as claimed in any preceding claim wherein, the module is configured and aligns the images by: detecting keypoints and descriptors between the sample image and the object image,

matching the descriptors;

find the homography matrix;

14. A system as claimed in claims 1 to 12 wherein, the module aligns the images by template matching.

15. A system as claimed in claims 1 to 12 wherein, the module aligns the images by region-based matching.

16. A system as claimed in claims 1 to 12 wherein, the module aligns the images by deep learning matching

17. A system as claimed in claims 1 to 12 wherein, the module aligns the images by fast Fourier transform (FFT) matching.

18. A system as claimed in any preceding claim wherein, the object image is processed using dithering.

19. A system as claimed in any preceding claim wherein, the object image is processed using alpha blending.