TW202316369A - Storage and representation method of machine vision image and system thereof wherein the method includes displaying a non-region of interest image in black - Google Patents

Abstract

The present invention mainly discloses a storage and representation method of machine vision images and its system. The method includes capturing an image of an object to be measured; detecting a region of interest (ROI) and a region of non-interest (non-ROI) in the image of the object to be measured; retrieving an ROI image of the ROI; displaying the ROI image; and displaying a non-ROI image in black. A storage and representation system of the ROI image is used to allow a user to inspect the object to be measured, detect the ROI and the non-ROI in the image of the object to be measured and retrieve the ROI image from the ROI. The user can observe the object to be measured through an observation unit. A processing unit displays the non-ROI in black on the observation unit and the ROI image on the observation unit.

Description

Storage and representation method and system of machine vision image

The invention relates to the technical field of artificial intelligence, in particular to a method and system for storing and representing machine vision images that separately store images of regions of interest.

Machine vision inspection technology is based on image processing algorithm and realized by digital image processing and pattern recognition. Compared with traditional manual inspection technology, it improves the efficiency and accuracy of defect detection. Machine vision can cover all industrial and non-industrial applications, combining hardware and software to operationally guide devices to perform image capture and processing functions. Machine vision systems rely on digital sensors protected in industrial cameras, and use professional optical components to capture images for computer hardware and software to process, analyze, and measure various features to make decisions. Industrial applications such as printed circuit board assembly applications, in the assembly process, it is necessary to check whether the component number, size and component position of the printed circuit board are correct. Poor solder paste printing or wrong component placement will affect the performance and service life of printed circuit boards. PCB defect detection includes two parts: solder joint defect detection and component detection. Traditional detection adopts manual detection method, which is easy to miss detection, slow detection speed, long detection time and high cost, which has gradually failed to meet production needs. Therefore, it is of great practical significance to design an efficient and accurate machine vision PCB inspection system equipped with industrial cameras to replace human eyes.

In order to help printed circuit board manufacturers to detect defects in the early stages of production process implementation, more and more printed circuit board manufacturers are currently integrating online machine vision technology in the printing devices they manufacture. The operator can find defects with the assistance of the machine in time, and return to the production line for repair in time.

In the technical field of artificial intelligence, in machine vision and image processing, the area to be processed is outlined from the processed image in the form of a box, circle, ellipse, irregular polygon, etc., which is called the region of interest (region of interest). interested, ROI). In the field of image processing, a region of interest is an image region selected from an image that is the focus of your image analysis. With the help of the machine, the operator can quickly find the region of interest in the printed circuit board and assist in observing the defect image in the region of interest. Circle the area for further processing. Using a region of interest to delineate the target you want to read can reduce processing time and increase accuracy. There are traditionally two ways to represent the region of interest:

(1) Frame the position of the region of interest directly on the original image: as shown in FIG. 1 , which is a schematic diagram of an image 100 of the object to be measured. The machine captures the image 100 of the object under test, and assists in finding the image 102 of the region of interest and the image 104 of the region of non-interest that does not contain the region of interest. The system stores the image of the ROI 102 and the image of the non-ROI 104 outside the ROI 102 at the same time, so it occupies the largest storage space of the system.

(2) As shown in Figure 2, Figure 2 is another schematic diagram. The object image 200 includes: a ROI image 202 , a non-ROI image 204 and a black background image 206 . The machine captures the ROI image 202, and then pastes the ROI image 202 on a black background image 206 of the same size as the DUT image 200. Although the storage space of the method (2) is smaller than that of the method (1), the method (2) still needs a storage space for storing the pixel data of the black background image. The storage space occupied by the two traditional representation methods cannot be greatly reduced.

In view of this, the present invention provides a method and system for storing and representing machine vision images to solve the deficiencies of the prior art.

The first object of the present invention is to provide a method for storing and representing machine vision images, which includes: capturing the image of the object under test; detecting the region of interest and the region of non-interest in the image of the object under test; capturing Get the ROI image of the ROI; display the ROI image, and display the non-ROI image in black.

The second object of the present invention is based on the storage and representation method of the above-mentioned machine vision image, wherein the steps of displaying the image of the region of interest and displaying the region of non-interest in black include: storing the image of the region of interest; reading acquiring the ROI image and displaying the ROI image; and displaying the non-ROI image in black.

The third object of the present invention is based on the storage and representation method of the above-mentioned machine vision image, wherein the steps of displaying the image of the region of interest and displaying the region of non-interest in black include: storing the image of the region of interest; The non-interest region is displayed in black; and, the image of the region of interest is read and the image of the region of interest is displayed.

The fourth object of the present invention is based on the above-mentioned method for storing and representing machine vision images, wherein the image of the object under test includes exchangeable graphic file format information.

The fifth object of the present invention is based on the above-mentioned method for storing and representing machine vision images, wherein the exchangeable image file format information includes image information and manufacturer's notes.

The sixth object of the present invention is based on the above-mentioned method for storing and representing machine vision images, wherein the manufacturer's note includes image size information and image coordinate information.

In order to achieve the above purpose or other purposes, the present invention provides a machine vision image storage and representation system for providing a user to detect the object under test, including: an image capture unit for capturing the image of the object under test The image of the measured object; the processing unit is connected to the image capture unit to detect the region of interest and the region of non-interest in the image of the object to be measured, and capture the image of the region of interest of the region of interest; and, store a unit, connected to the processing unit, for storing the image of the region of interest; and an observation unit, connected to the processing unit, for viewing the region of interest of the object under test; wherein, the user observes the region of interest through the observation unit For the object under test, the processing unit displays the non-interest area in black on the observation unit, and displays the image of the interest area on the observation unit.

The second object of this aspect of the present invention is based on the above-mentioned storage and representation system for machine vision images, wherein the image of the object under test includes exchangeable graphic file format information.

The third object of this aspect of the present invention is according to the above machine vision image storage and representation system, wherein the exchangeable graphic format information includes image information and manufacturer's annotation.

The fourth object of this aspect of the present invention is according to the above-mentioned machine vision image storage and representation system, wherein the manufacturer note includes image size information and image coordinate information.

The fifth object of this aspect of the present invention is based on the above-mentioned machine vision image storage and representation system, wherein the image information includes shooting information, and the shooting information includes aperture value and shutter value.

Compared with the prior art, the present invention provides a method for storing and representing machine vision images and its system. With the inventive concept of the present invention, it can have the following advantages: it can mainly avoid the selection of training data for training in the prior art. Many problems, such as model jitter during training, are not conducive to the robustness of the model, unbalanced data, insufficient data, overfitting, etc.

In order to fully understand the purpose, features and effects of the present invention, the present invention will be described in detail through the following specific embodiments and accompanying drawings, as follows:

In the present disclosure, "a" or "an" is used to describe the elements, elements and components described herein. This is done for convenience of description only and to provide a general sense of the scope of the invention. Accordingly, unless it is obvious that it is otherwise indicated, such description should be read to include one, at least one, and the singular also includes the plural.

As used herein, the terms "comprises", "including", "has", "containing" or any other similar terms are intended to cover a non-exclusive inclusion. For example, an element, structure, article, or device that contains a plurality of elements is not limited to those elements listed herein, but may include elements that are not explicitly listed but are generally inherent in the element, structure, article, or apparatus. other requirements. In addition, unless expressly stated to the contrary, the word "or" means an inclusive "or" and not an exclusive "or".

Please refer to FIG. 3 , FIG. 4A and FIG. 4B , wherein FIG. 3 is a flowchart of a method for storing and representing machine vision images according to an embodiment of the present invention. Fig. 4A is a schematic diagram of the analyte of the present invention. FIG. 4B is a schematic diagram of an image of the object to be tested in the present invention.

As shown in FIG. 4A , the analyte O includes: a region of interest R ₁ and a region of non-interest R ₂ .

As shown in FIG. 4B , the object image V _o includes the ROI image V _R1 and the non-ROI image V _R2 .

In one embodiment of the present invention, the method for storing and representing the ROI image Vo includes: Step S31 to Step S38, as shown in FIG. 3 .

In step S31, the object image V _o of the object O is captured. As shown in Figure 4A and Figure 4B. The object to be tested in this embodiment is a wafer as an example, and circuits are fabricated on the wafer. It should be noted that the storage and representation method of the region of interest image V _R1 of the present invention can be applied to various objects to be measured, and is not limited to the wafer in this embodiment. The object to be measured can also be a printed circuit board, a three-dimensional The mechanical structure and circuit layout diagram.

Step S32, detecting the region of interest R ₁ and the region of non-interest R ₂ in the object image V _o . The DUT image V _o includes Exchangeable image file format (Exif). The Exchangeable Graphics Format information is a file format specially set for digital camera photos, which can record attribute information and shooting data of digital photos. Exchangeable picture file format information is appended to JPEG, TIFF, RIFF and other files to add content related to digital camera shooting information and index images or version information of image processing software. The Exchangeable Graphics Format information includes, for example, image information, manufacturer's notes, exposure time, aperture value, and image resolution. It should be noted that the manufacturer's note includes image size information and image coordinate information.

According to the captured image V _o of the object under test, the artificial intelligence technology is used to detect the edge of the defective area on the object under test O, and mark the defective area to become at least one region of interest R ₁ . Relative to the region of interest R ₁ of the object O, other regions are defined as regions of non-interest R ₂ . It should be noted that, assuming that the yield rate is higher than the normal standard, that is, the number of defects is relatively small, the area of interest R ₁ is generally much smaller than the non-interest area R ₂ . The number of ROIs R ₁ in the present invention is at least one. This embodiment only uses one ROI for illustration, and the scope of the present invention is not limited to one ROI in this embodiment.

Step S33, because we are more interested in the region of interest R ₁ , that is, the defect region. Therefore, the ROI image V _R1 of the ROI R ₁ is captured.

Step S34, storing the ROI image V _R1 .

Step S35 , read the ROI image _VR1 and display the ROI image _VR1 . .

In step S36, the non-interest region _R2 is displayed in black to become the non-interest region image V _R2 . Finally, the ROI image _VR1 and the black non-ROI image _VR2 can be displayed simultaneously.

In another embodiment, the present invention may also execute step S37 after step S34 to display the non-interest region in black. And, step S38 , read the ROI image V _R1 , and display the ROI image. Finally, the ROI image V _R1 and the black non-ROI image _VR2 can be displayed simultaneously, as shown in FIG. 4B .

Because, what we need is that, with the method of the present invention, only the image of the defective region (ie, the region of interest R ₁ ) needs to be displayed, and the non-region of interest R ₂ is displayed in black when it is not needed. By displaying the result, the image of the defective area (ie, the region of interest R ₁ ) can be highlighted, making it easier to view the image of the defective area.

In this embodiment, only one ROI image V _R1 of the ROI R ₁ needs to be stored. The area of interest R ₁ is much smaller than the object O to be measured. Therefore, compared to storing the entire object O image V _o , the file size of storing the image V _o of the object O is greatly reduced compared to storing the image V _R1 of the region of interest. The storage and representation method of the ROI image _VR1 of the present invention can reduce the storage space and reduce the calculation amount of the image processing.

FIG. 5 is a block diagram of a system for storing and displaying ROI images according to an embodiment of the present invention. Please refer to FIG. 5 , the ROI image storage and presentation system 500 includes an image capture unit 510 , a processing unit 520 , an observation unit 530 and a storage unit 540 . With the assistance of the region-of-interest image storage and representation system 500 , the user detects defects in the object O to be tested.

As shown in Figure 4A and Figure 4B. The object to be tested in this embodiment is a wafer as an example, and circuits are fabricated on the wafer. It should be noted that the storage and representation method of the region of interest image Vo of the present invention can be applied to various objects to be tested, and is not limited to the wafer in this embodiment. The object to be tested can also be a printed circuit board, a three-dimensional Mechanical structure, circuit layout diagram.

The image capture unit 510 is used for capturing the object image Vo of the object O. The DUT image Vo includes Exchangeable image file format (Exif). The Exchangeable Graphics Format information is a file format specially set for digital camera photos, which can record attribute information and shooting data of digital photos. Exchangeable picture file format information is appended to JPEG, TIFF, RIFF and other files to add content related to digital camera shooting information and index images or version information of image processing software. The Exchangeable Graphics Format information includes, for example, image information, manufacturer's notes on exposure time, aperture value, and image resolution. It should be noted that the manufacturer's note includes image size information and image coordinate information.

The image capture unit 510 of the present invention captures an image Vo of the object under test O. The processing unit 520 is connected to the image capturing unit 510 . The processing unit 520 detects the region of interest R ₁ and the region of non-interest R ₂ in the object image Vo, and captures the region of interest image V _R1 of the region of interest R ₁ . Because, we are more interested in the region of interest R ₁ , which is the defect region. Therefore, the processing unit 520 detects the ROI R ₁ in the object image Vo, and captures the ROI image V _R1 of the ROI R ₁ . The ROI image _VR1 is stored in the storage unit 540 . The processing unit 520 reads the ROI image _VR1 and displays the ROI image _VR1 in the observation unit 530 . The processing unit 520 displays the non-interest region R ₂ in black to become the non-interest region image V _R2 . Finally, the user can see the ROI image _VR1 and the black non-ROI image _VR2 simultaneously through the observation unit 530 , as shown in FIG. 4B .

When the user observes the object O through the observation unit 530, the processing unit 520 displays the non-interest region _R2 in black on the observation unit 530, and displays the region of interest image V _R1 on the observation unit 530, and finally Simultaneously, an image of the ROI image V _R1 in a black background is displayed, as shown in FIG. 4B . With the assistance of the observation observation unit 530, the user can easily find the defective region (ie, the region of interest R ₁ ) in the object under test O, and reduce the storage space, and additionally process the data calculation processing of the image of the non-interest region .

In another embodiment, in the present invention, the image capture unit 510 can also capture the object image Vo of the object O. The processing unit 520 is connected to the image capturing unit 510 . The processing unit 520 detects the region of interest R ₁ and the region of non-interest R ₂ in the object image Vo, and captures the region of interest image V _R1 of the region of interest R ₁ . Because, we are more interested in the region of interest R ₁ , that is, the defect region. Therefore, the processing unit 520 detects the ROI R ₁ in the object image Vo, and captures the ROI image V _R1 of the ROI R ₁ . The ROI image _VR1 is stored in the storage unit 540 . The processing unit 520 reads the ROI image _VR1 and displays the ROI image _VR1 in the observation unit 530 . The processing unit 520 displays the non-interest region R ₂ in black to form a black non-interest region image V _R2 . The non-interest region image _VR2 is stored in the storage unit 540 . The processing unit 520 reads the ROI image _VR1 and the non-ROI image _VR2 in the storage unit 540, and displays the ROI image. Finally, the ROI image V _R1 and the black non-ROI image _VR2 can be displayed simultaneously, as shown in FIG. 4B .

The present invention has been disclosed above with preferred embodiments, but those skilled in the art should understand that the embodiments are only used to describe the present invention, and should not be construed as limiting the scope of the present invention. It should be noted that all changes and substitutions equivalent to the embodiment should be included in the scope of the present invention. Therefore, the scope of protection of the present invention should be defined by the scope of the patent application.

S31, S32, S33, S34, S35, S36, S37, S38: step 100: object image 102: region of interest 104: non-interest region image 200: object image 202: region of interest image 204: non- Region of interest image 206: black background image 500: storage and representation system of machine vision image 510: image capture unit 520: processing unit 530: observation unit 540: storage unit O: object under test V _o : object under test image V _R1 : Region of interest image V _R2 : Region of non-interest image _R1 : Region of interest _R2 : Region of non-interest

Figure 1 is a schematic diagram of the image of the object under test; Figure 2 is a schematic diagram of another image of the object under test; 3 is a flow chart of a method for storing and representing machine vision images shown in an embodiment of the present invention; Fig. 4A is a schematic diagram of the object to be tested in the present invention; FIG. 4B is a schematic diagram of an image of the object to be tested in the present invention; FIG. 5 is a block diagram of a storage and presentation system for machine vision images shown in an embodiment of the present invention.

S31, S32, S33, S34, S35, S36, S37, S38: steps

Claims (10)

A method for storing and representing machine vision images, comprising: capturing an image of an object under test; detecting an area of interest and an area of non-interest in the image of the object under test; capturing an area of interest in the area of interest an image of the region; and displaying the image of the region of interest, and displaying the region of non-interest in black. The method for storing and representing machine vision images as described in claim 1, wherein, in the step of displaying the image of the region of interest and displaying the region of non-interest in black, comprising: storing the image of the region of interest; reading the image of the region of interest and displaying the ROI image; and displaying the non-ROI image in black. The method for storing and representing machine vision images as described in Claim 1, wherein, the step of displaying the image of the region of interest and displaying the region of non-interest in black includes: storing the image of the region of interest; The region of interest is displayed in black; and the image of the region of interest is read, and the image of the region of interest is displayed. The method for storing and representing machine vision images as described in claim 2 or 3, wherein the image of the object under test includes exchangeable graphic file format information. The method for storing and representing machine vision images as described in Claim 4, wherein the exchangeable image format information includes image information and manufacturer notes. The method for storing and representing machine vision images as described in Claim 5, wherein the manufacturer's notes include image size information and image coordinate information. A storage and representation system of machine vision image, used to provide a user to detect the object under test, comprising: an image capture unit, used to capture the image of the object under test of the object under test; a processing unit, connected to the image capture A unit, used to detect the region of interest and the region of non-interest in the image of the object under test, and capture the image of the region of interest of the region of interest; and, the storage unit, connected to the processing unit, for storing the sensor an image of a region of interest; and an observation unit connected to the processing unit for viewing the region of interest of the object to be measured; wherein, when the user observes the object to be measured through the observation unit, the processing unit will The region is displayed in black on the observation unit, and the ROI image is displayed on the observation unit. The storage and representation system for machine vision images as described in Claim 7, wherein the image of the object under test includes exchangeable graphic file format information. The storage and representation system for machine vision images as described in claim 8, wherein the exchangeable image file format information includes image information and manufacturer notes. The storage and representation system for machine vision images as described in Claim 9, wherein the manufacturer's notes include image size information and image coordinate information.

Images