CN112964724A - Multi-target multi-zone visual detection method and system - Google Patents
Multi-target multi-zone visual detection method and system Download PDFInfo
- Publication number
- CN112964724A CN112964724A CN202110139585.5A CN202110139585A CN112964724A CN 112964724 A CN112964724 A CN 112964724A CN 202110139585 A CN202110139585 A CN 202110139585A CN 112964724 A CN112964724 A CN 112964724A
- Authority
- CN
- China
- Prior art keywords
- detected
- product
- defect detection
- image
- products
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 138
- 230000000007 visual effect Effects 0.000 title description 9
- 230000007547 defect Effects 0.000 claims abstract description 116
- 238000000034 method Methods 0.000 claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 35
- 238000011179 visual inspection Methods 0.000 claims abstract description 15
- 239000000284 extract Substances 0.000 claims abstract description 6
- 238000007689 inspection Methods 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 10
- 238000001914 filtration Methods 0.000 claims description 6
- 238000003384 imaging method Methods 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 10
- 230000008859 change Effects 0.000 description 7
- 238000012937 correction Methods 0.000 description 6
- 238000000605 extraction Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 239000003292 glue Substances 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000032798 delamination Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000007788 liquid Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003647 oxidation Effects 0.000 description 1
- 238000007254 oxidation reaction Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/93—Detection standards; Calibrating baseline adjustment, drift correction
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8854—Grading and classifying of flaws
- G01N2021/8877—Proximity analysis, local statistics
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8854—Grading and classifying of flaws
- G01N2021/888—Marking defects
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
- G01N2021/9511—Optical elements other than lenses, e.g. mirrors
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
One or more embodiments of the present disclosure provide a multi-target multi-zone visual inspection method and system. The method comprises the following steps: the product bearing device moves the product to be detected to the image acquisition position of the image acquisition device; the image acquisition device scans and images a product to be detected to obtain an image to be detected of the product to be detected and sends the image to be detected to the image processing device; the image processing device extracts a plurality of areas of a plurality of products to be detected based on the images to be detected to obtain a plurality of different types of areas to be detected of the plurality of products to be detected, performs defect detection based on the plurality of different types of areas to be detected of the plurality of products to be detected respectively to obtain partial defect detection results of each area to be detected, generates complete defect detection results based on the partial defect detection results and sends the complete defect detection results to the upper computer; and the upper computer displays the complete defect detection result. The method and the system of the embodiment can realize defect detection of products with multiple detection targets.
Description
Technical Field
One or more embodiments of the present disclosure relate to the field of semiconductor inspection technologies, and in particular, to a multi-target multi-region visual inspection method and system.
Background
Display fingerprint sensing, also known as screen fingerprint, is an emerging function of smart phones. With the improvement of screen fingerprint sensing technology and the trend of mobile phones to the design of full screens, screen fingerprint technology is rapidly developed.
At present, the fingerprint module generally adopts the technical scheme of optical image capture and ultrasonic return. The fingerprint module belongs to high-precision complex manufacturing process, and various defects are easy to appear in the manufacturing process, so that the fingerprint module needs to be detected.
However, in the prior art, AOI vision in this field is currently low in popularity, and main equipment is strictly customized according to the existing process to match the current model and detection requirements, so that multi-target multi-defect detection cannot be realized.
Disclosure of Invention
In view of the above, one or more embodiments of the present disclosure are directed to a multi-target multi-zone visual inspection method and system, so as to solve the defect detection problem of a product with multiple inspection targets.
In view of the above, one or more embodiments of the present disclosure provide a multi-target multi-region visual inspection method, including:
the product bearing device moves the product to be detected to the image acquisition position of the image acquisition device;
the image acquisition device scans and images the product to be detected, obtains an image to be detected of the product to be detected and sends the image to the image processing device;
the image processing device extracts a plurality of areas of a plurality of products to be detected based on the images to be detected to obtain a plurality of different types of areas to be detected of the products to be detected, performs defect detection based on the different types of areas to be detected of the products to be detected respectively to obtain partial defect detection results of the areas to be detected, generates complete defect detection results based on the partial defect detection results and sends the complete defect detection results to an upper computer;
and the upper computer displays the complete defect detection result.
Optionally, the method further includes: and the image acquisition device simultaneously transmits the complete image of the current frame of product to be detected and shoots the complete image of the next frame of product to be detected.
Optionally, the image acquisition device acquires an entire plate of the target to be detected as the image to be detected, where the entire plate of the target to be detected includes one or more products to be detected.
Optionally, before the image acquisition device scans and images the product to be detected, the method further includes: and acquiring the whole plate mark of the whole plate to-be-detected target.
Optionally, the method further includes: acquiring an interested area in the whole plate target to be detected, wherein the interested area comprises all products to be detected;
and performing row and column division on the region of interest, and determining the product number of each product to be detected based on the position relationship between the row and column division result and the product to be detected.
Optionally, the method further includes:
acquiring interesting regions arranged in an array in the whole plate of targets to be detected, wherein each interesting region comprises one product to be detected;
and determining the product number of each product to be detected based on the position relation between the interesting area and the product to be detected.
Optionally, the method further includes:
the image processing device determines an area mark of the area to be detected of the product to be detected, and processes the partial defect detection result belonging to the same product to be detected based on the product number and the area mark to generate a product defect detection result of the product to be detected.
Optionally, the method further includes:
and the image processing device generates the complete defect detection result of the whole plate target to be detected based on the whole plate mark and all the product defect detection results belonging to the whole plate target to be detected.
Optionally, the performing defect detection on a plurality of different types of regions to be detected of a plurality of products to be detected to obtain partial defect detection results of each region to be detected includes:
acquiring a prestored defect detection template based on the whole plate mark and the area mark; the defect detection template comprises a reference image and a mask image of a standard product;
generating a mask image to be detected of the area to be detected based on the area to be detected and the defect detection template;
performing adaptive enhancement filtering based on the mask image to be detected to obtain a filtered image to be detected;
and performing local dynamic threshold division on the basis of the to-be-detected filtered image, and obtaining the partial defect detection result of the to-be-detected region on the basis of the result of the local dynamic threshold division.
One or more embodiments of the present disclosure provide a multi-target multi-region visual inspection system, which is used to implement any one of the above-mentioned multi-target multi-region visual inspection methods, and includes a product carrying device, an image acquisition device, an image processing device, and an upper computer; wherein,
the product carrier configured to: moving a product to be detected to an image acquisition position of an image acquisition device;
the image acquisition apparatus configured to: scanning and imaging the product to be detected to obtain a complete image of the product to be detected and sending the complete image to an image processing device;
the image processing apparatus configured to: extracting a plurality of areas of a plurality of products to be detected to obtain a plurality of different types of areas to be detected of the plurality of products to be detected, respectively carrying out defect detection on the plurality of different types of areas to be detected of the plurality of products to be detected to obtain partial defect detection results of each area to be detected, generating complete defect detection results based on the partial defect detection results, and sending the complete defect detection results to an upper computer;
the upper computer is configured to: and displaying the complete defect detection result.
As can be seen from the above, in the multi-target multi-region visual inspection method and system provided in one or more embodiments of the present disclosure, the image acquisition device is used to acquire the complete image of the product to be inspected, the image inspection device is used to perform region division on the complete image of the product to be inspected, and then the defect inspection is performed on the complete image of the product to be inspected, and the inspection results are summarized to generate complete defect inspection results, which are then displayed by the upper computer, so that a universal multi-target inspection method is provided; meanwhile, the defect detection and the motion control are dispersed, so that the resource contention caused by high concurrency and strong calculation force of the defect detection and the motion control is avoided, and the failure rate is reduced; the defect detection is realized by the image detection device, the bottom layer code of the upper computer does not need to be modified during the change, the visual defect detection part is not influenced, and the expansibility is good.
Drawings
In order to more clearly illustrate one or more embodiments or prior art solutions of the present specification, the drawings that are needed in the description of the embodiments or prior art will be briefly described below, and it is obvious that the drawings in the following description are only one or more embodiments of the present specification, and that other drawings may be obtained by those skilled in the art without inventive effort from these drawings.
FIG. 1 is a schematic flow diagram of a multi-target multi-zone visual inspection method according to one or more embodiments of the present disclosure;
FIG. 2 is a schematic diagram of an image acquisition process in one or more embodiments of the present disclosure;
FIG. 3 is a schematic diagram of multi-target multi-zone detection in accordance with one or more embodiments of the present disclosure;
FIG. 4 is a schematic diagram of region of interest extraction in accordance with one or more embodiments of the present disclosure;
FIG. 5 is a schematic diagram of another region of interest extraction in accordance with one or more embodiments of the present disclosure;
FIG. 6 is a schematic diagram of defect detection in accordance with one or more embodiments of the present disclosure;
FIG. 7 is a block diagram of a multi-target multi-zone visual inspection system in accordance with one or more embodiments of the present disclosure.
Detailed Description
For the purpose of promoting a better understanding of the objects, aspects and advantages of the present disclosure, reference is made to the following detailed description taken in conjunction with the accompanying drawings.
It is to be noted that unless otherwise defined, technical or scientific terms used in one or more embodiments of the present specification should have the ordinary meaning as understood by those of ordinary skill in the art to which this disclosure belongs. The use of "first," "second," and similar terms in one or more embodiments of the specification is not intended to indicate any order, quantity, or importance, but rather is used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", and the like are used merely to indicate relative positional relationships, and when the absolute position of the object being described is changed, the relative positional relationships may also be changed accordingly.
As described in the background section, the fingerprint module belongs to a high-precision complex process, which includes a plurality of structural regions such as a circuit region, a golden finger region, a Mark region, a lattice region, and an edge, and various defects are easily generated during the manufacturing process. Among them, the common defects include bubbles, scratches, liquid leakage, deviation, delamination, residual glue, glue overflow, surface dirt, edge breakage, protrusion, mark, concave-convex points, saw teeth, notch, oxidation, corrosion, etc. The detection precision requirement is high, and the defect detection is difficult.
However, the AOI vision devices provided for different process segments at present have very different forms, such as single-track type, double-track type, reciprocating type and straight-through type, different device models have completely different logic coordination on vision schemes, many devices are biased to customized development, complex interfaces, high interaction difficulty, and after sequential configuration, parameters are difficult to modify, and when a model change occurs or a new product or a new defect needs to be modified from a code layer again, the workload is high, and the device is easy to crash, so that the capacity loss is caused.
The applicant finds that the fingerprint module can be better detected if multi-target multi-region composite scene detection can be uniformly realized in the process of realizing the disclosure.
Hereinafter, the technical means of the present disclosure will be described in further detail with reference to specific examples.
One or more embodiments of the present disclosure provide a multi-target multi-zone visual inspection method, as shown in fig. 1, the inspection method including:
and S101, the product bearing device moves the product to be detected to the image acquisition position of the image acquisition device.
The product to be detected in the step is a detection product with high precision, small visual field, multiple targets, multiple regions and multiple specifications, which is applied in industries such as fingerprint modules, display chips, optical communication chips, Micro Electro Mechanical Systems (MEMS) chips, universal silicon-based chips and the like. The product bearing device is a six-axis robot, and under the control of the upper computer, the six-axis robot clamps the product to be detected and moves to the image acquisition position of the image acquisition device.
And S102, scanning and imaging the product to be detected by the image acquisition device to obtain a complete image of the product to be detected and sending the complete image to the image processing device.
In this step, the image acquisition device may be a line-scan camera or an area-scan camera, the line-scan camera performs continuous linear scanning imaging using an external trigger signal of the I/O card, the area-scan camera performs image acquisition by internal triggering of software, and image acquisition under different precision and light fields is realized by parallel acquisition of a plurality of line-scan cameras and area-scan cameras.
Step S103, the image processing device extracts a plurality of areas of a plurality of products to be detected to obtain a plurality of different types of areas to be detected of the plurality of products to be detected, detects defects of the plurality of different types of areas to be detected of the plurality of products to be detected respectively to obtain partial defect detection results of the areas to be detected, generates complete defect detection results based on the partial defect detection results and sends the complete defect detection results to an upper computer.
The image processing device is provided with an image detection system, an image detection algorithm is built in the image processing device, and the image to be detected can be used for detecting the defects of the product to be detected. The image processing apparatus may be an Industrial Personal Computer (IPC). In this embodiment, as shown in fig. 2, the image capturing device supports parallel multi-thread detection, and the image capturing device transmits an image to be detected to the image processing device each time it captures one frame of image, and the image processing device performs detection, that is, the image capturing device simultaneously performs transmission of the image to be detected in the current frame and shooting of the image to be detected in the next frame, so as to improve the detection efficiency and avoid mutual competition between resources.
In this step, as shown in fig. 3, after the complete image of the product to be detected is obtained, the image processing apparatus performs region extraction, so as to obtain a plurality of types of detection regions, such as a circuit region, a golden finger region, a Mark region, a lattice region, an edge, etc., of each product to be detected, perform defect detection on each different region, summarize detection results, and then draw the summarized detection results on a complete image to be detected, so as to facilitate the inspection of the detection personnel. Optionally, different colors are assigned to different defects for ease of viewing. The complete defect detection result includes detection process data and a final detection result, such as an original image, a defect thumbnail, defect information, and the like.
And S104, displaying the complete defect detection result by the upper computer.
In this step, the upper computer shows according to the show requirement that the system predetermines to supply the measurement personnel to look over. Meanwhile, the upper computer can provide tracing and associated display to the image based on the defect judgment result, realize switching preview of historical product detection results one by one, and display the image, operator execution attribute, judgment result, data, state, flow result information and the like belonging to the current product to be detected.
In the multi-target multi-region visual detection method provided by the embodiment of the specification, the complete image of the product to be detected is acquired through the image acquisition device, the complete image of the product to be detected is subjected to region division through the image detection device and then is subjected to defect detection, and partial detected defect detection results are summarized to generate complete defect detection results which are displayed through the upper computer, so that a universal multi-target detection method is provided; meanwhile, the defect detection and the motion control are dispersed, so that the resource contention caused by high concurrency and strong calculation force of the defect detection and the motion control is avoided, and the failure rate is reduced; the defect detection is realized by the image detection device, the bottom layer code of the upper computer does not need to be modified during the change, the visual defect detection part is not influenced, and the expansibility is good.
In some embodiments of the present disclosure, the product bearing device may simultaneously bear one or more targets to be detected, that is, one or more targets to be detected exist in a whole plate form, after the product bearing device takes out the whole plate targets to be detected from the tray and moves to the image capturing position of the image capturing device, the image capturing device scans the whole plate targets to be detected to obtain images to be detected. In some optional embodiments, before the image acquisition device scans and images the product to be detected, the method further includes: and acquiring the whole plate mark of the whole plate to-be-detected target.
In this embodiment, before the image to be detected is collected, the five-bit string on the product to be detected is read to obtain the whole-board mark of the whole-board object to be detected, and the whole-board mark is sent to the image collecting device, and the image collecting device can determine the type, model, and other characteristics of the product to be detected based on the whole-board mark. The whole-plate mark is included during subsequent region extraction and defect detection, so that a complete defect detection chain with multiple cameras, multiple states, multiple targets, multiple regions, multiple parameters, multiple specifications and multiple marks is realized, and finally all partial defect detection results belonging to the same whole-plate object to be detected can be provided for detection personnel to check based on whole-plate mark collection. Optionally, after the whole plate mark of the whole plate to-be-detected target is obtained, the row and column information data of the whole plate to-be-detected target is obtained.
Optionally, the position recognition points (Mark points) on the whole plate to-be-detected target are moved in the XYZ axis direction, so that the angle correction and the position correction of the whole plate to-be-detected target are realized, and the whole plate to-be-detected target can be shot by the image acquisition device. In this embodiment, when the overall pose of the product to be detected deviates, the image acquisition device determines the pose deviation amount of the product to be detected in the image to be detected based on the position identification point (Mark point) of the product to be detected and the pre-stored reference image, and then adjusts the position of the product bearing device based on the pose deviation amount, thereby achieving angle deviation correction and position correction of the product bearing device.
After the angle correction and the position correction are completed, the six-axis robot clamps the whole plate to-be-detected target and enables the whole plate to-be-detected target to pass through an image acquisition area of the image acquisition device at a constant speed, and therefore an image to be detected is obtained.
In the whole plate of targets to be detected, the number and the positions of the products to be detected may not be fixed, that is, when the positions of the products to be detected in different whole plate targets to be detected appear randomly and the number is also random, as shown in fig. 4, the image processing device acquires the region of interest in the whole plate targets to be detected, then performs row-column division on the region of interest ROI, and determines the product number of each product to be detected based on the position relationship between the result of the row-column division and the products to be detected. In this embodiment, the region of interest includes all the products to be detected, when the subsequent detection is performed based on the region of interest, the coordinate information of each row and each column in the region of interest can be obtained by dividing the region of interest into rows and columns, and each product to be detected is numbered based on the positional relationship between the coordinate information of each row and each column in the region of interest and the products to be detected.
In other optional embodiments, when the positions of the products to be detected in the whole target to be detected are regularly arranged, no matter whether the number of the products to be detected is fixed, the plurality of regions of interest which are regularly arranged can be obtained. As shown in fig. 5, the products to be detected are arrayed in the whole plate of the targets to be detected, at this time, the regions of interest arrayed in the whole plate of the targets to be detected are obtained, and the product number of each product to be detected is determined based on the position relationship between the regions of interest and the products to be detected; wherein each region of interest comprises one of the products to be detected. Thus, the position of each product to be detected can be determined based on the position of each region of interest, and each product to be detected is numbered.
Therefore, no matter whether the number of products to be detected in the whole target to be detected fluctuates and changes, whether the position fluctuates and changes, whether the space fluctuates and changes, or even whether the local features also change, the multi-target multi-region visual detection can extract the region of interest and then carry out defect detection.
In other optional embodiments of the present description, when the image processing device extracts multiple regions of multiple products to be detected based on the images to be detected to obtain the regions to be detected, the image processing device determines the region markers of the regions to be detected of the products to be detected at the same time, and the subsequent defect detection and detection results both include the region markers, so as to implement subsequent data processing. Namely, the image processing device processes the partial defect detection results belonging to the same product to be detected based on the product number and the area mark to generate a product defect detection result of the product to be detected. Meanwhile, the image processing device also generates the complete defect detection result of the whole plate target to be detected based on the whole plate mark and all the product defect detection results belonging to the whole plate target to be detected.
In the above embodiments, multiple ROI multi-parameter grouping sharing techniques may be employed to add multiple regions of interest (ROIs) based on their functionality. The added multiple interesting regions can be in different types such as rectangle, circle, polygon and the like, and are grouped, different groups are marked by different colors so as to distinguish different groups, and input parameters, attribute parameters and tolerance parameters belonging to the same grouped interesting region are uniformly configured, edited and modified without independently modifying the independent configuration; meanwhile, the parameters of the non-grouped interested areas can be independently set, so that the interested areas of different types can be more conveniently identified, the detection of multiple areas is realized, and the use and the checking by a user are facilitated.
Optionally, in step S103, the performing defect detection on the to-be-detected regions of the to-be-detected products based on different types to obtain partial defect detection results of each to-be-detected region includes:
step S201, acquiring a prestored defect detection template based on the whole plate mark and the area mark; the defect detection template comprises a reference image and a mask image of a standard product.
Step S202, generating a mask image to be detected of the area to be detected based on the area to be detected and the defect detection template, extracting edge threshold pixels of brightness change of an edge profile of the target area, filtering out interference objects through area, polarity and the like, smoothing the interference objects into a closed area until the interference objects are attached to the edge wrapping the actual target, and converting the closed area into the mask image.
Step S203, performing self-adaptive enhancement filtering based on the mask image to be detected to obtain a filtered image to be detected after filtering.
And S204, performing local dynamic threshold division on the basis of the to-be-detected filtered image, and obtaining the partial defect detection result of the to-be-detected region on the basis of the result of the local dynamic threshold division.
In this embodiment, a product image of a standard sample without defects is first acquired as a reference image and a Mask (Mask) image, and an image to be detected during subsequent detection is compared with the reference image and the Mask image, so as to determine the brightness and the color of a difference pixel with defects.
And during detection, based on the type of the defect detection template used by the whole plate mark determination and the area mark determination of the current area to be detected, calling the defect detection template corresponding to the area to be detected to detect the defect.
As shown in fig. 6, an original image of a region to be detected is first generated into a mask image to be detected based on a defect detection template. And then processing the mask image to be detected by adopting a self-adaptive enhanced filtering algorithm, dividing the mask image to be detected based on a dynamic threshold, judging whether the current region is a defect or not based on the result of threshold division, and marking the current region and determining the information such as classification, grade and the like of the defect if the current region is the defect.
It should be noted that the above description describes certain embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The multi-target multi-region visual detection method provided by the embodiment of the specification provides the capability of multi-target simultaneous detection, realizes the tracking precision of 1 pixel after dividing and extracting a target and performing ROI (region of interest) array, and allows the fluctuation change of the number, the position, the space and even the local characteristics of the target to change; marking the region of the ROI to realize that the defects of different regions adopt different definition modes and are drawn to a client through a characteristic diagram; when multiple ROIs and multiple parameters are used, a complete defect detection chain with multiple cameras, multiple states, multiple targets, multiple regions, multiple parameters, multiple specifications and multiple marks of the whole software can be opened, and the actual detection scene of a client site is matched better.
Based on the same inventive concept, corresponding to any of the above embodiments, one or more embodiments of the present disclosure further provide a multi-target multi-region visual inspection system, which is used for implementing the multi-target multi-region visual inspection method according to any of the above embodiments. As shown in fig. 7, the detection system includes a product carrying device, an image acquisition device, an image processing device and an upper computer; wherein,
the product carrier configured to: moving a product to be detected to an image acquisition position of an image acquisition device;
the image acquisition apparatus configured to: scanning and imaging the product to be detected to obtain a complete image of the product to be detected and sending the complete image to an image processing device;
the image processing apparatus configured to: extracting a plurality of areas of a plurality of products to be detected to obtain a plurality of different types of areas to be detected of the plurality of products to be detected, respectively carrying out defect detection on the plurality of different types of areas to be detected of the plurality of products to be detected to obtain partial defect detection results of each area to be detected, generating complete defect detection results based on the partial defect detection results, and sending the complete defect detection results to an upper computer;
the upper computer is configured to: and displaying the complete defect detection result.
The apparatus of the foregoing embodiment is used to implement the corresponding multi-target multi-region visual detection method in the foregoing embodiment, and has the beneficial effects of the corresponding method embodiment, which are not described herein again.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, is limited to these examples; within the spirit of the present disclosure, features from the above embodiments or from different embodiments may also be combined, steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present description as described above, which are not provided in detail for the sake of brevity.
In addition, well-known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown in the provided figures, for simplicity of illustration and discussion, and so as not to obscure one or more embodiments of the disclosure. Furthermore, devices may be shown in block diagram form in order to avoid obscuring the understanding of one or more embodiments of the present description, and this also takes into account the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform within which the one or more embodiments of the present description are to be implemented (i.e., specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the disclosure, it should be apparent to one skilled in the art that one or more embodiments of the disclosure can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative instead of restrictive.
While the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic ram (dram)) may use the discussed embodiments.
It is intended that the one or more embodiments of the present specification embrace all such alternatives, modifications and variations as fall within the broad scope of the appended claims. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of one or more embodiments of the present disclosure are intended to be included within the scope of the present disclosure.
Claims (10)
1. A multi-target multi-zone visual inspection method is characterized by comprising the following steps:
the product bearing device moves the product to be detected to the image acquisition position of the image acquisition device;
the image acquisition device scans and images the product to be detected, obtains an image to be detected of the product to be detected and sends the image to the image processing device;
the image processing device extracts a plurality of areas of a plurality of products to be detected based on the images to be detected to obtain a plurality of different types of areas to be detected of the products to be detected, performs defect detection based on the different types of areas to be detected of the products to be detected respectively to obtain partial defect detection results of the areas to be detected, generates complete defect detection results based on the partial defect detection results and sends the complete defect detection results to an upper computer;
and the upper computer displays the complete defect detection result.
2. The detection method according to claim 1, further comprising: and the image acquisition device simultaneously transmits the complete image of the current frame of product to be detected and shoots the complete image of the next frame of product to be detected.
3. The inspection method according to claim 1, wherein the image acquisition device acquires a whole plate of the object to be inspected including one or more of the products to be inspected as the image to be inspected.
4. The inspection method according to claim 3, wherein before the image acquisition device scans and images the product to be inspected, the method further comprises:
and acquiring the whole plate mark of the whole plate to-be-detected target.
5. The detection method according to claim 3, further comprising:
acquiring an interested area in the whole plate target to be detected, wherein the interested area comprises all products to be detected;
and performing row and column division on the region of interest, and determining the product number of each product to be detected based on the position relationship between the row and column division result and the product to be detected.
6. The detection method according to claim 3, further comprising:
acquiring interesting regions arranged in an array in the whole plate of targets to be detected, wherein each interesting region comprises one product to be detected;
and determining the product number of each product to be detected based on the position relation between the interesting area and the product to be detected.
7. The detection method according to claim 5 or 6, further comprising:
the image processing device determines an area mark of the area to be detected of the product to be detected, and processes the partial defect detection result belonging to the same product to be detected based on the product number and the area mark to generate a product defect detection result of the product to be detected.
8. The detection method according to claim 7, further comprising:
and the image processing device generates the complete defect detection result of the whole plate target to be detected based on the whole plate mark and all the product defect detection results belonging to the whole plate target to be detected.
9. The inspection method according to claim 7, wherein performing defect inspection based on a plurality of different types of the to-be-inspected regions of a plurality of the to-be-inspected products to obtain partial defect inspection results of each of the to-be-inspected regions comprises:
acquiring a prestored defect detection template based on the whole plate mark and the area mark; the defect detection template comprises a reference image and a mask image of a standard product;
generating a mask image to be detected of the area to be detected based on the area to be detected and the defect detection template;
performing adaptive enhancement filtering based on the mask image to be detected to obtain a filtered image to be detected;
and performing local dynamic threshold division on the basis of the to-be-detected filtered image, and obtaining the partial defect detection result of the to-be-detected region on the basis of the result of the local dynamic threshold division.
10. A multi-target multi-zone visual inspection system for implementing the multi-target multi-zone visual inspection method according to any one of claims 1 to 9, comprising a product carrying device, an image acquisition device, an image processing device and an upper computer; wherein,
the product carrier configured to: moving a product to be detected to an image acquisition position of an image acquisition device;
the image acquisition apparatus configured to: scanning and imaging the product to be detected to obtain a complete image of the product to be detected and sending the complete image to an image processing device;
the image processing apparatus configured to: extracting a plurality of areas of a plurality of products to be detected to obtain a plurality of different types of areas to be detected of the plurality of products to be detected, respectively carrying out defect detection on the plurality of different types of areas to be detected of the plurality of products to be detected to obtain partial defect detection results of each area to be detected, generating complete defect detection results based on the partial defect detection results, and sending the complete defect detection results to an upper computer;
the upper computer is configured to: and displaying the complete defect detection result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110139585.5A CN112964724B (en) | 2021-02-01 | 2021-02-01 | Multi-target multi-region visual detection method and detection system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110139585.5A CN112964724B (en) | 2021-02-01 | 2021-02-01 | Multi-target multi-region visual detection method and detection system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112964724A true CN112964724A (en) | 2021-06-15 |
| CN112964724B CN112964724B (en) | 2024-02-20 |
Family
ID=76273022
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110139585.5A Active CN112964724B (en) | 2021-02-01 | 2021-02-01 | Multi-target multi-region visual detection method and detection system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112964724B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114155367A (en) * | 2022-02-09 | 2022-03-08 | 北京阿丘科技有限公司 | Method, device and equipment for detecting defects of printed circuit board and storage medium |
| CN114965487A (en) * | 2022-06-10 | 2022-08-30 | 招商局重庆交通科研设计院有限公司 | Calibration method and device of automatic monitoring equipment for tunnel typical diseases |
| CN115690103A (en) * | 2022-12-30 | 2023-02-03 | 北京阿丘科技有限公司 | Product appearance detection method, device, equipment and storage medium |
| CN116309574A (en) * | 2023-05-19 | 2023-06-23 | 成都数之联科技股份有限公司 | Method, system, equipment and storage medium for detecting panel leakage process defects |
| CN116843602A (en) * | 2022-03-25 | 2023-10-03 | 广州镭晨智能装备科技有限公司 | Defect detection method and visual detection equipment |
| CN117911411A (en) * | 2024-03-19 | 2024-04-19 | 南京认知物联网研究院有限公司 | Computer vision detection method and device based on parallel detection of picture streams |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103413150A (en) * | 2013-06-28 | 2013-11-27 | 广东电网公司电力科学研究院 | Power line defect diagnosis method based on visible light image |
| CN203535336U (en) * | 2013-11-04 | 2014-04-09 | 北京兆维电子(集团)有限责任公司 | LCD (Liquid Crystal Display) detecting system |
| CN105301007A (en) * | 2015-12-02 | 2016-02-03 | 中国计量学院 | Linear array CCD-based ABS gear ring defect online detection device and method |
| CN105445277A (en) * | 2015-11-12 | 2016-03-30 | 湖北工业大学 | Visual and intelligent detection method for surface quality of FPC (Flexible Printed Circuit) |
| CN105510348A (en) * | 2015-12-31 | 2016-04-20 | 南京协辰电子科技有限公司 | Flaw detection method and device of printed circuit board and detection equipment |
| CN107402221A (en) * | 2017-08-08 | 2017-11-28 | 广东工业大学 | A kind of defects of display panel recognition methods and system based on machine vision |
| CN107966447A (en) * | 2017-11-14 | 2018-04-27 | 浙江大学 | A kind of Surface Flaw Detection method based on convolutional neural networks |
| CN108636820A (en) * | 2018-03-30 | 2018-10-12 | 中国科学院自动化研究所 | The precision component automatic detection of view-based access control model and sorting system and method |
| CN109521022A (en) * | 2019-01-23 | 2019-03-26 | 苏州鼎纳自动化技术有限公司 | Touch screen defect detecting device based on the confocal camera of line |
| CN110441319A (en) * | 2019-09-09 | 2019-11-12 | 凌云光技术集团有限责任公司 | A kind of detection method and device of open defect |
| CN111650205A (en) * | 2020-05-11 | 2020-09-11 | 东风汽车集团有限公司 | Part surface defect detection method and system based on structured light image matching |
| CN111928797A (en) * | 2020-10-12 | 2020-11-13 | 山东海德智汇智能装备有限公司 | 3D high-precision detection system based on laser scanning imaging |
| CN111968082A (en) * | 2020-07-30 | 2020-11-20 | 陕西科技大学 | Product packaging defect detection and identification method based on machine vision |
| CN111999308A (en) * | 2020-08-27 | 2020-11-27 | 银河水滴科技（北京）有限公司 | Defect detection control method, control device and control system |
-
2021
- 2021-02-01 CN CN202110139585.5A patent/CN112964724B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103413150A (en) * | 2013-06-28 | 2013-11-27 | 广东电网公司电力科学研究院 | Power line defect diagnosis method based on visible light image |
| CN203535336U (en) * | 2013-11-04 | 2014-04-09 | 北京兆维电子(集团)有限责任公司 | LCD (Liquid Crystal Display) detecting system |
| CN105445277A (en) * | 2015-11-12 | 2016-03-30 | 湖北工业大学 | Visual and intelligent detection method for surface quality of FPC (Flexible Printed Circuit) |
| CN105301007A (en) * | 2015-12-02 | 2016-02-03 | 中国计量学院 | Linear array CCD-based ABS gear ring defect online detection device and method |
| CN105510348A (en) * | 2015-12-31 | 2016-04-20 | 南京协辰电子科技有限公司 | Flaw detection method and device of printed circuit board and detection equipment |
| CN107402221A (en) * | 2017-08-08 | 2017-11-28 | 广东工业大学 | A kind of defects of display panel recognition methods and system based on machine vision |
| CN107966447A (en) * | 2017-11-14 | 2018-04-27 | 浙江大学 | A kind of Surface Flaw Detection method based on convolutional neural networks |
| CN108636820A (en) * | 2018-03-30 | 2018-10-12 | 中国科学院自动化研究所 | The precision component automatic detection of view-based access control model and sorting system and method |
| CN109521022A (en) * | 2019-01-23 | 2019-03-26 | 苏州鼎纳自动化技术有限公司 | Touch screen defect detecting device based on the confocal camera of line |
| CN110441319A (en) * | 2019-09-09 | 2019-11-12 | 凌云光技术集团有限责任公司 | A kind of detection method and device of open defect |
| CN111650205A (en) * | 2020-05-11 | 2020-09-11 | 东风汽车集团有限公司 | Part surface defect detection method and system based on structured light image matching |
| CN111968082A (en) * | 2020-07-30 | 2020-11-20 | 陕西科技大学 | Product packaging defect detection and identification method based on machine vision |
| CN111999308A (en) * | 2020-08-27 | 2020-11-27 | 银河水滴科技（北京）有限公司 | Defect detection control method, control device and control system |
| CN111928797A (en) * | 2020-10-12 | 2020-11-13 | 山东海德智汇智能装备有限公司 | 3D high-precision detection system based on laser scanning imaging |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114155367A (en) * | 2022-02-09 | 2022-03-08 | 北京阿丘科技有限公司 | Method, device and equipment for detecting defects of printed circuit board and storage medium |
| CN116843602A (en) * | 2022-03-25 | 2023-10-03 | 广州镭晨智能装备科技有限公司 | Defect detection method and visual detection equipment |
| CN116843602B (en) * | 2022-03-25 | 2024-05-14 | 广州镭晨智能装备科技有限公司 | Defect detection method and visual detection equipment |
| CN114965487A (en) * | 2022-06-10 | 2022-08-30 | 招商局重庆交通科研设计院有限公司 | Calibration method and device of automatic monitoring equipment for tunnel typical diseases |
| CN115690103A (en) * | 2022-12-30 | 2023-02-03 | 北京阿丘科技有限公司 | Product appearance detection method, device, equipment and storage medium |
| CN116309574A (en) * | 2023-05-19 | 2023-06-23 | 成都数之联科技股份有限公司 | Method, system, equipment and storage medium for detecting panel leakage process defects |
| CN116309574B (en) * | 2023-05-19 | 2023-08-18 | 成都数之联科技股份有限公司 | Method, system, equipment and storage medium for detecting panel leakage process defects |
| CN117911411A (en) * | 2024-03-19 | 2024-04-19 | 南京认知物联网研究院有限公司 | Computer vision detection method and device based on parallel detection of picture streams |
| CN117911411B (en) * | 2024-03-19 | 2024-05-24 | 南京认知物联网研究院有限公司 | Computer vision detection method and device based on parallel detection of picture streams |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112964724B (en) | 2024-02-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112964724A (en) | Multi-target multi-zone visual detection method and system | |
| CN107992881B (en) | Robot dynamic grabbing method and system | |
| CN101633356B (en) | System and method for detecting pedestrians | |
| EP3499414B1 (en) | Lightweight 3d vision camera with intelligent segmentation engine for machine vision and auto identification | |
| US20210233250A1 (en) | System and method for finding and classifying lines in an image with a vision system | |
| Hsieh et al. | A kinect-based people-flow counting system | |
| CN109753953A (en) | The method, apparatus of localization of text, electronic equipment and storage medium in image | |
| CN106407875A (en) | Target feature extraction method and apparatus | |
| JP2008046903A (en) | Apparatus and method for detecting number of objects | |
| US7415362B2 (en) | Image defect inspection apparatus | |
| CN105223208B (en) | A kind of circuit board detecting template and preparation method thereof, circuit board detecting method | |
| KR102009740B1 (en) | Apparatus for inspecting of display panel and method thereof | |
| US20190347530A1 (en) | Method and System for Identifying Targets in Scenes Shot by a Camera | |
| CN106228541A (en) | Screen positioning method and device in visual inspection | |
| Abdelhedi et al. | Design of automatic vision-based inspection system for monitoring in an olive oil bottling line | |
| CN104749801B (en) | High Precision Automatic optical detecting method and system | |
| US20240078801A1 (en) | System and method for finding and classifying lines in an image with a vision system | |
| Sun et al. | Cascaded detection method for surface defects of lead frame based on high-resolution detection images | |
| CN112183148A (en) | Batch bar code positioning method and identification system | |
| CN104966283A (en) | Imaging layered registering method | |
| Chen et al. | Defect detection of MicroLED with low distinction based on deep learning | |
| CN101685000B (en) | Computer system and method for image boundary scan | |
| CN114708234A (en) | Method and device for identifying number of detonators on automatic bayonet coding all-in-one machine | |
| CN102565084A (en) | Online inspection system and method for printed electronic product | |
| Chen et al. | EEE-Net: Efficient edge enhanced network for surface defect detection of glass |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |