CN111626995B - Intelligent insert detection method and device for workpiece - Google Patents
Intelligent insert detection method and device for workpiece Download PDFInfo
- Publication number
- CN111626995B CN111626995B CN202010425679.4A CN202010425679A CN111626995B CN 111626995 B CN111626995 B CN 111626995B CN 202010425679 A CN202010425679 A CN 202010425679A CN 111626995 B CN111626995 B CN 111626995B
- Authority
- CN
- China
- Prior art keywords
- insert
- workpiece
- inserts
- detected
- combinations
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 68
- 238000013145 classification model Methods 0.000 claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 19
- 238000012549 training Methods 0.000 claims abstract description 7
- 238000013528 artificial neural network Methods 0.000 claims abstract description 5
- 238000004590 computer program Methods 0.000 claims description 8
- 238000012216 screening Methods 0.000 claims description 7
- 238000004519 manufacturing process Methods 0.000 claims description 4
- 230000007717 exclusion Effects 0.000 claims description 2
- 238000003860 storage Methods 0.000 claims description 2
- 238000001914 filtration Methods 0.000 claims 2
- 238000005520 cutting process Methods 0.000 claims 1
- 238000009434 installation Methods 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 abstract description 5
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 238000010191 image analysis Methods 0.000 abstract description 2
- 238000012795 verification Methods 0.000 abstract 1
- 238000001746 injection moulding Methods 0.000 description 10
- 238000002347 injection Methods 0.000 description 6
- 239000007924 injection Substances 0.000 description 6
- 238000007689 inspection Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- -1 polyethylene Polymers 0.000 description 2
- 239000004698 Polyethylene Substances 0.000 description 1
- 239000004743 Polypropylene Substances 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000011900 installation process Methods 0.000 description 1
- 239000003960 organic solvent Substances 0.000 description 1
- 229920000573 polyethylene Polymers 0.000 description 1
- 229920001155 polypropylene Polymers 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an intelligent insert detection method and device for a workpiece, wherein the method comprises the following steps: and identifying the positions and types of the inserts in the workpiece images of the workpiece to be detected by adopting an insert detection classification model generated by training a neural network based on a large number of images of the inserts to be detected, and making an insert data standard of the workpiece according to the types of the inserts and the position relation between the inserts when all the inserts on the workpiece are correctly installed, so as to identify whether the inserts on the workpiece are correctly installed. Wherein the workpiece insert data criteria are generated by the insert detection classification model in combination with manual verification. The scheme provided by the invention combines an image analysis technology and an artificial intelligence technology, realizes the installation and detection of the inserts on the automatic detection workpieces, saves labor, and greatly improves the efficiency compared with the traditional manual detection.
Description
Technical Field
The technical scheme provided by the invention relates to the field of automatic detection/inspection, and is used for replacing the traditional method that the inserts installed on the workpieces/accessories are required to be inspected and checked manually. In particular to an intelligent insert detection method and device for a workpiece.
Background
Noun parsing:
1. injection molding: refers to various injection molded products produced by injection molding machines, including various packages, parts, and the like. Is mainly prepared from polyethylene or polypropylene and a plurality of organic solvents. Short for the sake of brevity: a workpiece.
2. Insert: is a fitting inlaid on an injection molding piece and is used for fixing the injection molding piece or assembling a plurality of injection molding pieces.
3. Tool equipment: i.e., process equipment, refers to the generic term for the various tools used in the manufacturing process. Including tools/jigs/dies/gauges/inspection tools/assistive tools/pliers tools/station tools, etc.
In the automotive injection molding industry, inserts are required to be installed on many workpieces. The inserts are used as important component parts of injection molding pieces, the installation process is complex, the mechanical installation difficulty is high, the cost is high, and at present, the installation mainly takes place through a large amount of manpower, and the manual installation is caused by fatigue and other reasons and occurs when the assembly is wrong.
Missing or reverse mounting of the inserts, misattachment of the two-dimensional codes and the bar codes on the workpieces or incapability of identification can cause quality complaints and even claims of whole factories or assembly factories. Similar problems occur for primary workpieces, which can cause immeasurable losses. The method aims at the requirements of complaints and even loses the contract if the correction is not in place. With the increase of automobile types, the variety of workpieces can be more numerous, hundreds of workpieces can be produced even thousands of workpieces in one workshop every year, the used inserts are tens of workpieces, the difference of the workpieces is very large, the color, the size, the shape and the mounting position of each workpiece are different, and great working difficulty is brought to the injection molding industry. At present, manufacturers generally adopt manual work to perform spot inspection on inserts, two-dimensional codes and bar codes on workpieces. The finished workpiece is required to be manually inspected and marked, so that the labor intensity is high, the inspection efficiency is low, the quality is unstable, the labor cost is high, and even the labor is difficult to be incurred.
At present, although a plurality of large batches of workpieces can be automatically inspected based on a machine vision technology in a mode of combining special tools with traditional picture comparison. For example, publication number CN110632086a, entitled "method and system for detecting surface defects of injection molded parts based on machine vision", discloses the use of special tools to detect inserts installed on specific workpieces. However, since the inserts are various and correspond to a plurality of even tens of workpieces, the shooting positions and angles of the same workpiece are required to be fixed, and in order to ensure accuracy, shooting conditions (lighting and camera parameters) are required to be the same, and limiting conditions are very harsh. Every time a workpiece is replaced, a special tool is needed, and software development is repeated. The method has high cost and low efficiency, and particularly the development trend of the current automobile industry is small batch and multiple batches, so that the inspection method cannot meet the requirements of the automobile injection molding industry.
Disclosure of Invention
In order to solve the defects of high cost and low efficiency in the field of small-batch and multi-batch workpiece detection by adopting the traditional machine vision and poor applicability to workpieces to be detected. According to the invention, the insert detection classification model is trained by means of the deep neural network, and only corresponding insert data standards are required to be manufactured for different workpieces to be detected for comparison, so that hardware is not required to be changed.
Because of the same workpiece, the same surface is placed on the horizontal plane, and the distance and the angle between projections of the inserts on the horizontal plane are unchanged, no matter how the workpiece is placed, and no matter where the workpiece is placed. Therefore, as long as the inserts can be identified in the photos taken by the camera, whether the inserts are correctly installed can be accurately judged through the distance and the angle between the inserts and the type of the inserts.
The invention provides an intelligent insert detection method for a workpiece according to the principle, which comprises the following steps: the method comprises the steps that a deep neural network is adopted to train a large number of images of inserts to be detected to generate an insert detection classification model for identifying the positions and types of the inserts in workpiece images of the workpieces to be detected; manufacturing an insert data standard of the workpiece according to the type of the insert and the position relation among the inserts when all the inserts on the workpiece are correctly installed; based on the type and the positional relationship among each insert in the insert data standard of the workpiece and the recognition result of the insert detection classification model, it is determined whether the insert on the workpiece is properly mounted. In order to be able to shoot the image of the workpiece to be detected as comprehensively as possible, 3 cameras respectively positioned right above, left above and right above the workpiece placing platform are adopted to shoot the workpiece to be detected placed on the workpiece placing platform at the same time so as to obtain the workpiece image of the workpiece to be detected. Wherein the internal references of the 3 cameras are calibrated under the same coordinate system.
Further, the insert detection classification model is a YOLOv3 target detection model. The training process comprises the following steps: after marking the positions and the corresponding type information of all the inserts in the pictures for a large number of picture samples containing the inserts to be tested, intercepting the parts of the inserts to be tested from the picture samples, and then transmitting the parts of the inserts to be tested and the corresponding type information into a YOLOv3 target detection network together for training to obtain an insert detection classification model capable of identifying and classifying the inserts mounted on the workpiece to be tested. And the position of the insert to be detected is represented by a binding box.
Further, calling the 3 cameras through a camera interface, shooting standard workpieces on a workpiece placement platform, wherein all inserts are correctly installed on the standard workpieces, calling the insert detection classification model to identify positions and types of all inserts in pictures shot by the 3 cameras, respectively giving unique marks to each insert identified in the pictures shot by the 3 cameras by configuration personnel, and ensuring that the unique marks of the same insert in the pictures shot by the 3 cameras are consistent; calculating the coordinates of each insert identified by each picture in the same world coordinate system, and further obtaining the position relation of each insert in the standard workpiece relative to other inserts in the standard workpiece; storing the type of each insert, the corresponding unique number and the positional relationship between the unique number and other inserts in the standard workpiece as an insert data standard of the workpiece.
Correspondingly, for the workpiece images shot by the 3 cameras, all the inserts are detected and identified from the workpiece pictures shot by each camera by the insert detection classification model of the workpiece. And identifying all inserts in the pictures shot by each camera, and combining and arranging the insert marks in the insert data standard of the workpiece. Excluding all the above-mentioned permutation and combination according to the type of each insert label, the space distance between inserts with different labels and the angle formed between 3 inserts with different labels in the insert data standard of the workpiece so as to obtain a group of insert possibility combinations respectively corresponding to each camera; respectively calculating the positions of the same reference number insert in the above-mentioned various groups of insert possibility combinations in the same world coordinate system, and combining and screening the various groups of insert possibility combinations according to the principle that the world coordinates of the same reference number insert are the same or the difference of the world coordinates is within a certain range to obtain the candidate combination; and determining whether the insert on the workpiece to be tested is installed correctly according to the candidate combination.
Corresponding to the method, the invention also provides an intelligent insert detection device, which comprises: a memory, a processor, and a computer program stored on the memory and executable on the processor; the processor implements the intelligent insert detection method for the workpiece when executing the computer program.
Drawings
FIG. 1 is a schematic view ofSchematic diagram of a workpiece with all inserts properly installed.
Detailed Description
In order to make the technical problems, technical schemes and beneficial effects solved by the invention more clear, the invention is further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
In the field of automotive injection molding, injection molded parts as a workpiece are required to be inlaid with a variety of different inserts for fixing the workpiece or to assemble a plurality of injection molded parts. When the insert is properly installed on a work piece, as shown in fig. 1, the injection molded part as the work piece is required to be fitted with a plurality of different inserts 1 to 6 for fixing the work piece or assembling a plurality of injection molded parts.
The invention provides an intelligent insert detection method for a workpiece, which is used for detecting whether an insert on the workpiece is correctly installed. The method comprises the following steps: the method comprises the steps that a deep neural network is adopted to train a large number of images of inserts to be detected to generate an insert detection classification model for identifying the positions and types of the inserts in workpiece images of the workpieces to be detected; manufacturing an insert data standard of the workpiece based on the type of the insert and the positional relationship between the inserts when all the inserts on the workpiece are correctly installed; based on the type and the positional relationship among each insert in the insert data standard of the workpiece and the recognition result of the insert detection classification model, it is determined whether the insert on the workpiece is properly mounted.
Specifically, the YOLOv3 image target detection algorithm is a better target detection algorithm in recent years, and has high detection speed and accuracy meeting the requirements. In addition, the deep learning system has an open source deep learning framework, namely dark net, and is relatively simple to use. The invention adopts a YOLOv3 target detection model. The training process comprises the following steps: after marking the positions and the corresponding type information of all the inserts in the pictures for a large number of picture samples containing the inserts to be tested, intercepting the parts of the inserts to be tested from the picture samples, and then transmitting the parts of the inserts to be tested and the corresponding type information into a YOLOv3 target detection network together for training to obtain an insert detection classification model capable of identifying and classifying the inserts mounted on the workpiece to be tested. The position of the insert to be tested is a binding box with the characterizable position and the image size information. In order to capture the workpiece image of the workpiece to be detected as completely and comprehensively as possible, 3 cameras positioned right above, left above and right above the workpiece placement platform are used for simultaneously capturing the workpiece to be detected placed on the workpiece placement platform to obtain the workpiece image of the workpiece to be detected, wherein internal parameters of the 3 cameras are calibrated under the same coordinate system.
In addition, in order to identify whether the insert is correctly mounted on the workpiece to be tested by using the insert detection classification model, it is also necessary to make an insert data standard of the workpiece with respect to the type of all the inserts and the positional relationship between the inserts when all the inserts are correctly mounted on the workpiece. The standard of the insert data of the workpiece can call the 3 cameras through the camera interface and shoot the workpiece on the workpiece placement platform, which is provided with all inserts correctly; invoking the insert detection classification model to identify all insert positions and types in the pictures shot by the 3 cameras, and respectively endowing each insert identified in the pictures shot by the 3 cameras with a unique label by configuration personnel, wherein the unique label is used for identifying a specific insert in a computer and ensuring that the unique labels of the same insert in the pictures shot by the 3 cameras are consistent. Calculating the coordinates of each insert in the standard workpiece in a world coordinate system and the position relation between the coordinates and other inserts in the standard workpiece; storing the type of each insert, the corresponding unique number and the positional relationship between the unique number and other inserts in the standard workpiece as an insert data standard of the workpiece.
Further, the determining whether the insert on the workpiece is properly installed is implemented as: and screening candidate combinations from all possible combinations of the inserts identified in the workpiece image in spatial positions, and identifying whether the inserts on the workpiece are correctly installed according to the candidate combinations. Specifically, for the workpiece images captured by the 3 cameras, the insert is detected from the workpiece pictures captured by each camera using the insert detection classification model of the workpiece. And identifying all inserts in the pictures shot by each camera, and combining and arranging the insert marks in the insert data standard of the workpiece. The inserts for the standard work piece are assumed to be: a1, A2, … …, an. Assuming that the AI inference interface recognizes that the inserts in the pictures taken by the 3 cameras are respectively:
insert in camera 1 picture: C1Q1, C1Q2, … …, C1Qm;
insert in camera 2 picture: C2Q1, C2Q2, … …, C2Ql;
insert in camera 3 picture: C3Q1, C3Q2, … …, C3Qk.
Calculating the combination of insert possibilities in each camera picture, such as: calculating the combination of insert possibilities in the camera 1 picture: each insert may be one of A1, A2, … …, an, misrecognition. All the possible numbers are that m is selected from n element sets to be arranged and combined. For example, for the workpiece shown in fig. 1, if 5 inserts are identified in the image captured by the camera located directly above the workpiece placement table, the above identified 5 inserts are respectively set as any 5 inserts A1 to A6 in fig. 1, and as known from the permutation theory, there are 6×5×4×3×2=360 combinations. And then, performing exclusion screening on all the arrangement combinations according to the type of each insert label, the space distance between inserts with different labels and the angle formed between 3 inserts with different labels in the insert data standard of the workpiece so as to obtain a group of insert possibility combinations respectively corresponding to each camera. For example, for the above assumption, if C1Q1 is A2, but the insert type of C1Q1 is not the same as the insert type of A2, then this possibility may be eliminated. If C1Q1 is A2 and C1Q2 is A3, the world coordinate of C1Q1 can be calculated according to the known technique of monocular detection of the height of the camera in which C1Q1 is located and A2, and the world coordinate of C1Q2 can be calculated in the same way. This possibility can be excluded if the distances of C1Q1 and C1Q2 are not the same as the distances of A2 and A3. As another example, assuming that C1Q1 is A2, C1Q2 is A3, and C1Q3 is A4, if three angles of a triangle having C1Q1, C1Q2, and C1Q3 as vertices and three angles of a triangle having A2, A3, and A4 as vertices are different, this possibility can be eliminated as long as the degrees of one angle are different.
In order to integrate the detection and identification results of the workpiece images of the workpiece to be tested, which are shot by the three cameras, the above-mentioned possible combinations of the inserts corresponding to each camera are combined and screened to obtain candidate combinations for judging whether the inserts on the workpiece to be tested are correctly installed. Therefore, the positions of the inserts corresponding to the labels in the above-mentioned various groups of insert possibility combinations in the same world coordinate system are calculated according to the respective internal parameters, external parameters and the respective photographed standard workpiece images of the 3 cameras, and the candidate combinations are obtained by combining and screening the various groups of insert possibility combinations according to the principle that the world coordinates of the inserts with the same label are the same or the difference between the world coordinates is within a certain range. Internal parameters refer to intrinsic parameters of the camera such as image size, focal length, distortion and the like, are obtained by internal debugging and can be regarded as machine constants of equipment. The external parameter refers to the position relation between the camera and the plane of the detection platform, and is related to the installation mode of the equipment, the equipment can be obtained through debugging after being fixed, and if the equipment is fixed, the external parameter does not need to be modified. The camera calibration principle can be seen in various papers and books which exist in a large number in the prior art.
The combination calculation of the above-mentioned combinations of the possibility of each group of inserts may be performed by combining the possibility combinations of inserts in the workpiece images captured by two of the three cameras, and then combining the possibility combinations of inserts in the workpiece images captured by the remaining one camera. The specific combination process is as follows:
a. for the possible combinations corresponding to the two cameras to be combined, calculating the world coordinates of the same label inserts in the same world coordinate system in each possible combination;
b. selecting one possibility combination from all the possibility combinations corresponding to one of the two cameras one by one, and respectively forming a combination pair with each possibility combination corresponding to the other of the two cameras to carry out combination calculation: only the label of the insert appearing in the possible combination of a single camera is marked independently, and the labels in the two possible combinations of one merging pair appear in the rest, if the world coordinates calculated by the inserts corresponding to the same label are different or the distance difference exceeds a preset threshold range, the two possible combinations are not required to be merged, and the matching of the next merging pair is directly carried out; if the situation does not exist, combining the inserts with the same label in a combining pair, and temporarily retaining the label of the insert marked independently in the combined possibility combination; and for the possibility combination with the single marked insert marks after combination, calculating the position relation between the single marked insert marks and other inserts in the possibility combination after combination, and comparing the position relation calculated by the positions corresponding to all inserts in the possibility combination after combination in the insert data standard of the workpiece and the position calculated by the positions of other inserts in the possibility combination after combination to determine whether to exclude the possibility combination after combination.
Correspondingly, the determining whether the insert on the workpiece is correctly installed based on the type and the position relation among the inserts in the insert data standard of the workpiece and the identification result of the insert detection classification model can be specifically implemented as follows: if only one candidate combination exists, prompting that the insert on the workpiece is installed correctly; if the candidate combination does not exist, prompting that the insert on the workpiece is not installed correctly; and if the candidate combinations are more than two, prompting a user to adjust the placement pose of the current workpiece to be tested.
Further, in order to standardize that the workpiece to be measured is placed on the workpiece placement table, in order to prevent the workpiece to be measured from being shot in non-detection or the workpiece to be measured from being shot, a detection area can be defined on the workpiece placement table, and information outside the detection area can be ignored.
Corresponding to the method, the invention also provides an intelligent insert detection device, which comprises: a memory, a processor, and a computer program stored on the memory and executable on the processor; the processor implements the intelligent insert detection method for the workpiece when executing the computer program. Further, the present invention also provides a computer-readable storage medium storing a computer program comprising program instructions that, when executed by a processor, cause the processor to perform the above-described intelligent insert detection method for a workpiece.
The scheme provided by the invention combines an image analysis technology and an artificial intelligence technology, realizes the installation and detection of the inserts on the automatic detection workpieces, saves labor, greatly improves the efficiency compared with the traditional manual detection, and reduces the labor cost of enterprises.
Claims (10)
1. An intelligent insert detection method for a workpiece is characterized by comprising the following steps: the method comprises the steps that a deep neural network is adopted to train a large number of images of inserts to be detected to generate an insert detection classification model for identifying the positions and types of the inserts in workpiece images of the workpieces to be detected; the workpiece image of the workpiece to be detected is obtained by simultaneously shooting the workpiece to be detected placed on the workpiece placement platform by using 3 cameras respectively positioned above the workpiece placement platform at different angles, and internal references of the 3 cameras are calibrated under the same coordinate system;
manufacturing an insert data standard of the workpiece to be detected based on the type of the insert and the position relation among the inserts when all the inserts on the workpiece to be detected are correctly installed; the insert data standard for the workpiece to be inspected is generated by: invoking the 3 cameras through a camera interface, photographing standard workpieces on a workpiece placement platform, wherein all inserts are correctly installed on the standard workpieces, invoking the insert detection classification model to identify positions and types of all inserts in pictures photographed by the 3 cameras, respectively assigning unique labels to each insert identified in the pictures photographed by the 3 cameras by configuration personnel, and ensuring that the unique labels of the same insert in the pictures photographed by the 3 cameras are consistent; calculating the coordinates of each insert identified by each picture in the same world coordinate system, and further obtaining the position relation of each insert in the standard workpiece relative to other inserts in the standard workpiece; storing the type of each insert, the corresponding unique mark and the position relation between the unique mark and other inserts in the standard workpiece as an insert data standard of the workpiece to be detected;
and determining whether the inserts on the workpiece to be detected are correctly installed or not based on the types and the position relations among the inserts in the insert data standard and the identification results of the insert detection classification model.
2. The intelligent insert detection method according to claim 1, wherein the insert detection classification model is a YOLOv3 image target detection model.
3. The intelligent insert detection method according to claim 2, wherein the process of insert detection classification model training comprises: marking the positions of all the inserts in the pictures and the corresponding type information of a certain number of picture samples containing the inserts to be detected, cutting out parts of the inserts to be detected from the picture samples, and transmitting the parts of the inserts to be detected and the corresponding type information into a YOLOv3 target detection network together for training to obtain an insert detection classification model capable of identifying and classifying the inserts installed on the workpiece to be detected; and the position of the insert to be detected is represented by a binding box.
4. The intelligent insert detecting method according to claim 1, wherein the determining whether the insert is properly mounted on the workpiece to be detected is based on the type of each insert in the insert data standard and the positional relationship between each other and the recognition result of the insert detection classification model; the realization is as follows: all inserts respectively identified from the workpiece images shot by each of the 3 cameras are combined in spatial position, so that 3 groups of combined possibility are obtained; and combining and filtering the 3 groups of possible combinations to obtain candidate combinations, and identifying whether the insert on the workpiece is correctly installed according to the candidate combinations.
5. The intelligent insert detection method according to claim 4, wherein the combination of the spatial position possibilities of all inserts respectively identified from the workpiece images photographed by each of the 3 cameras specifically includes: identifying all inserts from the pictures shot by each camera by adopting the insert detection classification model of the workpiece, and carrying out combination arrangement according to the insert marks in the insert data standard of the workpiece; and then, performing exclusion screening on all the arrangement combinations according to the type of each insert label, the space distance between inserts with different labels and the angle formed between 3 inserts with different labels in the insert data standard of the workpiece so as to obtain a group of insert possibility combinations respectively corresponding to each camera.
6. The intelligent insert detection method according to claim 4 or 5, wherein the combining and filtering all possible combinations to obtain candidate combinations is achieved by: respectively calculating the positions of the same reference number insert in the above-mentioned various groups of insert possibility combinations in the same world coordinate system, and combining and screening the various groups of insert possibility combinations according to the principle that the world coordinates of the same reference number insert are the same or the difference of the world coordinates is within a certain range to obtain the candidate combination; and determining whether the insert on the workpiece to be detected is installed correctly according to the candidate combination.
7. The intelligent insert detection method according to claim 6, wherein the combining and screening the candidate combinations according to the principle that world coordinates of inserts with the same reference number are the same or different within a certain range is implemented as follows:
combining two groups of insert possibility combinations: combining each of the first set of possible combinations with each of the other 1 of the two sets one by one according to the insert number, retaining and individually marking the insert numbers that appear in only one of the possible combinations when combined; calculating coordinates of the inserts corresponding to each label in the combined combination under the same world coordinate system, and directly excluding the combined combination if the coordinates of the inserts with the same label are different or the distance difference exceeds a preset threshold range; for the insert marks with independent marks in the combined combination, calculating the position relation between the insert corresponding to the insert mark with independent marks and other inserts in the combined possibility combination, comparing the position relation between the insert corresponding to the insert mark with independent marks and other inserts in the workpiece insert data standard, and if the positions of the inserts are inconsistent, eliminating the combined possibility combination; and combining the two sets of insert possibility combinations, and then combining the two sets of insert possibility combinations with a third set of insert possibility combinations according to the sample to obtain the candidate combination.
8. The intelligent insert detection method according to any one of claims 4-5, 7, wherein identifying whether an insert on the workpiece is properly installed based on the candidate combination is specifically: if only one candidate combination exists, prompting that the insert on the workpiece is installed correctly; if the candidate combination does not exist, prompting that the insert on the workpiece is not installed correctly; and if the candidate combinations are more than two, prompting a user to adjust the placement pose of the current workpiece to be tested.
9. A smart inlay detection device, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the smart inlay detection method for a workpiece according to any of claims 1 to 8 when executing the computer program.
10. A storage medium storing a computer program comprising program instructions that when executed by a processor cause the processor to perform the intelligent insert detection method for a workpiece according to any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010425679.4A CN111626995B (en) | 2020-05-19 | 2020-05-19 | Intelligent insert detection method and device for workpiece |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010425679.4A CN111626995B (en) | 2020-05-19 | 2020-05-19 | Intelligent insert detection method and device for workpiece |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111626995A CN111626995A (en) | 2020-09-04 |
| CN111626995B true CN111626995B (en) | 2024-03-01 |
Family
ID=72258948
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010425679.4A Active CN111626995B (en) | 2020-05-19 | 2020-05-19 | Intelligent insert detection method and device for workpiece |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111626995B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112419208A (en) * | 2020-11-23 | 2021-02-26 | 泰兴市建设工程施工图审查服务中心 | Construction drawing review-based vector drawing compiling method and system |
| CN114273245A (en) * | 2021-11-10 | 2022-04-05 | 上海艾豚科技有限公司 | Continuous mixed detection method and system for multiple kinds of workpieces of automotive interior parts |
| CN114273261A (en) * | 2021-11-10 | 2022-04-05 | 上海艾豚科技有限公司 | Method and system for simultaneously detecting multiple workpiece kits of automotive interior parts |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018000731A1 (en) * | 2016-06-28 | 2018-01-04 | 华南理工大学 | Method for automatically detecting curved surface defect and device thereof |
| WO2018120460A1 (en) * | 2016-12-28 | 2018-07-05 | 平安科技（深圳）有限公司 | Image focal length detection method, apparatus and device, and computer-readable storage medium |
| CN109590952A (en) * | 2018-12-17 | 2019-04-09 | 嘉兴运达智能设备有限公司 | The intelligent detecting method and detection workbench of set technique buck plate |
| CN110136202A (en) * | 2019-05-21 | 2019-08-16 | 杭州电子科技大学 | A kind of multi-targets recognition and localization method based on SSD and dual camera |
| CN110207951A (en) * | 2019-05-23 | 2019-09-06 | 北京航空航天大学 | A kind of aircraft cable support assembled state detection method of view-based access control model |
| CN110580723A (en) * | 2019-07-05 | 2019-12-17 | 成都智明达电子股份有限公司 | method for carrying out accurate positioning by utilizing deep learning and computer vision |
| CN111007073A (en) * | 2019-12-23 | 2020-04-14 | 华中科技大学 | Method and system for online detection of part defects in additive manufacturing process |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8860760B2 (en) * | 2010-09-25 | 2014-10-14 | Teledyne Scientific & Imaging, Llc | Augmented reality (AR) system and method for tracking parts and visually cueing a user to identify and locate parts in a scene |
| JP7119606B2 (en) * | 2018-06-11 | 2022-08-17 | オムロン株式会社 | Measuring system and measuring method |
-
2020
- 2020-05-19 CN CN202010425679.4A patent/CN111626995B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018000731A1 (en) * | 2016-06-28 | 2018-01-04 | 华南理工大学 | Method for automatically detecting curved surface defect and device thereof |
| WO2018120460A1 (en) * | 2016-12-28 | 2018-07-05 | 平安科技（深圳）有限公司 | Image focal length detection method, apparatus and device, and computer-readable storage medium |
| CN109590952A (en) * | 2018-12-17 | 2019-04-09 | 嘉兴运达智能设备有限公司 | The intelligent detecting method and detection workbench of set technique buck plate |
| CN110136202A (en) * | 2019-05-21 | 2019-08-16 | 杭州电子科技大学 | A kind of multi-targets recognition and localization method based on SSD and dual camera |
| CN110207951A (en) * | 2019-05-23 | 2019-09-06 | 北京航空航天大学 | A kind of aircraft cable support assembled state detection method of view-based access control model |
| CN110580723A (en) * | 2019-07-05 | 2019-12-17 | 成都智明达电子股份有限公司 | method for carrying out accurate positioning by utilizing deep learning and computer vision |
| CN111007073A (en) * | 2019-12-23 | 2020-04-14 | 华中科技大学 | Method and system for online detection of part defects in additive manufacturing process |
Non-Patent Citations (2)
| Title |
|---|
| 赵耀霞 ; 吴桐 ; 韩焱 ; .基于卷积神经网络的复杂构件内部零件装配正确性识别.电子学报.2018,(08),全文. * |
| 魏中雨 ; 黄海松 ; 姚立国 ; .基于机器视觉和深度神经网络的零件装配检测.组合机床与自动化加工技术.2020,(03),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111626995A (en) | 2020-09-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111626995B (en) | Intelligent insert detection method and device for workpiece | |
| CN111537517B (en) | Unmanned intelligent stamping defect identification method | |
| EP3458919B1 (en) | Robot assisted object learning vision system | |
| CN107945184B (en) | Surface-mounted component detection method based on color image segmentation and gradient projection positioning | |
| US11132787B2 (en) | Method for monitoring manufacture of assembly units | |
| KR101910484B1 (en) | A method for three dimensional (3d) vision inspection | |
| CN106226325A (en) | A kind of seat surface defect detecting system based on machine vision and method thereof | |
| KR20180106856A (en) | Automatic optical inspection system and operating method thereof | |
| CN110108712A (en) | Multifunctional visual sense defect detecting system | |
| CN109840900A (en) | A kind of line detection system for failure and detection method applied to intelligence manufacture workshop | |
| CN105303573B (en) | The pin detection method and system of acupuncture needle class component | |
| CN105478363A (en) | Defective product detection and classification method and system based on three-dimensional figures | |
| CN114029946A (en) | Method, device and equipment for guiding robot to position and grab based on 3D grating | |
| CN115131268A (en) | Automatic welding system based on image feature extraction and three-dimensional model matching | |
| CN114529510B (en) | Automatic detection and classification method for cathode copper on-line quality | |
| CN113269234B (en) | Connecting piece assembly detection method and system based on target detection | |
| CN116563391B (en) | Automatic laser structure calibration method based on machine vision | |
| CN111486790A (en) | Full-size detection method and device for battery | |
| CN112109374A (en) | Method for positioning and controlling assembling and disassembling of bending die based on computer vision system | |
| CN116329738A (en) | Positioning method and device of circuit board target, electronic equipment and storage medium | |
| US10241000B2 (en) | Method for checking the position of characteristic points in light distributions | |
| CN107931828B (en) | One-key positioning method in laser processing, laser processing method and system | |
| CN113970560B (en) | Defect three-dimensional detection method based on multi-sensor fusion | |
| CN111015208A (en) | Automatic assembly detection device and method for charging equipment | |
| CN213179855U (en) | Full-size detection device for battery |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |